Guide to the robots.txt file - How to follow the Google webmaster guidelines
"Make use of the robots.txt file on your web server. This file tells crawlers which directories can or cannot be crawled. Make sure it's current for your site so that you don't accidentally block the Googlebot crawler." - from the Google webmaster guideli...
網址安全性掃描由 
提供
