How to Set Up a robots.txt to Control Search Engine Spiders (thesitewizard.com)
The first block of text disallows all spiders from the images directory and the cgi-bin directory. The second block of code disallows the Googlebot-Image spider from every directory. It is possible to exclude a spider from indexing a particular file. For ...
網址安全性掃描由 
提供
