驗證網址www.robotstxt.org安全性

robots.txt

The Web Robots Pages Web Robots (also known as Web Wanderers, Crawlers, or Spiders), are programs that traverse the Web automatically. Search engines such as Google use them to index the web content, spammers use them to scan for email addresses, and ......

網址安全性掃描由 google 提供