# robots.txt # Usage # Create folder public_html/norobots/ # Insert all files that bots should not crawl # SpeedyCom.net Team User-agent: * Disallow: /norobots/