Site Search

Find sitemap.xml, robots.txt, ads.txt, app-ads.txt and sellers.json - Real-Time Search

Not http https www subdomain

musicomh.com robots.txt

musicomh.com
Site: musicomh.com

The robots.txt file is a file at the root a domain indicates parts of the site should not be accessed by search engine crawlers.

(*) Thumbnail Screenshots by ShrinkTheWeb



User-agent: *
Crawl-delay: 60

User-agent: *
Disallow: /cgi-bin/

User-Agent: Googlebot
Crawl-delay: 20

User-Agent: Googlebot-Mobile
Crawl-delay: 20

User-Agent: Mediapartners-Google
Crawl-delay: 20

User-Agent: Slurp
Crawl-delay: 20

User-agent: msnbot
Crawl-delay: 20

User-Agent: msnbot-media
Crawl-delay: 20

User-Agent: Teoma
Crawl-delay: 20

              
Thursday, 5 December 2019