Site Search

Find sitemap.xml, robots.txt, ads.txt, app-ads.txt and sellers.json - Real-Time Search

Not http https www subdomain robots.txt

The robots.txt file is a file at the root a domain indicates parts of the site should not be accessed by search engine crawlers.

(*) Thumbnail Screenshots by Thumbshots

User-Agent: *
Disallow: /

User-Agent: Yandex
Disallow: /

User-Agent: GoogleBot
Disallow: /

User-Agent: TwitterBot
Disallow: /

User-Agent: spbot
Disallow: /

User-agent: AhrefsBot
Disallow: /

Wednesday, 21 April 2021