Find sitemap.xml, robots.txt, ads.txt, app-ads.txt and sellers.json - Real-Time Search
The robots.txt file is a file at the root a domain indicates parts of the site should not be accessed by search engine crawlers.
User-agent: *
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/cache
Disallow: /trackback
Disallow: /*.php$
Disallow: /*.inc$
Disallow: /*.gz$
Disallow: /wp-login.php
Sitemap: https://www-stage.rossel.be/sitemap_index.xml
