Find sitemap.xml, robots.txt, ads.txt, app-ads.txt and sellers.json - Real-Time Search
The robots.txt file is a file at the root a domain indicates parts of the site should not be accessed by search engine crawlers.
User-agent: *
Allow: /.well-known/
Disallow: /File/
Disallow: /static/
Disallow: /design/
Disallow: /forum/
Crawl-delay:30
# Sitemap files
Sitemap: https://www.squarefoot.com.hk/sitemap.xml
