Find sitemap.xml, robots.txt, ads.txt, app-ads.txt and sellers.json - Real-Time Search
The robots.txt file is a file at the root a domain indicates parts of the site should not be accessed by search engine crawlers.
# This virtual robots.txt file was created by the Virtual Robots.txt WordPress plugin: https://www.wordpress.org/plugins/pc-robotstxt/
Sitemap: https://www.smartin.in/post-sitemap.xml
User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/
Disallow: /tag/
Disallow: /feed/
Disallow: /privacy/
Disallow: /go/
Allow: /wp-content/uploads/
Allow: /wp-content/cache/
User-agent: Mediapartners-Google
Allow: /
User-agent: Googlebot-Image
Allow: /wp-content/uploads/
User-agent: Adsbot-Google
Allow: /
User-agent: Googlebot-Mobile
Allow: /
