Site Search

Find sitemap.xml, robots.txt and ads.txt - Real-Time Search

Not http https www subdomain

weekendavisen.dk robots.txt

weekendavisen.dk
Site: weekendavisen.dk

The robots.txt file is a file at the root a domain indicates parts of the site should not be accessed by search engine crawlers.

(*) Thumbnail Screenshots by ShrinkTheWeb



# www.robotstxt.org/
# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449

User-agent: *
Crawl-delay: 10
Disallow: /api
Disallow: /sonata
Disallow: /reset_password
Disallow: /logout
Disallow: /login
Disallow: /profiloprettet
Disallow: /faa-adgang
Disallow: /buy
Disallow: /opener-reload-and-window-close
Disallow: /showMore
Disallow: /search

              
Monday, 10 December 2018