Site Search

Find sitemap.xml, robots.txt, ads.txt, app-ads.txt and sellers.json - Real-Time Search

Not http https www subdomain

mikle1.livejournal.com robots.txt

mikle1.livejournal.com
Site: mikle1.livejournal.com

The robots.txt file is a file at the root a domain indicates parts of the site should not be accessed by search engine crawlers.

(*) Thumbnail Screenshots by Thumbshots



User-Agent: *
Disallow: /*.html*mode=reply
Disallow: /*.html*replyto
Disallow: /data/foaf/
Disallow: /tag/
Disallow: /friendstimes
Disallow: /d*
Disallow: /ratings/users

User-Agent: Mediapartners-Google*
Allow: /

User-Agent: Yandex
Allow: /
Disallow: /data/foaf/
Disallow: /ratings/users
Clean-param: nojs&tag&mode&thread
Host: https://mikle1.livejournal.com
Crawl-delay: 10

User-Agent: GoogleBot
Allow: /
Disallow: /ratings/users
Disallow: /data/foaf/

User-Agent: spbot
Disallow: /

User-agent: AhrefsBot
Disallow: /

Sitemap: https://mikle1.livejournal.com/sitemap.xml

              
Monday, 21 October 2019