How does XML-sitemap generator identify itself in logs?
« on: January 27, 2009, 11:49:13 AM »
Hi

How does the online generator identify itself in traffic logs?  What user agent name does it use?  I want to make sure we don't block it in our robots.txt file, as we typically block unidientified spiders.

Thanks

Pete
Re: How does XML-sitemap generator identify itself in logs?
« Reply #1 on: January 27, 2009, 11:29:10 PM »
Hello,

sitemap generator User-agent header is:
Mozilla/5.0 (XML Sitemaps Generator 1.0)

It checks the disallow directives in robots.txt for googlebot and "User-agent: *".