XML Sitemaps Generator

Author Topic: Pages are not added to site.  (Read 17966 times)


  • Registered Customer
  • Approved member
  • *
  • Posts: 1
Pages are not added to site.
« on: May 19, 2008, 08:05:22 AM »

I'm having some problems crawling my site ([external links are visible to admins only])
There are some links that are not added to the sitemap although they are finding out by the crawl process.
Also when I run the Free Online Sitemap Generator those links are added to sitemap correctly.
All those links located in ArtistPage/.
This is example of links that was found by the crawler but was not added to the sitemap:

Links depth: 2
Current page: ArtistPage/Dr._Rich_Ragle.aspx
Pages added to sitemap: 102
Pages scanned: 400 (3,574.4 KB)
Pages left: 544 (+ 548 queued for the next depth level)
Time passed: 1:51
Time left: 2:31
Memory usage: 1,156.3 Kb

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10599
Re: Pages are not added to site.
« Reply #1 on: May 19, 2008, 10:31:57 PM »

try to set xs_crawl_ident option in generator/config.inc.php file like:
Code: [Select]
'xs_crawl_ident' => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008051206 Firefox/3.0',
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.


SMF 2.0.12 | SMF © 2014, Simple Machines