Sitemap Generator not finding all pages
« on: July 30, 2010, 07:59:35 PM »
I just purchased the sitemap generator and ran the first crawl, it only found 1,082 pages and there are many more than that (ecommerce site with 30,000+ products). What to do?
Re: Sitemap Generator not finding all pages
« Reply #1 on: July 30, 2010, 10:01:36 PM »

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached from homepage?
Re: Sitemap Generator not finding all pages
« Reply #2 on: August 01, 2010, 04:55:08 PM »

I had the same problem. While watching the program run, I noticed that after it indexed the folder where my PHPBB files are located, it kept running through the list of all the remaining pages in other folders, but wouldn't index any of them. I then added the PHPBB folder to the do not index list, ran again, and it went from 982 pages to 6038 pages, which is about right.

The strange thing is that it still appeared to be indexing the PHPBB files. I would prefer to understand what what hapening, but with my busy schedule, I'll settle to just having it work anyway it takes.

BTW, it didn't matter whether or not including the pictures was selected when I was having indexing problems. It is definitely nice to have the images indexed now!

- Kirt Blattenberger