Not Indexing All Pages
« on: December 14, 2010, 08:00:07 PM »

I have been using this generator for a couple months now on 3 different sites, and I have noticed a really annoying trend. The generator is not very dependable with a cronjob because it usually doesn't index all of the pages. For example, one of my sites has 180 pages, and it only indexes 18. This is just about everyday, I wake up to the email notifying me that 18 pages were indexed, then I have to go to the generator page with my browser and start a manual crawl in order for it to index all 180 pages.

Another thing is the so-called "broken links". They are not broken when I check them.

Please advise. Thank you
Re: Not Indexing All Pages
« Reply #1 on: December 14, 2010, 10:08:05 PM »

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached starting from homepage?