sitemap crawl stopped and won't restart
« on: August 04, 2008, 10:44:47 PM »
Hi,
Sitemap has run on cron succesfully for about 4 months. Last week it stopped and shows this message in the crawl section:


Continue the interrupted session (2008-07-27 04:46:01, URLs added: 3585, estimated URLs left: 14396)


The site has 3585 pages so was complete - don't know what the estimated 14396 are?

Anyway, if i restart the sitemap it appears to run but then fails. The above error message remains. Any ideas how to fix it please?

Chris
Re: sitemap crawl stopped and won't restart
« Reply #1 on: August 05, 2008, 04:46:12 PM »
Hello,

so the sitemaps generated earlier for your site had less than 4,000 URLs? Did you change something on your site recently?
Re: sitemap crawl stopped and won't restart
« Reply #2 on: August 05, 2008, 10:47:00 PM »
Nothing fundamental has changed on the site.

Previous sitemaps were all less than 3200 pages, however the site has moved on and now has over 4000 pages and is growing at a rate of about 250 pages per day at the minute.

I hope that helps to diagnose the problem.

Re: sitemap crawl stopped and won't restart
« Reply #3 on: August 06, 2008, 05:40:34 PM »
Don't you see any "new" URLs in the progress details when sitemap is generated that were not included earlier (or should not be included in sitemap).
Re: sitemap crawl stopped and won't restart
« Reply #4 on: August 06, 2008, 07:36:18 PM »
No, nothing has changed either with the restart point or the map.

I have PM'd you the login details so you can take a look.
Re: sitemap crawl stopped and won't restart
« Reply #5 on: August 07, 2008, 05:01:53 PM »
Update: sitemap has been generated successfully:
Request date:
7 August 2008, 11:47
Processing time:
1650.10s
Pages indexed:
5501

I've added "pjpeg" to the "Exclude extensions" list since there were links to images of *.pjpeg file type.
Re: sitemap crawl stopped and won't restart
« Reply #6 on: August 08, 2008, 08:47:59 PM »
Thank you. All working well thanks.