On your config page;

Set an interrupt of 60 seconds per 1000 requests.

Also set the maximum time to '0' or long enough to crawl your site with the included interrupts.

My crawl kept stopping near the end and the sitemap was not being created, once I included the interrupt the sitemap crawl and generation was completed. 12,000+ pages

Re: Try this if your sitemap crawl stops / hangs and does not complete
« Reply #1 on: June 18, 2007, 10:17:08 AM »

thank you for your comment, Steve.
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.