On your config page;

Set an interrupt of 60 seconds per 1000 requests.

Also set the maximum time to '0' or long enough to crawl your site with the included interrupts.


My crawl kept stopping near the end and the sitemap was not being created, once I included the interrupt the sitemap crawl and generation was completed. 12,000+ pages

Steve