Crawling one page every 1 or 2 seconds

[william1]

Hi All

My crawls are taking forever and after looking at teh access logs i see that a page is being requested every 1 or 2 seconds. how can i speed this up?

Ww

[XML-Sitemaps Support]

Hello,

sitemap generator has a built-in crawler and it requests the page after the previous one has been processed, so in case if page load time is about 1 second, then that's a time period between requests.

[william1]

Thanks for the pointers, i will see what i can do to improve me site speed.

Is there a way of disabling this as my server has very little load on it at the moment.

Ww

[XML-Sitemaps Support]

That is the way generator uses to find all pages on the site, it cannot be disabled.