Sitemap Generator Forum

the Generator works fine. But after a while it is getting slowly. It is scrawling one page each 2 seconds.
It would take a while with 5000 pages :-((

If I close the browser and continue with it, it is working fast for the next appr. 30 seconds

Hello,

The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.

I did some finetuning. Now it is running in 2 minutes for 7000 pages.


thx