Sitemap Generator Forum

Hello I have an account with yours (pro), but the crawler was frozen on 10% thank you

Hello,

from what I see, crawling is currently in progress for your account:
--------
Time elapsed: 18:35:07, Pages processed: 78336 (78126 added in sitemap), Queued: 121884, Depth level: 4, Last updated: seconds ago
--------

Please note that crawler is paused from time to time to avoid sending too many requests to your website.

Yes thank you. But is too much slowly..... almost 3 days and now with 40%.... its normal?

Hello,

we limit crawler to have one bot thread per website to avoid sending too many requests to customers websites (and overloading them). We can increase it per request if needed though.

Yes please you can increase it? Now we have huge plan Up to 400,000 pages

thank you

Hello,

it has been increased now.

ok thank you