Crawling screen never refreshed

[pduhaut]

When I start the crawler, the screen seems to frozen after only few minutes... Does it mean the script is stopped? How can I know if the script is in run or not?

[XML-Sitemaps Support]

Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.

[info204]

I am having the same problem, what should these variables be set to? I have a site with almost 50 000 pages

[XML-Sitemaps Support]

Hello,

max_execution_time depends on the page generation time of your site mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page on your site, then 1000 pages will be crawled in about 16 minutes (and max_execution_time should be at least 1000).

[mcany]

Hi,

I tried to put the memory at 500MB, crawl to max level of 3 (8000 pages)  and put a delay of 1s between every request
after 1 or 2 mn it freeze

Links depth: 2
Current page: french-alps/isere
Pages added to sitemap: 278
Pages scanned: 320 (10,179.3 KB)
Pages left: 1405 (+ 513 queued for the next depth level)
Time passed: 1:15
Time left: 5:30
Memory usage: 1,498.2 Kb

I don't think the problem is coming from the memory, can you help me please?

Thanks

Mat

[XML-Sitemaps Support]

There should be no issues with memory limit with this number of links. Please PM me your generator URL/login so that I can check it.