help ! The crawler never finishes. Any idea why ?

[mpamphile]

I have over 100,000 pages on my website.  The crawler never finishes.  Any idea why ?

[admin]

Hello,

as described in Cannot complete sitemap generation - crawler stops after some time:

- increase max_execution_time setting in php.ini file at your server and restart apache
- execute sitemap generator from command line (if you have ssh access to your server)
- use "Save state" option at Sitemap Generator configuration page and execute the crawler multiple times with "Resume generation" enabled until full sitemap is created
- limit the number of pages to index in Sitemap Generator configuration

[bagros]

Maybe U can add <java script> and test, run again to end crawling?

[admin]

Thank you for suggestion, we will consider this option for the next versions of sitemap generator.