Sitemap does not work / takes extremely long time
« on: June 10, 2016, 08:07:14 AM »
Hi,

I have a sitemap + images on [ External links are visible to forum administrators only ]. It works fine when I ask a sitemap for 1000 pages, which I did to test. 1000 pages and 41000 images. However, when I tried the whole site (63500 pages and 125000 images), it did do nothing, even after 1,5 weeks (!!!) of running, even though I could not see if it actually did something.

As the 1000 page sitemap worked fine, it seems that the installation is OK. What to do ?
« Last Edit: June 10, 2016, 09:01:26 AM by harte9241 »
Re: Sitemap does not work / takes extremely long time
« Reply #1 on: June 10, 2016, 10:18:25 AM »
Hello,

The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.

If  1,5 weeks have passed then sitemap generator was stopped after some time and you need to run it again, resuming the process.
Re: Sitemap does not work / takes extremely long time
« Reply #2 on: June 15, 2016, 07:08:58 PM »
I tried several times to restart, but even 2500 pages does not work within a day. Why is that not working ? I need a good sitemap for my site, and this is getting useless...

[ External links are visible to forum administrators only ]
Re: Sitemap does not work / takes extremely long time
« Reply #3 on: June 16, 2016, 06:58:49 PM »
Looks like sitemap has been created:
Created on:
30 May 2016, 06:26
Processing time:
0:04:46s
Pages indexed:
1000
Re: Sitemap does not work / takes extremely long time
« Reply #4 on: June 16, 2016, 06:59:37 PM »
Also, you need to set "Save the script state, every X seconds:" setting to "30" (default) or "60", otherwise script state is not saved and you never resume it as a result.
Re: Sitemap does not work / takes extremely long time
« Reply #5 on: June 17, 2016, 06:18:50 AM »
The 1000 page sitemap works, but anything larger does not. I'll try the othe roption.