Sitemap Generator Forum

Hi,
since I've updated from 6.0 to 6.1, the process became much slower, do you know why could be the cause of this?
My server runs the crawling process through cronjobs as you recommend, it does it once a day, but now it never finishes, every day it says that it continued from latest session, but it never ends or it delays several days. If I run it manually it does all the job, but delays much more than in previous version.
Why could this be happening? did you add some funcionality in 6.1 that I can disable so it returns to normal speed?
Thanks
Sebastian - [ External links are visible to forum administrators only ]

Hello,

The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.

Hi, thanks for your reply. This is what is happening if I run it through cronjobs. It takes a lot of times to end, somehow the session ends before it is finished, so next time it continues and the return emails are something like this:

Content-type: text/html

Resuming the last session (last updated: 2014-01-21 09:09:46)
681 | 565 | 38,928.8 | 0:09:48 | 0:08:08 | 2 | 3,733.3 Kb | 575 | 656 | 3733
682 | 564 | 39,003.8 | 0:09:49 | 0:08:07 | 2 | 3,744.3 Kb | 576 | 658 | 11
683 | 563 | 39,091.6 | 0:09:50 | 0:08:06 | 2 | 3,780.3 Kb | 577 | 659 | 36
684 | 562 | 39,167.1 | 0:09:51 | 0:08:06 | 2 | 3,748.6 Kb | 578 | 660 | -32
685 | 561 | 39,232.9 | 0:09:52 | 0:08:05 | 2 | 3,714.4 Kb | 579 | 661 | -34
686 | 560 | 39,323.3 | 0:09:53 | 0:08:04 | 2 | 3,802.8 Kb | 580 | 662 | 88
687 | 559 | 39,418.7 | 0:09:54 | 0:08:03 | 2 | 3,818.8 Kb | 581 | 663 | 16
688 | 558 | 39,509.4 | 0:09:55 | 0:08:02 | 2 | 3,808.0 Kb | 582 | 664 | -10
689 | 557 | 39,611.0 | 0:09:56 | 0:08:02 | 2 | 3,850.3 Kb | 583 | 665 | 42
690 | 556 | 39,692.7 | 0:09:56 | 0:08:00 | 2 | 3,800.4 Kb | 584 | 666 | -50
(and so on.....)

And then it runs once again, returning a similar email, and over and over again, perhaps it takes four or five runs to end. Before updating it only took two cronjob runs, but something similar happened. Do you have any hints? is this something regarding my cronjobs that I could change, or it has something to do with my generator settings?
Thanks in advance

Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.