Hi,

I'm using the Standalone Sitemap Generator (PHP) v8.2 since its release but it stopped working, recently.
I upgraded PHP to 8.1 and the pages to be crawled increased to about 28000.
When I start the crawler, it shows, that it's running and a sitemap.xml is being created.
I have to continue interrupted session a couple of times, as the time limit is reached.
Then I get the following info:

Code: [Select]
Links depth: 5
Current page: db/apple/watch_7
Pages added to sitemap: 1452
Pages scanned: 1457 (3,045.7 KB)
Pages left: 0 (+ 0 queued for the next depth level)
Time passed: 0:13:36
Time left: 0:00:00
Memory usage: 1,477.0 Kb

and at the end:
Code: [Select]
Crawling completed
Crawling phase completed
Total pages indexed: 1452
Creating sitemaps...
55%
| 87.8 |
/ 0 / 1672190253| 141.6 |
(memory: 1,476.7 Kb)
| 141.6 |
/ 100 / 54| 188.2 |
(memory: 1,477.0 Kb)
| 188.2 |
/ 200 / 47| 234.3 |
(memory: 1,477.0 Kb)
| 234.4 |
/ 300 / 46| 279.5 |
(memory: 1,477.0 Kb)
| 279.6 |
/ 400 / 45| 327.5 |
(memory: 1,477.0 Kb)
| 327.5 |
/ 500 / 48| 374.8 |
(memory: 1,477.0 Kb)
| 374.8 |
/ 600 / 47| 420.8 |
(memory: 1,477.0 Kb)
| 420.8 |
/ 700 / 46| 465.2 |
(memory: 1,477.0 Kb)
| 465.2 |
/ 800 / 45| 510.5 |
(memory: 1,477.0 Kb)
| 510.5 |
/ 900 / 45

Below, there is:

Code: [Select]
Auto-restart monitoring: Wed Dec 28 2022 02:28:02 GMT+0100 (Mitteleuropäische Normalzeit) (5 second(s) since last update)
This is updated endlessly.

Can I send you a test login by PM, so you can check the output and possible problems or can you identity something from the output above?
Re: Script runs endlessly and sitemap.xml is created over and over again
« Reply #1 on: December 29, 2022, 08:49:10 AM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
 
Re: Script runs endlessly and sitemap.xml is created over and over again
« Reply #2 on: December 29, 2022, 11:20:53 PM »
Thanks,
During the last years, it worked. I'm running on a shared host and have fixed

max_execution_time = 120
memory_limit = 256M

I thought, the XML Sitemaps script takes care of these limits and restarts to finish crawling.
In the output, I see "Crawling completed. Crawling phase completed" which seems to indicate, this works. Afterwards, I see "Total pages indexed: 1453
Creating sitemaps...
34%
| 88.3 |
/ 0 / 1672355058| 144.8 |
(memory: 1,478.7 Kb)

Then Only "resuming generator (5 seconds with no response)" is shown infinitely.

It seems, the script is stuck at 34%. Where do you see, if the memory & max time is exceeded?

Thanks for support!
Re: Script runs endlessly and sitemap.xml is created over and over again
« Reply #4 on: December 30, 2022, 09:35:49 PM »
Thanks, done.