Infinite loop when creating sitemap, video included
« on: October 08, 2012, 12:46:14 PM »
As you can see in this video: [ External links are visible to forum administrators only ]
crawling is completed and sitemap generation is started.

After running for 4700 seconds, it still regenerates the generator.gz file, and if I do refreshes on FTP it says different and different file size.

memory limit is set for 1GB
max execution time is 0 (infinite)

I've used htop, and I see the process is running and consuming 100% PCU.

I can confirm that this happened after I updated the script with the newest version, the same configuration settings were working fine with previous version.

Please fix urgently.
« Last Edit: October 08, 2012, 01:03:03 PM by kodokmarton »
Re: Infinite loop when creating sitemap, video included
« Reply #2 on: October 09, 2012, 03:21:58 PM »
I run the command from shell and still does the same loop, and the file is repeatedly overwritten as you see in the video.
Re: Infinite loop when creating sitemap, video included
« Reply #4 on: October 10, 2012, 06:52:56 AM »
I show you three screens:

1. command in shell, you can see that a bunch of columns report wrong numbers (the last 2 colums)
2. monitoring the file, through watch, you will see how the file size changes randomly when the site generation occurs (investigate why happens this)
3. showing the current processes via htop, and you see the process is at 100%

there is somewhere an infinite loop, as the process doesn't stop at all, I can leave there too run for 1-2 hour too

See video here: [ External links are visible to forum administrators only ]

There is absolutely wrong something in this v6 version, as the v5 version works fine.
Re: Infinite loop when creating sitemap, video included
« Reply #6 on: October 11, 2012, 08:27:36 AM »
Well it is very risky to provide that, because if you start the process, it will run forever, and after a while after enters in the infinite loop it will consume all CPU and will affect the server, and eventually the server load will be so high that the apache won't be able to server visitors.

There is no way you can stop a process that you have started in the website after enters the sitemap generation phase.

And I don't have access to root, and If I have to call someone to kill the process each time when runs from the interface.

I manually started under my SSH user (non sudo) the process to be able to cancel from the shell, but I won't give you the SSH access to the server.

Let me know what would you like to know and I will look it up for you.
Re: Infinite loop when creating sitemap, video included
« Reply #8 on: October 12, 2012, 02:26:54 PM »
I've tried that and when the time consumes the sitemap is not built. The same stuff happens.

When can you provide a fix for the problem?
Re: Infinite loop when creating sitemap, video included
« Reply #9 on: October 15, 2012, 06:55:38 AM »
> I've tried that and when the time consumes the sitemap is not built. The same stuff happens.

You need to run it again then, so that it resumes the process, until it completes it.
Re: Infinite loop when creating sitemap, video included
« Reply #10 on: October 15, 2012, 09:09:30 AM »
same stuff happens, it enters into a loop and never ends
I had to kill again after 30 minutes as server load was high.