500 Error after 6 hours of crawling
« on: September 24, 2016, 12:17:54 AM »
I did a trial of your component and it created a sitemap of 500 urls quickly with no problem. I bought the standalone version and it has been running for 6 hours and finally gives me a 500 server error. Now I can't get back in to change the settings. Help?
Re: 500 Error after 6 hours of crawling
« Reply #1 on: September 24, 2016, 09:01:25 AM »
Hello,

it will take more time/resources to create larger sitemap when it's not limited.
it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Re: 500 Error after 6 hours of crawling
« Reply #2 on: September 26, 2016, 04:22:16 PM »
What are the recommended settings? Mine currently are:
max_execution_time = 900
max_input_time = 120
memory_limit = 768M
and I still get the 500 error.
Re: 500 Error after 6 hours of crawling
« Reply #3 on: September 26, 2016, 07:51:10 PM »
How many pages suppose to be crawl on your web site?

I mean what are estimated pages on your web site ?
Re: 500 Error after 6 hours of crawling
« Reply #4 on: September 26, 2016, 07:55:16 PM »
Please try
max_execution_time = 9000

Also, in this case I would recommend to run generator in command line if you have ssh access to your server.