Fatal error: Allowed memory size
« on: January 21, 2008, 06:51:15 PM »
Hello,
I get this error, after it scans about 10000 pages:

Fatal error: Allowed memory size of 50331648 bytes exhausted (tried to allocate 7077889 bytes) in /home/******/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 6

thanks.
Re: Fatal error: Allowed memory size
« Reply #1 on: January 21, 2008, 10:42:29 PM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Re: Fatal error: Allowed memory size
« Reply #2 on: January 22, 2008, 03:53:28 PM »
My host asks:

How much memory the process require?

Thanks
Re: Fatal error: Allowed memory size
« Reply #3 on: January 22, 2008, 10:34:05 PM »
Hello,

the memory requirements depend on how many total URLs are there on your site, do you have an estimation on that number?
Re: Fatal error: Allowed memory size
« Reply #4 on: January 23, 2008, 02:33:00 PM »
I am having a similar problem, I keep increasing the memory limit which gets me a bit further each time, however I am yet to complete a site map.

Is there any way of estimating the memory limit i will need based on the number of pages? It is a large site and as an esitmation i would guess there is somewhere in the region of upwards of 50,000 pages.
Re: Fatal error: Allowed memory size
« Reply #5 on: January 23, 2008, 10:00:30 PM »
Hello,

you can check the current memory usage in progress window and the number of URLs crawled so far to estimate required memory limit.
Re: Fatal error: Allowed memory size
« Reply #6 on: January 24, 2008, 02:53:18 PM »
The host said they allocated they're best, 64Mb... but I stii get this error:

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 17039477 bytes) in /home/in3sting/public_html/generator/pages/class.utils.inc.php(2) : eval()'d code on line 6

Pages added to sitemap: 14630
Pages scanned: 15300 (70,993.9 Kb)
Pages left: 6250 (+ 7639 queued for the next depth level)

What now? :(
Re: Fatal error: Allowed memory size
« Reply #7 on: January 25, 2008, 12:45:24 AM »
Hello,

please PM me your generator URL. If it's not possible to increase memory limit even more, we'll try to optimize crawler setting for your specific website to get it finished.
Re: Fatal error: Allowed memory size
« Reply #8 on: January 25, 2008, 04:05:45 PM »
Ok,
I've sent you a PM. Please have a look!
Re: Fatal error: Allowed memory size
« Reply #9 on: January 29, 2008, 10:08:26 PM »
An update:

your gallery had infinite loops of "User login" and "User password recovery" links that caused huge number of "noise pages". I've excluded them in sitemap generator configuration and full sitemap has been created successfully now.
Re: Fatal error: Allowed memory size
« Reply #10 on: February 23, 2008, 10:39:05 AM »
I seem to be having the same problem, but I cannot see where the site loops...
[ External links are visible to forum administrators only ]

I already blocked reclame.php and report.php

Re: Fatal error: Allowed memory size
« Reply #11 on: February 24, 2008, 09:09:18 PM »
Hello,

I've added ".html" into "Do not parse" option (they are still included in sitemap though) and sitemap has been created successfully.