Sitemap Generator Forum

keep getting this error. please can you help?

Fatal error: Out of memory (allocated 117702656) (tried to allocate 44 bytes) in /home/mydomain/public_html/generator/pages/class.utils.inc.php on line 102

Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.

I'm getting the same error.  Our server has a memory limit of 128 M, so I set XML-site map to a limit of 125 M, but it still timed out.

The max execution time is 30 (seconds), which may be the problem.  What's the correct setting for max execution time - unlimited?  We have around 390,000 pages ...

Hello,

with website of this size the best option is to create a limited sitemap - with "Maximum depth" or "Maximume URLs" option limited so that it would gather about 100-150,000 URLs, which would be main pages representing "roadmap" sitemap for search engines.

The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.

Some of the real-world examples of big db-driven websites:
about 35,000 URLs indexed - 1h 40min total generation time
about 200,000 URLs indexed - 38hours total generation time

With "Max urls" options defined it would be much faster than that.