Sitemap Generator Forum

Hi,

just bought it today.

my site has >4.000.000 pages - and when I use your script on my hosting it doesn't get enough of memory (I have 64Mb) and the error "Allowed memory size of 33554432 bytes exhausted (tried to allocate 6029459 bytes) in /home/usjobdb/public_html/generator/pages/class.utils.inc.php on line 102" shows up (I've stated 32Mb limit int the configuration tab - but still have the same thing)

so I've tried to crawl my site locally under Denwer - unfortunatelly it doesn't work well either.
it crawls only 7 pages (comapre to Pages scanned: 1220 before it reaches memory limit while on hosting)

how I may crawl all my website? is it possible with your script?

Hello,

with website of this size the best option is to create a limited sitemap - with "Maximum depth" or "Maximume URLs" option limited so that it would gather about 100-200,000 URLs, which would be main pages representing "roadmap" sitemap for search engines.

Also, you would need to increase memory_limit to say 256M.

ok, Oleg, but why it doesn't work well on my local computer where there are no memory limitations?

that's the only reason why I bought the script for such a big site - I've planned to use it locally with my computer which for sure has no hosting limitations.

will you help me to set up it with denwer on my local computer?

Hello,

could you please PM me an example URL that is not included in sitemap and how it can be reached starting from homepage?