Hope someone can give a little guidance on this.
Putting a little SEO work into the site currently with the sitemap and various other bits and pieces. We are reasonably well spidered currently but I'm sure it could be better. Anyhow the main part of the site is a forum, it's been running a few years and has around 180,000 threads (which I'm sure isn't too large as forums go) along with blog posts, archives of other content etc.
I've set standalone sitemap generator running with unlimited depth and it runs out of memory. Upped php limits a few times but it still doesn't make it all the way through. As such I'm not sure how to proceed
a) Limit crawl depth for the map so it completes with the memory available and leave it at that (already done this!)
b) Max out php memory in order the map completes at least once. Maybe leave the map with google for a few weeks in order they index as much as possible cleanly.
c) Ensure there is enough memory available the map can complete every time. Think I'd be needing at least 512mb...