sever loads and custom settings
« on: March 24, 2011, 06:26:41 PM »
currently my first build (been running 10hours+) is using a lot of resources  load average: 8.42, 9.10, 9.85.  What i am worried about is that the sitemap has to re-build everything (crawl through everything again) to update.

Is there a setting where I can do an initial full crawl -> the rest of the scans are front page  + 2 levels (to see if there are any changes).

I currently run a huge wordpress blog and it has many posts/image galleries so re-crawling everything each time the sitemap is re-built is very hard.

Re: sever loads and custom settings
« Reply #1 on: March 25, 2011, 03:48:27 PM »

yes, you can store existing full sitemap under a different filename (sitemapfull.xml for instance), then set Depth level to "2" in generator config, and all new sitemaps will be limited as a result.