Sitemap Collapses Near To End Of Crawl
« on: October 16, 2012, 08:40:14 AM »
My sitemap crawl collapses just before the end of the crawl.

Can I use the crawl_dump.log top generate a site map with what has been crawled to that point?
Re: Sitemap Collapses Near To End Of Crawl
« Reply #1 on: October 17, 2012, 01:20:51 AM »
You can set "Maximum pages" setting to a lower number of resume generator and it will not crawl any more pages.
Re: Sitemap Collapses Near To End Of Crawl
« Reply #2 on: October 18, 2012, 11:31:00 AM »
No thats not what i am asking.

Is there anyway of generating a site map from the pages which have been crawled to that point.

The problem is that the crawl for the 9k pages took about one week to generate [and has had to be manually re-started a few dozen times] - so I would wish to avoid running it again.

Please advise?

Re: Sitemap Collapses Near To End Of Crawl
« Reply #3 on: October 18, 2012, 01:02:54 PM »
That's correct - you would run generator again, resuming the process so it will read the progress form crawl_dump file and create sitemap without crawling any more pages.
Re: Sitemap Collapses Near To End Of Crawl
« Reply #4 on: October 18, 2012, 01:14:11 PM »
I now see!

thank you.