Website won't work during crawl
« on: November 01, 2008, 06:48:06 PM »
The sitemap generator crawls my site pretty fast, but during the crawl, my website doesn't work. Other sites on my server work, so the problem is specific to the website being crawled.

This wouldn't be a problem, but with around 50,000 pages, it takes several hours to crawl my site. I can't have it down the entire time. How do I fix this?
Re: Website won't work during crawl
« Reply #1 on: November 02, 2008, 08:30:01 AM »

your websites works fine during the crawling for other visitors, it just may be inaccessible for *you* only if you have open sitemap generator in the browser since it keeps ongoing connection with you and that in some cases prevent to open other pages in the same browser. If you have other browser installed, you can try it.