Website+Vbulletin forum, The Crawler times out
« on: February 25, 2011, 05:00:50 PM »
Hi,

We have recently installed the latest Sitemap Generator and having issues in generating the sitemaps automatically. The Crawler doesn't crawls more than 10000 pages. It has only generated one sitemap by Manual Crawling in which we have llimited the number of crawl pages to 1000.

Please advice.
Re: Website+Vbulletin forum, The Crawler times out
« Reply #1 on: February 26, 2011, 08:34:17 AM »
Hello,

make sure that you have selected "URLs exclusion preset" for vBulletin in generator configuration.
Re: Website+Vbulletin forum, The Crawler times out
« Reply #2 on: February 26, 2011, 08:29:39 PM »
Thanks Oleg. We will test it out.