• Welcome to Sitemap Generator Forum.
 

Am I crazy?

Started by ONC Hosting, March 25, 2007, 02:57:26 PM

Previous topic - Next topic

ONC Hosting

Is it my imagination or does the script use more resources and get slower as it crawls...

My site is about 1.5 million pages.   I am on 200,000 pages in my crawl and it is going MUCH slower than in the early pages...

Am I right?

XML-Sitemaps Support

Hello,

yes, it is possible that crawling becomes slower when significant amount of pages are already crawled, the actual decrease in speed depends on the number of pages and server configuration.