Am I crazy?
« on: March 25, 2007, 02:57:26 PM »
Is it my imagination or does the script use more resources and get slower as it crawls...

My site is about 1.5 million pages.   I am on 200,000 pages in my crawl and it is going MUCH slower than in the early pages...

Am I right?
Re: Am I crazy?
« Reply #1 on: March 25, 2007, 11:33:55 PM »

yes, it is possible that crawling becomes slower when significant amount of pages are already crawled, the actual decrease in speed depends on the number of pages and server configuration.
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.