XML Sitemaps Generator

Author Topic: Am I crazy?  (Read 18349 times)

ONC Hosting

  • Registered Customer
  • Approved member
  • *
  • Posts: 9
Am I crazy?
« on: March 25, 2007, 01:57:26 PM »
Is it my imagination or does the script use more resources and get slower as it crawls...

My site is about 1.5 million pages.   I am on 200,000 pages in my crawl and it is going MUCH slower than in the early pages...

Am I right?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10621
Re: Am I crazy?
« Reply #1 on: March 25, 2007, 10:33:55 PM »
Hello,

yes, it is possible that crawling becomes slower when significant amount of pages are already crawled, the actual decrease in speed depends on the number of pages and server configuration.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2