XML Sitemaps Generator

Author Topic: Slow?  (Read 26672 times)


  • Registered Customer
  • Approved member
  • *
  • Posts: 2
« on: July 11, 2006, 08:05:06 AM »
I noticed that SG is slowing down when indexing more pages. For 4 days is scanned about 40K pages, but for last day only 500. I would like to index over 200K pages, but i don't know when it finish indexing.

Pages added to sitemap: 40375
Pages scanned: 40540 (847,164.3 Kb)
Pages left: 103579 (+ 44164 queued for the next depth level)
Time passed: 4288:38
Time left: 10957:25
Memory usage: 64,821.3 Kb

Also very often i have problem to access Crawling page.
In addition i noticed that the script slowing down whole server.

btw. Have you thought about database version, where links could be stored?

Yours Jack
« Last Edit: July 11, 2006, 08:06:53 AM by fun »

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10599
Re: Slow?
« Reply #1 on: July 12, 2006, 09:38:53 PM »

since sitemap generator crawls your site with http requests, the main speed factor is how fast your pages are generated at the server.
You can find out more details here:
How long it will take to crawl my large website
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.


SMF 2.0.12 | SMF © 2014, Simple Machines