XML Sitemaps Generator

Author Topic: Slow?  (Read 26880 times)

fun

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Slow?
« on: July 11, 2006, 08:05:06 AM »
Hello!
I noticed that SG is slowing down when indexing more pages. For 4 days is scanned about 40K pages, but for last day only 500. I would like to index over 200K pages, but i don't know when it finish indexing.

Pages added to sitemap: 40375
Pages scanned: 40540 (847,164.3 Kb)
Pages left: 103579 (+ 44164 queued for the next depth level)
Time passed: 4288:38
Time left: 10957:25
Memory usage: 64,821.3 Kb


Also very often i have problem to access Crawling page.
In addition i noticed that the script slowing down whole server.

btw. Have you thought about database version, where links could be stored?

Yours Jack
« Last Edit: July 11, 2006, 08:06:53 AM by fun »

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Slow?
« Reply #1 on: July 12, 2006, 09:38:53 PM »
Hello,

since sitemap generator crawls your site with http requests, the main speed factor is how fast your pages are generated at the server.
You can find out more details here:
How long it will take to crawl my large website
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2