XML Sitemaps Generator

Author Topic: Links depth: 5 error  (Read 18126 times)

kevin9

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Links depth: 5 error
« on: January 23, 2008, 12:15:54 PM »
I am trying to run a crawl on a site with about 20,000 identified pages but it keeps freezing at Links depth: 5  (13000 pages scanned and 8000 queued). i have run and re-run the crawl on a few occasions and it keeps freezing at this point. I have also set the crawl to delay for 60s after each  1000 requests if that helps?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Links depth: 5 error
« Reply #1 on: January 23, 2008, 09:58:52 PM »
Hello,

do you have "resume generation" checkbox when running the generator again?
There is no need in so long delays, something like 5 seconds every 200 request would be better.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

kevin9

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Re: Links depth: 5 error
« Reply #2 on: January 24, 2008, 12:59:47 PM »
Hi

Yes i have the 'resume generation' turned on and i have changed the delay to 6 seconds but it is still pausing at Link depth 5. I have tested the software to make sure that it is working and it seems to create sitemaps for them with no problem (they are a lot smaller though).

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Links depth: 5 error
« Reply #3 on: January 25, 2008, 12:43:50 AM »
Hello,

please let me know your generator URL in private message.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2