XML Sitemaps Generator

Author Topic: Generator starts and stops, have 500k pages to crawl  (Read 10393 times)

sdeeth

  • Registered Customer
  • Approved member
  • *
  • Posts: 1
Generator starts and stops, have 500k pages to crawl
« on: December 27, 2009, 08:09:06 PM »
I have a windows server.  I have the generator to pause 60 seconds after each 1000 pages.  It crawls for a while then stops.  I see other people have had same issue, but then you pm them for help.  Can you help me with this?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Generator starts and stops, have 500k pages to crawl
« Reply #1 on: December 27, 2009, 10:36:23 PM »
Hello,

having 60 seconds delay is too long, a few seconds is sufficient.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2