XML Sitemaps Generator

Author Topic: server kills sitemap generator  (Read 20586 times)

mrk2mrkt

  • Registered Customer
  • Approved member
  • *
  • Posts: 5
server kills sitemap generator
« on: June 05, 2008, 07:19:22 PM »
I have a site with about 4900 pages and 775,000 links on those pages.

I never had troubles until recently. Now the site map generator stalls on the 3rd level. My server provider indicates that the sitemap generator was using 100% of the cpu and had to be stopped.

This problem is consistent and does not changes from attemt to attempt.

The only change I have made has been to add about 700 pages (which is included in the 4900 number above). When the task is aborted, it is not crawling one of those pages.

Tech Support (Lexiconn server) seems to think that there is a bug in the sitemap generator cauksing it to go into a loop.

I'm not sure how to go about debugging this further.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: server kills sitemap generator
« Reply #1 on: June 05, 2008, 08:49:41 PM »
Hello,

I suggest you to use "Make delay after each X request" option to reduce the server load while crawling. In case if it interrupts the process, you can resume generation from that point.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

mrk2mrkt

  • Registered Customer
  • Approved member
  • *
  • Posts: 5
Re: server kills sitemap generator
« Reply #2 on: June 06, 2008, 07:08:12 PM »
 I tried a 1 sec delay after 5 accesses  and 1 sec delay after 2 access…server shut us down both times..

 


XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: server kills sitemap generator
« Reply #3 on: June 07, 2008, 12:47:48 AM »
Does that happen quickly ? How many URLs are scanned to that point?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

mrk2mrkt

  • Registered Customer
  • Approved member
  • *
  • Posts: 5
Re: server kills sitemap generator
« Reply #4 on: June 23, 2008, 07:14:14 PM »
sorry for the delay in getting back to you..

server killing site map was fixed...software issue..

problem now is that your generator stops at level 3...indicates it is done...



any ideas?
« Last Edit: June 23, 2008, 10:03:43 PM by admin »

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: server kills sitemap generator
« Reply #5 on: June 23, 2008, 10:03:25 PM »
Please install the latest version of Sitemap Generator. Replied to your email.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2