XML Sitemaps Generator

Author Topic: Sitemap generation crawling problems  (Read 25634 times)

sales53

  • Registered Customer
  • Approved member
  • *
  • Posts: 1
Sitemap generation crawling problems
« on: June 26, 2006, 03:59:16 PM »
Everything worked fine and I was running the generator through a cron job. Today I noticed that 5 days ago the script did stop generating sitemaps. I ran it manually and it hangs up at the step shown below:

Please wait. Sitemap generation in progress...
Links depth: 3
Current page: kb/article.php?id=032
Pages added to sitemap: 95
Pages scanned: 240 (2,327.8 Kb)
Pages left: 15 (+ 209 queued for the next depth level)
Time passed: 0:13
Time left: 0:00
Memory usage: 571.7 Kb


Looking a t the apache log files the last entry is:

129.42.208.182 - - [26/Jun/2006:11:51:28 -0400] "GET /map/index.php?op=crawlproc&bg=&resume= HTTP/1.1" 200 1812 "[external links are visible to admins only]" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.4) Gecko/20060508 Firefox/1.5.0.4"

I'm not sure what to do next?  Any ideas?

Thanks   ... Udo

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Sitemap generation crawling problems
« Reply #1 on: June 27, 2006, 09:55:58 PM »
Hello Udo,

when you execute sitemap generator from web interface, the script is usually running with different settings than running as cron job and the maximum execution time is often limited. So, you see the screen stops updating after some time.
Please run sitemap generator from the SSH command line instead (yu can use the same command as specified for cron job) and see how it goes.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2