XML Sitemaps Generator

Author Topic: Crawling keeps stopping at 5000 ish  (Read 17456 times)

admin103

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Crawling keeps stopping at 5000 ish
« on: October 17, 2007, 08:13:01 AM »
Hiya,

I keep trying to fully XML-Sitemap my site, but the script keep stopping around the 5000 links mark.

Any ideas?  Need to see anything from my site?


Many thanks.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Crawling keeps stopping at 5000 ish
« Reply #1 on: October 17, 2007, 10:57:46 AM »
Hello,

please try to increase max_execution_time and memory_limit settings in php configuration on your host.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

admin103

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Crawling keeps stopping at 5000 ish
« Reply #2 on: October 17, 2007, 05:12:04 PM »
Hiya,

I did do. I read a lot of forum posted first to try and work it out myself. Settings are:-

max_execution_time = 9000
memory_limit = 500M

The script seems to run for about 5 - 10 mins. I then have to "Resume last session" for it continue. This general does work, but it stops when it gets to approx 5000 links. Its currently stuck on "Continue the interrupted session (2007-10-13 16:04:16, URLs added: 4783, estimated URLs left: 17363)".

Any ideas?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Crawling keeps stopping at 5000 ish
« Reply #3 on: October 18, 2007, 11:07:10 AM »
Hello,

could you please let me know your generator URL in private message?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

admin103

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Crawling keeps stopping at 5000 ish
« Reply #4 on: October 18, 2007, 11:25:53 AM »
PM sent.

Thanks.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2