XML Sitemaps Generator

Author Topic: After 8 hours and some 35,000 pages crawling suddenly stops  (Read 8209 times)

pspglobalnet

  • Registered Customer
  • Approved member
  • *
  • Posts: 1
After 8 hours and some 35,000 pages crawling suddenly stops
« on: August 18, 2011, 10:27:03 AM »
Hello!

Our site has aproximately 90,000 pages. After 8 hours and some 35,000 pages crawling suddenly stops.

Trying to resume I select:
"Continue the interrupted session (2011-08-18 06:50:15, URLs added: 35685, estimated URLs left in a queue: 20148)"

Yet, crawling does not continue.
At this point crawl_dump.log is already 34 MB in size.

sitemap.xml and sitemap.xml.gz are set to 666 and data folder is set to 777.


I have enclosed my configuration file and teh screen shots of the two crawling pages.

Please advise.
Thanks,

peter

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: After 8 hours and some 35,000 pages crawling suddenly stops
« Reply #1 on: August 18, 2011, 01:10:13 PM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2