XML Sitemaps Generator

Author Topic: Generator stop after crawling few thousand  (Read 9822 times)

alexghiurco

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
Generator stop after crawling few thousand
« on: January 27, 2011, 02:33:37 PM »
Hello.
Just like the title say, generator stops after crawling about 3,5k or 4 url`s.
It stops in manual and also in cron job.
I don`t know what to do , to finish the crawling in auto mode.
Please help me .
Thank you.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Generator stop after crawling few thousand
« Reply #1 on: January 27, 2011, 03:17:22 PM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

design2

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
Re: Generator stop after crawling few thousand
« Reply #2 on: January 28, 2011, 12:22:49 PM »
I am having the same problem (it stops at 1630 files). My php.ini max_execution_time is set at 60. What should it ideally be?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Generator stop after crawling few thousand
« Reply #3 on: January 28, 2011, 11:11:39 PM »
That depends on the number of pages that will be found, try to increase it to 600.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

design2

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
Re: Generator stop after crawling few thousand
« Reply #4 on: January 29, 2011, 12:10:48 PM »
Unfortunately increasing the php.ini max_execution_time to 600 made no difference, it still stops at 1629 files.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Generator stop after crawling few thousand
« Reply #5 on: January 29, 2011, 01:24:28 PM »
Hello,

please let me know your generator URL/login in private message to check this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

design2

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
Re: Generator stop after crawling few thousand
« Reply #6 on: January 29, 2011, 02:32:56 PM »
I just sent you a PM

piemant

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Generator stop after crawling few thousand
« Reply #7 on: January 29, 2011, 03:15:03 PM »
Similar problem, I've sent a PM.

alexghiurco

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
Re: Generator stop after crawling few thousand
« Reply #8 on: January 29, 2011, 10:05:29 PM »
I have incresead memory limit at 128M and time at 600 but generator still stops at 5k url. Any ideeas?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Generator stop after crawling few thousand
« Reply #9 on: January 30, 2011, 07:34:13 AM »
Hello,

please let me know your generator URL/login in private message to check this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2