XML Sitemaps Generator

Author Topic: Crawling is stopping after a while  (Read 5662 times)

pette84

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
Crawling is stopping after a while
« on: August 14, 2014, 02:03:33 PM »
I have a problem with the script, I installed it correctly and started the crawling. But everytime its stopped, I waited now a long time, but nothing happens...

The error
Quote
Links depth: 3
Current page: smartphone-samsung/discussions/review?id=43
Pages added to sitemap: 669
Pages scanned: 975 (70,709.1 KB)
Pages left: 18 (+ 16 queued for the next depth level)
Time passed: 0:13:29
Time left: 0:00:14
Memory usage: 2,174.8 Kb
Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at [external links are visible to admins only] to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.

Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.
Quote

I have in the file php.ini this setting:
safe_mode=Off
memory_limit=128M
max_execution_time=30
post_max_size=64M
upload_max_filesize=64M

Can you help me?
Thank you very much
kind regards
Andrea

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Crawling is stopping after a while
« Reply #1 on: August 14, 2014, 08:04:15 PM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
 
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

pette84

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
Re: Crawling is stopping after a while
« Reply #2 on: August 14, 2014, 09:00:06 PM »
Hi, thank you very much for your support!

parameter max_execution_time to what I should set it to run the script correctly?
While the memory_limit parameter is set to 128 MB

Thank you very much
kind regards
Andrea

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Crawling is stopping after a while
« Reply #3 on: August 15, 2014, 06:10:28 AM »
Please try
memory_limit=256M
max_execution_time=9000
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2