Crawler Gets stuck
« on: July 11, 2007, 12:38:42 PM »
Have restarted twice but crawler gets stuck, i restart at the save point...it runs for a few seconds and stop s again.

It reports:

Links depth: 3
Current page: index.asp?pageID=168&profileID=15
Pages added to sitemap: 2122
Pages scanned: 2160 (35,576.0 Kb)
Pages left: 29 (+ 804 queued for the next depth level)
Time passed: 29:31
Time left: 0:23
Memory usage: 2,510.9 Kb
Resuming the last session (last updated: 2007-07-11 20:37:53)
Re: Crawler Gets stuck
« Reply #1 on: July 11, 2007, 11:08:46 PM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Crawler Gets stuck
« Reply #2 on: July 12, 2007, 03:16:17 AM »

The following values are set:

max_execution_time   30   30
memory_limit   8M   8M

Memory seems fine.

With regard to execution time shouldn't this just pickup from where it left off and make progress each time?
Re: Crawler Gets stuck
« Reply #3 on: July 12, 2007, 11:13:47 PM »
Yes, try to increase both settings just in case.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.