Stopped after one refresh
« on: April 20, 2009, 06:06:19 AM »
My Crawling stopped after just one refresh, after one hour i decided that there will be no further refresh.

Links depth: 3
Current page: product_info.php?products_id=42334
Pages added to sitemap: 3795
Pages scanned: 6160 (222,459.9 KB)
Pages left: 13686 (+ 25194 queued for the next depth level)
Time passed: 24:17
Time left: 53:58
Memory usage: 30,818.5 Kb
Resuming the last session (last updated: 2009-04-19 23:07:22)

Can you assume what the problem might be?

Re: Stopped after one refresh
« Reply #1 on: April 20, 2009, 09:55:00 AM »
Problem seems to be fixed.

But now there is another Problem, after some time, about 20 minutes, the crawler stops. Nothing implies that the Crawler is still running. I have to restarte the process.

As i have to scan about 700.000 pages, this stop and restart, will be not good at all.

Is there a Fix to correct this?

Thanks in advance,
Re: Stopped after one refresh
« Reply #2 on: April 21, 2009, 01:33:34 AM »
Replied to your email.