I have a problem, I trying to crawl my site but I receive the error evertime that I do the task:
I got this error:
Links depth: 2
Current page: order-online-en.html
Pages added to sitemap: 820
Pages scanned: 820 (22,643.2 KB)
Pages left: 310 (+ 3893 queued for the next depth level)
Time passed: 0:10:50
Time left: 0:04:05
Memory usage: 3,053.5 Kb
Resuming the last session (last updated: 2012-07-06 00:17:39)
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there was an error in a CGI script.
If you think this is a server error, please contact the webmaster.
Fri Jul 6 02:16:34 2012
Any help please..!!!!!