Hi, I've just installed and run the crawl script, but it seems to loop endlessly, ie.
Links depth: 1
Current page: /
Pages scanned: 1 (29.1 Kb)
Pages left: 170
Time passed: 0:00
Time left: 0:06
It just scans the pages (20 at a time) then returns to this information, over and over
I guess I'm doing something wrong
, but the config seems to work.