The crawler has been crawling my site for the past 10 hours. Every time it reaches one particular URL it stalls, restarts, stalls, restarts. It has been doing this endlessly.
Here is the info displayed:
Links depth: 4
Current page: gallery/photo.php?exhibition=61&photo=3501&ee_lang=eng
Pages added to sitemap: 11695
Pages scanned: 11720 (171,319.6 KB)
Pages left: 48308 (+ 0 queued for the next depth level)
Time passed: 0:44:44
Time left: 3:04:26
Memory usage: 51,680.1 Kb
I checked the URL and it seems unremarkable. It's just another photo in a gallery app.