I just installed 2.9 yesterday in hopes of correcting this same issue. The new version doesn't work any differently. I never get an error msg. It just stops at random.
Links depth: 2 
Current page: Art/59897/185/Wedding-Favor-Selection-Some-Considerations.html 
Pages added to sitemap: 1463 
Pages scanned: 1480 (44,802.2 KB) 
Pages left: 36 (+ 7127 queued for the next depth level) 
Time passed: 30:28 
Time left: 0:44 
Memory usage: - 
It will stop between 500 and 1500 pages. Restarting means it will crawl another 1000pages +- before it gets stuck again.
I tried at at least 20 times with different settings and I put the IP address in the box at the bottom of the config screen.
Thanks.