I have my own server. I have already used your script to index a 225,000 page site in 50 hours without any problems.
On the same server is another site with about 300,000 pages. I am using the exact same settings I used for the first site. However, after about 116,000 pages and 24 hours it just stops.
It's not a memory issue. There is no php error in the php log. Your script just seems to "break".
When I click on the Crawling page, it does not load completely. After:
Run in background
Do not interrupt the script even after closing the browser window until the crawling is complete
There is nothing else. The second checkbox option and the submit button are missing.
This has happened twice now. The first time I blew it off, cleared the data directory, and started over.
But this time it happened again, and it looks like it happened at the same point during the crawl.