Just today I noticed when I go to the manual crawl page at mysite/generator/index.php I get an Internal Server error 500. This is displayed immediately the mysite/generator/index.php never appears.
The automated nightly cron job is still working however as there is an email report every day and the new xml site map file is created in the proper place.
What is wrong? If it helps some here is the error message displayed at the bottom of the Internal Server Error 500 page:
Apache/2.0.63 (Unix) mod_ssl/2.0.63 OpenSSL/0.9.8e-fips-rhel5 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/184.108.40.20635 Server at mysite.com Port 80