Trying to restart, but it is timing out and providing an error page

[ttanaka]

Hi, we had the crawl running well for several days.  It stopped and this time, when we tried to relaunch the crawler, it would never start again.  I believe the first couple times I saw an error message about couldn't open stream something.  But now, I just get a page not found after long time, and the crawl doesn't continue.  Any suggestions on how to get the crawl to continue would be greatly appreciated!

Thanks

[ttanaka]

Looks like I got it running again!  I rebooted our server and changed the configuration to wait 3 seconds for every 5 requests.  That seems to have gotten it going again!  I don't know if that delay is too long or too short, but seems to be working ok.

[admin]

Hello,

ok, great.
You can try to configure a bit shorter delays though (or increase the number of requests between delays).

[ttanaka]

We had to reboot our server, and now we can't get the process started, getting the error messages below:

Resuming the last session (last updated: 2006-04-12 02:17:50)
Warning: fopen(/home/eproxim/public_html/XML-Sitemap-Generator/data/crawl_dump.log): failed to open stream: Permission denied in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 15

Warning: fwrite(): supplied argument is not a valid stream resource in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 16

Warning: fclose(): supplied argument is not a valid stream resource in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 17

Warning: fopen(/home/eproxim/public_html/XML-Sitemap-Generator/data/crawl_dump.log): failed to open stream: Permission denied in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 15

Warning: fwrite(): supplied argument is not a valid stream resource in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 16

Warning: fclose(): supplied argument is not a valid stream resource in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 17

Any ideas?  Thanks.

[ttanaka]

Still running into this problem, anyone else having this issue?

[ash]

Check that the data directory is writeable.

I don't have a permissions problem but I got

Warning: gzfile(/home/crm/public_html/generator/data/sitemap.xml.gz): failed to open stream: No such file or directory in /home/crm/public_html/generator/pages/page-view.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 34

Warning: implode(): Bad arguments. in /home/crm/public_html/generator/pages/page-view.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 34

I left Oleg a PM as advised in another thread.

[admin]

Hello,

if you are getting the permissions error message, you should make sure that not only data/ folder has 0777 permissions set, but also all files in data/ folder are set to 0666.