Sitemap Generator Forum

Hi, we had the crawl running well for several days.  It stopped and this time, when we tried to relaunch the crawler, it would never start again.  I believe the first couple times I saw an error message about couldn't open stream something.  But now, I just get a page not found after long time, and the crawl doesn't continue.  Any suggestions on how to get the crawl to continue would be greatly appreciated!

Thanks

Looks like I got it running again!  I rebooted our server and changed the configuration to wait 3 seconds for every 5 requests.  That seems to have gotten it going again!  I don't know if that delay is too long or too short, but seems to be working ok.

Hello,

ok, great.
You can try to configure a bit shorter delays though (or increase the number of requests between delays).

We had to reboot our server, and now we can't get the process started, getting the error messages below:

Resuming the last session (last updated: 2006-04-12 02:17:50)
Warning: fopen(/home/eproxim/public_html/XML-Sitemap-Generator/data/crawl_dump.log): failed to open stream: Permission denied in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 15

Warning: fwrite(): supplied argument is not a valid stream resource in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 16

Warning: fclose(): supplied argument is not a valid stream resource in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 17

Warning: fopen(/home/eproxim/public_html/XML-Sitemap-Generator/data/crawl_dump.log): failed to open stream: Permission denied in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 15

Warning: fwrite(): supplied argument is not a valid stream resource in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 16

Warning: fclose(): supplied argument is not a valid stream resource in /home/eproxim/public_html/XML-Sitemap-Generator/pages/class.utils.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 17

Any ideas?  Thanks.

Still running into this problem, anyone else having this issue?

Check that the data directory is writeable.

I don't have a permissions problem but I got

Warning: gzfile(/home/crm/public_html/generator/data/sitemap.xml.gz): failed to open stream: No such file or directory in /home/crm/public_html/generator/pages/page-view.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 34

Warning: implode(): Bad arguments. in /home/crm/public_html/generator/pages/page-view.inc.php(2) : eval()'d code(1) : eval()'d code(1) : eval()'d code on line 34

I left Oleg a PM as advised in another thread.

Hello,

if you are getting the permissions error message, you should make sure that not only data/ folder has 0777 permissions set, but also all files in data/ folder are set to 0666.