Don't think my crawler runs at all??
« on: August 08, 2006, 01:48:02 PM »
Hi,

I just bought and installed the standalone version today (the web based free crawler worked great for my site but 500 pages was slightly too small for me.)

My site I think is around just 600 pages.

I set up the config with:-
Max pages: 0
Max exec time: 9000
Save script: 0

And ran the crawler with Run in Background checked and I get "Please wait. Sitemap generation in progress..." and nothing more.  I left this for a few hours and nothing changed. 
Even when I navigate away from the Crawler page and back to it I get no indication that anything had happened.  It invites me to hit the Run button again and if I do says "Please wait. Sitemap generation in progress...".

So to test if it was doing anything I changed the Max pages to 1 and Save Script to 60.. and still all I can do it hit Run, get the please wait screen and nothing else happens.

This is in firefox - I just thought of trying running via IE and now in the box under "Please Wait. Sitemap generation....." I get a Page not Found error.

Don't see any obvious missing files.  wwwroot/sitemap.xml has been created zero size and set to 666 and config.inc.php is set to 666 also.

Anyone got any pointers as to what's going on?

PHP version is 4.4.2

Cheers,
Dave.
Re: Don't think my crawler runs at all??
« Reply #1 on: August 08, 2006, 07:13:29 PM »
BTW happy to PM a url to anyone who might be able to help!
Re: Don't think my crawler runs at all??
« Reply #2 on: August 13, 2006, 10:12:29 AM »
Hello Dave,

sorry for delay.
How many pages your site has (estimated)? For the larger sites it is recommended to run sitemap generator from command line (via the ssh access to server).
Re: Don't think my crawler runs at all??
« Reply #3 on: August 13, 2006, 10:18:32 AM »
I'd say my site has just 500-600 pages.
Re: Don't think my crawler runs at all??
« Reply #4 on: August 13, 2006, 10:26:36 AM »
Oh I should say also I have a hosted server and only get FTP access to it.
Re: Don't think my crawler runs at all??
« Reply #5 on: August 13, 2006, 10:41:07 AM »
You may need to use "Save state" feature of sitemap generator and resume generation multiple times until it is completed.