Help with "Crawing" tab...
« on: February 22, 2007, 02:14:06 PM »
OK, I have a very large site, over 200,000 pages.

I started the Crawl about a week ago, and for the first few days, when I went to the"Crawl" tab, it showed me the progress (How many pages were crawled, how many scheculed for next level, etc".

But not, when I click on the Crawling Tab, it takes about a minute and just somes back saying:

Quote
CRAWLING

Run In Background
O  Do not interrupt the script even after closing the browser window until the crawling is complete

It no longer shows me any stats on it's progress.  Is this normal?

Thank you.
« Last Edit: February 22, 2007, 02:19:27 PM by ONC Hosting »
Re: Help with "Crawing" tab...
« Reply #1 on: February 23, 2007, 12:05:10 AM »
Hello,

perhaps the script times out when reading the progress from the current dump file (which may be the case when dump is huge). I suggest you to execute sitemap generator from command line via ssh for better performance.
Re: Help with "Crawing" tab...
« Reply #2 on: February 23, 2007, 02:22:46 AM »
OK, I do have access to the server root, but am not proficient in ssh.   I do have Putty.

Can you please tell me exactly how to run the script in SSH to do this?   Please be specific as I do not know ssh well at all.

Thanks.