Sitemap Generator Forum

I have a large site.   Support ran the script for me once, from SSH and it was done in two days. 

Now, I have more pages, and I ran the crawler about 3 weeks ago, and it just runs and runs, but never seems to finish.   I look at my web stats and see that it is constantly looking at pages.   But it just does not seem to finish.

When I click on the Crawling Tab, it takes forever, and then just says it is crawling.   What should I do?

Should I stop the current crawl and restart it from SSH?   If so, I have two questions...

1.  How do I stop the current crwal, without it overwriting my last sitemaps?

2.   How (In detail please) do I run the script in SSH?

Thank you in advance for your help.

Hello,

1. you can upload the empty "interrupt.log" file in generator/data/ folder to stop the crawler.
2. running the script from SSH is as easy as:
- connect to your server via SSH as usually
- open sitemap generator in browser and select "Crawling" tab, there will be a command line under the "Cron job setup" heading
- enter that command line in SSH window

My site is up to 500,000 pages.   Is it usual for it to take this long?   Should I just wait it out?   Like I said, my site stats show it crawling...