Large Sitemap
« on: February 25, 2019, 01:42:58 AM »
While Using Cron to run the sitemap crawler at midnight, it runs for about 1 to 2 hours and then stops or is shutoff by the server.  Can I setup a second cron job to run the same night a couple of hours later.  Will it continue from where it left off or would it start over.  I know when I use the manual cron link it runs for about 45 minutes and then stops,  I can either hit the refresh on the Chrome or just rehit the link.  The site takes almost 7 hours over running time to finish the sitemap.  Only 3 images on the site,  but there are 16,ooo posts.
Re: Large Sitemap
« Reply #1 on: February 25, 2019, 06:29:43 AM »
Hello,

yes, you can setup another cron task in this case and it will resume the process automatically.
Re: Large Sitemap
« Reply #2 on: March 01, 2019, 11:56:28 PM »
I setup 3 cron jobs 1 at midnight, next at 3 am, and the last at 6 am.  Those are my times.  I looked at the generator page at 8:30a.m. my time that is EST.  And the page says it had been running at 32 minutes before interruption.  No member of previous 4 hours of cron jobs.  Now does /usr/bin/php /home1/theliwg2/public_html/recipes4thecook/generator/runcrawl.php continue privious crawl or does it start over?
Re: Large Sitemap
« Reply #3 on: March 03, 2019, 11:18:08 AM »
It continues the process. Make sure that "Save the script state" setting is not set to 0 (in this case state is not saved).