I've been trying to find a way to start the crawler per cronjob but so far nothing has worked.

My webhoster advised me to create a separate php file which the cronjob should activate, which then will activate the crawler. Unfortunately this launches endless new processes of crawlers so that the server crashes.

The php file has the following entry:
Code: [Select]
<?php
system
("/usr/local/php-5.3/bin/php /home/www/doc/123456/domain.com/www/sitemap/runcrawl.php");
?>


My webhoster has asked if in the configurations of the crawler it can be set that it only launches once and not over and over again...

Does anyone have an idea how to solve this ?
Hello,

if the script you quoted is started once, then generator will be started one time too. You should make sure that cron task is setup correctly (possibly you have set "*" for "minutes" setting, which means running every minute)