XML Sitemaps Generator

Author Topic: Cronjob on a file to launch crawler, hundred of crawler processes get started  (Read 8950 times)

info1304

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 13
I've been trying to find a way to start the crawler per cronjob but so far nothing has worked.

My webhoster advised me to create a separate php file which the cronjob should activate, which then will activate the crawler. Unfortunately this launches endless new processes of crawlers so that the server crashes.

The php file has the following entry:
Code: [external links are visible to admins only]
<?php
system
("/usr/local/php-5.3/bin/php /home/www/doc/123456/domain.com/www/sitemap/runcrawl.php");
?>


My webhoster has asked if in the configurations of the crawler it can be set that it only launches once and not over and over again...

Does anyone have an idea how to solve this ?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Hello,

if the script you quoted is started once, then generator will be started one time too. You should make sure that cron task is setup correctly (possibly you have set "*" for "minutes" setting, which means running every minute)
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2