cronjob
« on: October 26, 2009, 12:05:44 PM »
I create a cronjob on my server using Cron-5min.php (there's no cpanel or something along those lines)
What do I put into the Cron-5min.php file so that my site gets crawled?
Re: cronjob
« Reply #1 on: October 27, 2009, 04:02:07 PM »
Hello,

try to put this:
Code: [Select]
system('/usr/bin/php /path/to/your/generator/runcrawl.php');
Re: cronjob
« Reply #2 on: October 28, 2009, 10:20:14 AM »
Thanks for the reply,

Just to be sure I'm doing it correctly...

I got my file cron-5min.php
in this file I put the following:

Code: [Select]
<?php
system
('/usr/bin/php /path/to/my/generator/runcrawl.php');
?>

Because when I look at my generator tool in the crawling tab it says the command line should be the following:
Code: [Select]
/usr/local/bin/php /path/to/my/generator/runcrawl.php
Now I tried both but the tool isn't crawling.

The difference is in the 'local".
Re: cronjob
« Reply #3 on: October 28, 2009, 09:02:57 PM »
It's possible that php is located in a different folder in your case, you can ask your hosting support to find it out exactly.
Re: cronjob
« Reply #4 on: October 29, 2009, 01:02:42 PM »
Yeah I figured as much so I already mailed them. They seem to be finding it hard to grasp the concept of a command line instead of using curl... I'll get there though :)
Re: cronjob
« Reply #5 on: November 02, 2009, 08:44:49 AM »
After a fair amount of mailing they told me to put the following in the cronjob file, cron-5min.php:
Code: [Select]
<?php
system
('wget -O /dev/null http://www.my-domain.com/location/of/generator-folder/runcrawl.php');
?>

If I browse to my cronjob file it returns the following:
Malformed header from CGI script: => `/dev/null' Resolving [ External links are visible to forum administrators only ]... **.**.**.** Connecting to [ External links are visible to forum administrators only ]|**.**.**.**|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] 0K 4.86 MB/s 09:33:22 (4.86 MB/s) - `/dev/null' saved [51] X-Powered-By: PHP/5.2.6 Content-type: text/html

But the sitemap isn't updated nor is the "Request date" changed in the generator tool.

**EDIT**

After some more mailing, my host did some testing and they came up with the following:

Code: [Select]
<?php
system
('wget --delete-after http://www.my-domain.com/path/to/generator-tool/runcrawl.php');
?>

Which results into:
"I'd like to inform that the source of inconvenience seems to be in runcrawl.php file which cannot provide proper headers to php interpreter.
I afford to modify Your cron file, and replace actual wget command parameter to --delete-after. This action shows that header is malformed by runcrawl.php file.
Unfortunately, mentioned file is encrypted, so we cannot analyze its content.
We suggest You to contact author of the script to get more information concerning Your request."
« Last Edit: November 02, 2009, 09:42:27 AM by glimbeek »
Re: cronjob
« Reply #6 on: November 02, 2009, 09:41:42 PM »
You can replace runcrawl.php file with:
Code: [Select]
<?php
$op $_REQUEST['op'] = 'crawlproc';
$cmdlined true;
chdir(dirname(__FILE__));
$_REQUEST['bg'] = true;
$_REQUEST['resume'] = true;
include './index.php';
?>
Re: cronjob
« Reply #7 on: November 03, 2009, 07:05:55 AM »
Seems to be working. Thanks for the fast reply!