XML Sitemaps Generator

Author Topic: Setting up an automatic daily crawl  (Read 23978 times)

nickm

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 20
Setting up an automatic daily crawl
« on: November 09, 2006, 11:32:23 AM »
No doubt so one has already asked this, or its so obvious as to not require explanation, but I'd like toset up an automatic daily crawl. It doesn't have to be daily - merely a time period driven crawl.

Please be sure to assume no prior knowledge when answering:)

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Setting up an automatic daily crawl
« Reply #1 on: November 10, 2006, 09:47:54 PM »
Hello,

basically, scheduled tasks on Unix servers are handled with "cron jobs" that can be configured in shell command line: http://www.google.com/search?ie=UTF-8&oe=UTF-8&sourceid=navclient&gfns=1&q=crontab

But in most cases this also can be done within hosting control panel.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

nickm

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 20
Re: Setting up an automatic daily crawl
« Reply #2 on: January 05, 2007, 05:31:59 AM »
I've had our ISP set up a Cron Job on the Server we use as follows:

0 6 * * * /usr/bin/php /home/www/html-data/sitemap_generator/runcrawl.php

We get the following error message:

/bin/sh: line 1: /usr/bin/php: No such file or directory

What are we doing wrong or, more accurately, what have we advised our ISP to do that's wrong!

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Setting up an automatic daily crawl
« Reply #3 on: January 05, 2007, 10:31:33 PM »
It means that php location is different on your server. it might be "/usr/local/bin/php", but your hosting support should know better.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2