XML Sitemaps Generator

Author Topic: Problema running the script via Crontab  (Read 8102 times)

webmaster254

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Problema running the script via Crontab
« on: July 03, 2012, 01:52:03 PM »
Hello Forum, I would like to discuss here a strange situation I am experiencing with XML-Sitemaps Standalone Generator.

Firstly, the script is currently installed and fully working.

SCENARIO ONE
With the script installed, I manually run with a SSH session the following command:
/usr/bin/php /var/www/mysite.com/generator/runcrawl.php
The result is 2560 URL in sitemap.

SCENARIO TWO
The same script installed. I have set in CRONTAB the very same command:
/usr/bin/php /var/www/mysite.com/generator/runcrawl.php (the command runs at 3AM)
The result is 1950 URL in sitemap.

THE QUESTION IS: why running the very same script with a crontab command, it yields a less number of URL in sitemap?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Problema running the script via Crontab
« Reply #1 on: July 04, 2012, 04:16:58 PM »
Hello,

is the result consistent? i.e. if you run it in ssh session again, it will create 2560 URLs sitemap again?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

webmaster254

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Problema running the script via Crontab
« Reply #2 on: July 05, 2012, 07:11:36 AM »
Yes, everytime I run via SSH I get 2560 URLs.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Problema running the script via Crontab
« Reply #3 on: July 05, 2012, 01:06:07 PM »
Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached starting from homepage?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

webmaster254

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Problema running the script via Crontab
« Reply #4 on: July 06, 2012, 08:41:14 AM »
OKAY, start from the homepage:

1) [external links are visible to admins only] then click on GIOCARE MODERNO
2) [external links are visible to admins only] then click on Squadre FUORI CATALOGO
3) [external links are visible to admins only] then click on SUCCESSIVO (the blu arrow for "next" 63 pages)

[external links are visible to admins only]
THIS PAGE IS NOT in sitemap.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Problema running the script via Crontab
« Reply #5 on: July 06, 2012, 07:58:54 PM »
If you check the html source of the page, the pagination link looks like:
<a class="pagina_successiva" href="/page/1-20/2-20" title="Successivo">Successivo

i.e. points to domain root, and later it's corrected with javascript, which crawler bots will not see.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2