XML Sitemaps Generator

Author Topic: The whole site is not getting crawled  (Read 34485 times)

cds

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
The whole site is not getting crawled
« on: January 30, 2006, 01:46:35 AM »

Hi guys,

I've just bought the standalone site generator and I'm having trouble trying to gent completely crawled.

For example:

[external links are visible to admins only]

Has about 21 links, and only these are in the urllist.txt file:

[external links are visible to admins only]
[external links are visible to admins only]

Also, there are some URL's that I need to know where are they coming from (referer). Is there a way to know it? I haven't been able to find it.

Best regards,

CÚsar Enrique


XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: The whole site is not getting crawled
« Reply #1 on: January 30, 2006, 02:01:43 AM »
Hello,

please try to redownload the generator again (using the same link) and install it.
Let me know if that helps.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

cds

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
Re: The whole site is not getting crawled
« Reply #2 on: January 30, 2006, 02:56:39 AM »

Hello,

It  worked!. Thanks a lot.

I'll let you know how's everytghing going. So far it's ok.

Regards

CÚsar Enrique

cds

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
Re: The whole site is not getting crawled (Date Problem)
« Reply #3 on: January 30, 2006, 01:25:09 PM »
Hello Again,

I'm checking my the xml generated sitemap and I see some dates like this:

- <url>
  <loc>[external links are visible to admins only]</loc>
  <priority>0.9</priority>
  <lastmod>2008-05-01T00:00:00+00:00</lastmod>
  <changefreq>weekly</changefreq>
  </url>
- <url>
- <url>
  <loc>[external links are visible to admins only]</loc>
  <priority>0.9</priority>
  <lastmod>2008-05-01T00:00:00+00:00</lastmod>
  <changefreq>weekly</changefreq>
  </url>
+ <url>

My server date is ok, and actually, the file "listados" is the same for both url, so I think it should be a problem in the scripts.

Please give me a feedback. Best regards,

CÚsar Enrique

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: The whole site is not getting crawled
« Reply #4 on: January 30, 2006, 03:43:22 PM »
Hi,

please PM me your Generator instance URL so that I can check it.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

gillybrat

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Re: The whole site is not getting crawled
« Reply #5 on: February 06, 2006, 02:54:14 PM »
I am having the same problem. I've redownloaded the script, reinstalled it, run it, and getting the same results.

I've checked my root folder, no robot.txt in sight...

Any ideas?

Thanks ???

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: The whole site is not getting crawled
« Reply #6 on: February 06, 2006, 08:14:58 PM »
Hello,


please PM me the access details for your Sitemap Generator instance and let me know the examples of URLs that are not included in sitemap.

Thank you!
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

gillybrat

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Re: The whole site is not getting crawled
« Reply #7 on: February 06, 2006, 10:34:39 PM »
Actually, I figured it out on my own....  Nothing wrong with the script, it was the moron using it!  LOL :D

Thanks though,
Anne

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: The whole site is not getting crawled
« Reply #8 on: February 07, 2006, 10:11:35 PM »
Hi Anne,

Ah, I'm glad it works fine! :)
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2