cds

*
  • *
  • 3
The whole site is not getting crawled
« on: January 30, 2006, 01:46:35 AM »

Hi guys,

I've just bought the standalone site generator and I'm having trouble trying to gent completely crawled.

For example:

http://www.hoteles.com.ve/localidades

Has about 21 links, and only these are in the urllist.txt file:

http://www.hoteles.com.ve/localidades/margarita
http://www.hoteles.com.ve/en/localidades/margarita

Also, there are some URL's that I need to know where are they coming from (referer). Is there a way to know it? I haven't been able to find it.

Best regards,

CÚsar Enrique

Re: The whole site is not getting crawled
« Reply #1 on: January 30, 2006, 02:01:43 AM »
Hello,

please try to redownload the generator again (using the same link) and install it.
Let me know if that helps.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

cds

*
  • *
  • 3
Re: The whole site is not getting crawled
« Reply #2 on: January 30, 2006, 02:56:39 AM »

Hello,

It  worked!. Thanks a lot.

I'll let you know how's everytghing going. So far it's ok.

Regards

CÚsar Enrique

cds

*
  • *
  • 3
Re: The whole site is not getting crawled (Date Problem)
« Reply #3 on: January 30, 2006, 01:25:09 PM »
Hello Again,

I'm checking my the xml generated sitemap and I see some dates like this:

- <url>
  <loc>http://www.hoteles.com.ve/listados/anzoategui/puerto_la_cruz/6/</loc>
  <priority>0.9</priority>
  <lastmod>2008-05-01T00:00:00+00:00</lastmod>
  <changefreq>weekly</changefreq>
  </url>
- <url>
- <url>
  <loc>http://www.hoteles.com.ve/listados/anzoategui/puerto_la_cruz/6/</loc>
  <priority>0.9</priority>
  <lastmod>2008-05-01T00:00:00+00:00</lastmod>
  <changefreq>weekly</changefreq>
  </url>
+ <url>

My server date is ok, and actually, the file "listados" is the same for both url, so I think it should be a problem in the scripts.

Please give me a feedback. Best regards,

CÚsar Enrique
Re: The whole site is not getting crawled
« Reply #4 on: January 30, 2006, 03:43:22 PM »
Hi,

please PM me your Generator instance URL so that I can check it.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: The whole site is not getting crawled
« Reply #5 on: February 06, 2006, 02:54:14 PM »
I am having the same problem. I've redownloaded the script, reinstalled it, run it, and getting the same results.

I've checked my root folder, no robot.txt in sight...

Any ideas?

Thanks ???
Re: The whole site is not getting crawled
« Reply #6 on: February 06, 2006, 08:14:58 PM »
Hello,


please PM me the access details for your Sitemap Generator instance and let me know the examples of URLs that are not included in sitemap.

Thank you!
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: The whole site is not getting crawled
« Reply #7 on: February 06, 2006, 10:34:39 PM »
Actually, I figured it out on my own....  Nothing wrong with the script, it was the moron using it!  LOL :D

Thanks though,
Anne
Re: The whole site is not getting crawled
« Reply #8 on: February 07, 2006, 10:11:35 PM »
Hi Anne,

Ah, I'm glad it works fine! :)
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.