XML Sitemaps Generator

Author Topic: meta nofollow but in sitemap  (Read 8762 times)

mathias

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
meta nofollow but in sitemap
« on: July 07, 2010, 08:34:58 PM »
hi!

all my pages i don't want to be in the sitemap.xml have a
<meta name="robots" content="noindex,nofollow" />
tag

but some of them are in the sitemap.xml.
is there something wrong with the meta tag or how does this work?
thanks
mat

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: meta nofollow but in sitemap
« Reply #1 on: July 08, 2010, 08:28:41 AM »
Hello,

what is an example URL that is included? Don't you have it specified in "Doi not parse" setting in sitemap generator configuration? (in this case it's still included)
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

mathias

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: meta nofollow but in sitemap
« Reply #2 on: July 08, 2010, 04:01:25 PM »
hi!
an example would be [external links are visible to admins only]
it has the meta tag in the source code: robots noindex, nofollow.

i automatically include this meta tag when a category page has 0 results. i don't want the searchengines to index these pages and so i don't want them to be in the sitemap too.

i can't add them manually to the exclude filter as this meta tag is only on pages with 0 results and they can differ from time to time as categories may have items in it in the future and others that have items now will not have them in the future maybe.

so the question is, does the generator look at the meta-tag, or ist the robots info on the page irrelevant. as i read, that robots.txt rules are followed by the generator i hoped that the page specific meta-tags are too ;)

thanks for your info about this!
mathias

mathias

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: meta nofollow but in sitemap
« Reply #3 on: July 09, 2010, 07:18:01 AM »
when i look at this post http://www.xml-sitemaps.com/forum/index.php/topic,1868.html
it seems that the meta robots tag should also work for the generator
why is it not working for me?
m

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: meta nofollow but in sitemap
« Reply #4 on: July 09, 2010, 08:59:06 AM »
Hello,

I just checked the URL you mentioned above and I don't see the robots meta tag in it's html source, please check that.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

mathias

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: meta nofollow but in sitemap
« Reply #5 on: July 09, 2010, 10:21:59 AM »
alright, try this one [external links are visible to admins only] - i see the meta tag in the source code
maybe the other page was updatet in the meantime, i don't know any more which url it was as it his just visible for admins ;)
thanks for recheck!

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: meta nofollow but in sitemap
« Reply #6 on: July 09, 2010, 11:53:28 AM »
Possibly the meta tag was not there at the moment when sitemap generator was crawloing your site? You can regenerate sitemap to check that.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

mathias

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: meta nofollow but in sitemap
« Reply #7 on: July 09, 2010, 02:34:19 PM »
i rechecked. the tag was there... what can the problem else be?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: meta nofollow but in sitemap
« Reply #8 on: July 10, 2010, 02:41:30 PM »
You can regenerate sitemap to check that.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

mathias

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: meta nofollow but in sitemap
« Reply #9 on: July 13, 2010, 02:28:29 PM »
yeah, you already said that!
i can recheck and find what it is. i just want to knwo if it is a feature of the generator to automatically exclude pages with a robots noindex attribute. if it should work like this - i will find out as the problem seems to be on my side...

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: meta nofollow but in sitemap
« Reply #10 on: July 14, 2010, 03:56:11 PM »
Yes, definitely - sitemap generator excludes pages with robots noindex meta tag (unless you have those pages specified in "do not parse" setting, as mentioend above)
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2