Crawl but dont list?
« on: February 01, 2009, 08:52:23 PM »
I have a set of pages (X) that contain links to other sub-pages (Y). I want all of X and Y crawled, but I only want Y to show up in the sitemap, not X. Y are not linked to from any other pages than X.

Is this possible?

Re: Crawl but dont list?
« Reply #1 on: February 02, 2009, 01:27:13 AM »

please try to add this in html code of X pages:
<meta name="robots" content="noindex,follow" />