I only want canonical url's in my sitemap - Please Help
« on: September 05, 2013, 06:22:46 PM »
I am running the latest version of the sitemap generator.

Here is my issue. My website has many pages with paging added. For example.
[ External links are visible to forum administrators only ]   
The above is my "canonical" url

But then many pages I will have paging like this.
[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]

and so on. The issue is I do not want to include the sub paging in my sitemap. I only want to display in my sitemap the canonical url like this.
[ External links are visible to forum administrators only ]

So in the sitemap admin page if I exclude the variable (/?) It will take out all the sub pages like I want.

But the issue is all the content (products) listed an all those sub pages with paging are excluded because the crawler will not crawl the paging because I am excluding that variable.

How can I tell the crawler to not include all those sub pages with the paging but DO include all the products on those sub pages??? I have tried everything I can to work around but no luck.

Any help would be appreciated!
Thanks
Re: I only want canonical url's in my sitemap - Please Help
« Reply #1 on: September 08, 2013, 08:16:17 AM »
Hello,

you can add robots meta tag on those pages:
<meta name="robots" content="noindex,follow" />

i.e., it will follow all links form those pages, but will not index pages themselves.
Re: I only want canonical url's in my sitemap - Please Help
« Reply #2 on: September 18, 2013, 06:24:47 PM »
you mean the sitemap crawler???