Canonical links don't show up in sitemap.xml
« on: March 25, 2018, 01:57:23 PM »
I make use of canonical links on my website a lot and was happy to see that xml-sitemaps seem to support them.
Unfortunately it doesn't work or not like I thought it would.

For example on my site [ External links are visible to forum administrators only ] I have a database with more than 5000 mobile phones.
Each phone is linked this way: [ External links are visible to forum administrators only ]
The link is transformed into [ External links are visible to forum administrators only ] be mod rewrite.
On the page there is a canonical entry to [ External links are visible to forum administrators only ]
I was assuming that I will get the canonical entry into the XML file.
While the canonical entries are shown as being checked while crawling I have no entries in the resulting xml file. On the other hand I have multiple [ External links are visible to forum administrators only ] entries.

You may check my page on your own to see what I mean.
Re: Canonical links don't show up in sitemap.xml
« Reply #1 on: March 25, 2018, 04:54:05 PM »
Hello,

canonical URL should not be redirected to other page, otherwise it won't get indexed.
Re: Canonical links don't show up in sitemap.xml
« Reply #2 on: March 25, 2018, 06:50:01 PM »
Why? Google does index canonical links which redirect to other pages.
Google for "skyynet/db/Nokia/6310i" and you'll find what I'm intending to get indexed.

Re: Canonical links don't show up in sitemap.xml
« Reply #3 on: March 26, 2018, 04:42:47 AM »
Hello,

sitemap generator includes only links that return content with http code 200.
Re: Canonical links don't show up in sitemap.xml
« Reply #4 on: March 26, 2018, 09:37:04 PM »
I understand. Any chance to make that optional? As Google indexes the canonical links which return a 301 Moved permanently I don't see a reason for XML Sitemaps not to support this.
Re: Canonical links don't show up in sitemap.xml
« Reply #5 on: March 27, 2018, 04:10:04 AM »
Hello,

you can add "db/" in "Add directly in sitemap" setting - in this case those pages will be indexed without fetching from server.
Re: Canonical links don't show up in sitemap.xml
« Reply #6 on: March 28, 2018, 09:20:54 PM »
db/ is not a real directory but only an indicator for mod rewrite to transform the url
[ External links are visible to forum administrators only ]
to
[ External links are visible to forum administrators only ]

what is the db/ entry supposed to improve?

I just think that the canonical feature is crippled by design when you don't support 301 Moved Permanently answers.
Why not allow this (optionally)?

Also the crawling process supports a maximum script run time which sounds good for shared hosters. Unfortunately the crawling is not continued by the script but has to be done by a browser add-on. This also has to be improved.

Right now these are the two points which force me to do the crawling manually instead of regular jobs.
Re: Canonical links don't show up in sitemap.xml
« Reply #7 on: March 29, 2018, 04:39:13 PM »
Hello,

Quote
what is the db/ entry supposed to improve?
That would instruct generator to include those pages in sitemap directly.