XML Sitemaps Generator

Author Topic: should a robots.txt file block URL's from being generated by the sitemap?  (Read 11416 times)

Phaze

  • Approved member
  • *
  • Posts: 1
Hello,

I have recently generated a sitemap and I am having some issues with it. For some reason, when I create the sitemap, I am getting a URL as if the products on my site have been added to my shopping cart, as well as the normal store URL's. Here's an example:

<url>
  <loc>[external links are visible to admins only]</loc>
  <changefreq>weekly</changefreq>
  <priority>0.64</priority>
</url>
<url>
  <loc>[external links are visible to admins only]</loc>
  <changefreq>weekly</changefreq>
  <priority>0.64</priority>
</url>

Any ides how this could be happening? This is happening for most of my products on my site. I do have my shopping cart and checkout page being limited by my robots.txt file, but that doesn't appear to effect the creation of the xml sitemap (but should it?).

Any help here would be greatly appreciated.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Hello,

I see that "add=1" URLs are not included in sitemap: http://www.xml-sitemaps.com/details-www.rcplanesandcopters.com.html
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2