XML Sitemaps Generator

Author Topic: Does sitemap generator ignore url's that don't exist as actual files?  (Read 12062 times)

hostasaurus

  • Approved member
  • *
  • Posts: 2
I've got a site that uses virtual url's handled via .htaccess similar to how wordpress does with permalinks enabled.  Sitemap generator appears to have no problem crawling them, as verified by the site web access log, but it does refuse to index them for some reason.  For example:

Current page: product/GEMINIJETS/GJAML700.html
Pages added to sitemap: 41
Pages scanned: 2300 (26,398.8 KB)

That file does not actually exist, nor is there a products directory, but if you request that link on the site you get the page and a status code 200 since the .htaccess rewrites the request into something else internally and serves the file.  Sitemap generator is crawling these virtual pages, as shown in the above count of 2300 pages so far, but it is not indexing them.  .html extensions are not excluded nor is any URL in the config.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Does sitemap generator ignore url's that don't exist as actual files?
« Reply #1 on: January 05, 2011, 11:45:34 PM »
Hello,

I just checked your sitemap (referenced in http://www.xml-sitemaps.com/forum/index.php/topic,5394.html ) and that URL is included in sitemap.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

hostasaurus

  • Approved member
  • *
  • Posts: 2
Re: Does sitemap generator ignore url's that don't exist as actual files?
« Reply #2 on: January 05, 2011, 11:51:59 PM »
Oops, didn't realize our customer had opened a ticket too.  The issue ended up being a canonical meta tag that was being added to the pages, causing the sitemap generator to store that url instead of the one it had crawled.  Updated the template and now all is well.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2