• Welcome to Sitemap Generator Forum.
 

Won't crawl if port number is present

Started by sales805, June 13, 2011, 11:49:35 PM

Previous topic - Next topic

sales805

I am trying to crawl pages from a module on my site that appends the port number to the site name. For example: [ External links are visible to forum administrators only ]

It appears that xml-sitemaps is handling this an an offsite link instead of removing the :80 and handling it as a local link.

Is there any way to convince xml-sitemaps to process these links correctly?

Thank you.

XML-Sitemaps Support

Hello,

please try to re-download generator package (I've just udpated it), reinstall it and it should work as expected.

sales805

I really appreciate the help, but it looks like there is a second problem with that new module that's preventing indexiing. Here is an example URL that's not indexed:

[ External links are visible to forum administrators only ]

You are welcome to crawl my entire site if it will help. There should be more than 300 pages.

[ External links are visible to forum administrators only ]


sales805

I have resolved the second problem. I didn't realize that XML-Sitemaps reads robots.txt.

Thank you very much for the new version to resolve the port problem. That was critical to getting everything running. Great support and I really appreciate it.