XML Sitemaps Generator

Author Topic: Sitemap Generator Trying to index and follow malformed links  (Read 7001 times)

john

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Sitemap Generator Trying to index and follow malformed links
« on: December 02, 2012, 04:59:58 PM »
Over the years my content site accumilated some articles with malformed links like the example below.  If someone clicks one of these links it redirects to the same page.   They are scattered among several thousand articles so it is next to impossible to find and remove them all.

Anyway, these links have never caused a problem with sitemap generator until this week.  In noticed instead of finishing up in around 4 hours with about 90K pages, it was running 10+ hours and up to 200K page and still going.  I paused the generator and notice that it is indexing these malformed html links.

Below is an example link and what sitemap generator is doing with them.  How can I get sitemap generator to ignore these links like it did before.  I don't think I changed any settings so I am not sure why this is a problem now...

Example of a typical malformed link:

[external links are visible to admins only].*****.com/article.php/111/1111/Golf/www.mcontractors.com/drainage/www.mcontractors.m/natural/

Sitemap generator is indexing:

article.php/111/1111/Golf/www.mcontractors.com/drainage/www.mcontractors.com/drainage/www.mcontractors.com/natural/www.mcontractors.com/natural/www.mcontractors.com/natural/www.mcontractors.com/drainage/www.mcontractors.com/drainage/www.mcontractors.com/drainage/www.mcontractors.com/natural/www.mcontractors.com/natural/www.mcontractors.com/drainage/www.mcontractors.com/natural/www.mcontractors.com/natural/www.mcontractors.com/natural/ (0.0)


XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10621
Re: Sitemap Generator Trying to index and follow malformed links
« Reply #1 on: December 02, 2012, 07:11:43 PM »
Hello,

please try to add "/www" in Exclude URLs setting.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

john

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: Sitemap Generator Trying to index and follow malformed links
« Reply #2 on: December 02, 2012, 11:53:29 PM »
Thanks, giving it a try. 

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2