Over the years my content site accumilated some articles with malformed links like the example below. If someone clicks one of these links it redirects to the same page. They are scattered among several thousand articles so it is next to impossible to find and remove them all.
Anyway, these links have never caused a problem with sitemap generator until this week. In noticed instead of finishing up in around 4 hours with about 90K pages, it was running 10+ hours and up to 200K page and still going. I paused the generator and notice that it is indexing these malformed html links.
Below is an example link and what sitemap generator is doing with them. How can I get sitemap generator to ignore these links like it did before. I don't think I changed any settings so I am not sure why this is a problem now...
Example of a typical malformed link:
[ External links are visible to forum administrators only ].*****.com/article.php/111/1111/Golf/www.mcontractors.com/drainage/www.mcontractors.m/natural/
Sitemap generator is indexing:
article.php/111/1111/Golf/www.mcontractors.com/drainage/www.mcontractors.com/drainage/www.mcontractors.com/natural/www.mcontractors.com/natural/www.mcontractors.com/natural/www.mcontractors.com/drainage/www.mcontractors.com/drainage/www.mcontractors.com/drainage/www.mcontractors.com/natural/www.mcontractors.com/natural/www.mcontractors.com/drainage/www.mcontractors.com/natural/www.mcontractors.com/natural/www.mcontractors.com/natural/ (0.0)
Guest