XML Sitemaps Generator

Author Topic: Sitemap removing URLs with each crawl  (Read 14661 times)


  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Sitemap removing URLs with each crawl
« on: September 29, 2008, 05:24:33 AM »
I upgraded to the new sitemap and I have noticed that with the new sitemap generator it is always removing URLs everytime it crawls!! Any reason why this is happening?

BTW the removed URLs are not broken links and are not reported as such in the broken links tab.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10601
Re: Sitemap removing URLs with each crawl
« Reply #1 on: September 30, 2008, 12:20:07 AM »

are those pages still accessible starting from homepage (i.e. there is a way to find them via crawling).
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.


  • Registered Customer
  • Jr. Member
  • *
  • Posts: 34
Re: Sitemap removing URLs with each crawl
« Reply #2 on: October 25, 2008, 09:24:30 PM »
To further what Admin has said,

If you have a page which is no longer linked, it will remove the URL, an easy fix is to add a link to your html sitemap, and the crawler will read the sitemap page, so even if you remove the link to that page, the crawler will not remove the URL as it finds the URL in the sitemap. If you delete pages, and remove the links, the crawler will see the broken links in the sitemap.html and thus remove them.


SMF 2.0.12 | SMF © 2014, Simple Machines