Sitemap removing URLs with each crawl
« on: September 29, 2008, 06:24:33 AM »
I upgraded to the new sitemap and I have noticed that with the new sitemap generator it is always removing URLs everytime it crawls!! Any reason why this is happening?

BTW the removed URLs are not broken links and are not reported as such in the broken links tab.
Re: Sitemap removing URLs with each crawl
« Reply #1 on: September 30, 2008, 01:20:07 AM »
Hello,

are those pages still accessible starting from homepage (i.e. there is a way to find them via crawling).
Re: Sitemap removing URLs with each crawl
« Reply #2 on: October 25, 2008, 10:24:30 PM »
To further what Admin has said,

If you have a page which is no longer linked, it will remove the URL, an easy fix is to add a link to your html sitemap, and the crawler will read the sitemap page, so even if you remove the link to that page, the crawler will not remove the URL as it finds the URL in the sitemap. If you delete pages, and remove the links, the crawler will see the broken links in the sitemap.html and thus remove them.
Rob