• Welcome to Sitemap Generator Forum.
 

Unclear reason of removing URLs from sitemap

Started by VladUA, April 28, 2014, 02:45:28 PM

Previous topic - Next topic

VladUA

Dear developers,

Could you help me to understand why some URLs was removed from a sitemap, please?

All removed URLs are correct:
[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]

Thank you in advance for your help,
Vlad

XML-Sitemaps Support

Hello,

the typical reason would be that there are no links pointing to those pages anymore and generator bot cannot find them as a result.

VladUA

Dear Oleg,

But these URLs was included into the block "Skipped URLs - crawled but not added in sitemap".
If I correctly understood (based on the block description) these pages was found/crawled at the last run of a sitemap script, but was skipped. And for the each URL after symbol "-" must be specified reason of exclusion ("404 Not Found", "text/xml", "lu", "301 Moved Permanently" or other).
If my point of view is not correct could you please explain difference between blocks "Removed URLs" and "Skipped URLs - crawled but not added in sitemap".

All my pages are static and on the next run of the sitemap script these pages was found and included into sitemap.xml (please see attached image #1).

I have faced with this problem again, but with other URL (please see attached image #2):
[ External links are visible to forum administrators only ]

This URL included into the list of URLs of the page [ External links are visible to forum administrators only ] (please see attached image #3)

URL [ External links are visible to forum administrators only ] included into sitemap.xml (please see attached picture #4)

May be problem in my configuration of the Sitemap Generator?

Thank you in advance for your help,
Vlad

XML-Sitemaps Support

Hello,

it means that generator bot got an error when trying to fetch those pages (one of the reasons could be that website server was overloaded at that moment and couldn't handle the request).

VladUA

Dear Oleg,

Is there ability definitely receive reason of the error?

XML-Sitemaps Support

Hello,

you can try to check your server error logs for details. From the generator's crawler side it appears as "connection failed" attempt.