XML Sitemaps Generator

Author Topic: Unclear reason of removing URLs from sitemap  (Read 6364 times)

VladUA

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 11
Unclear reason of removing URLs from sitemap
« on: April 28, 2014, 01:45:28 PM »
Dear developers,

Could you help me to understand why some URLs was removed from a sitemap, please?

All removed URLs are correct:
[external links are visible to admins only]
[external links are visible to admins only]
[external links are visible to admins only]
[external links are visible to admins only]
[external links are visible to admins only]

Thank you in advance for your help,
Vlad

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Unclear reason of removing URLs from sitemap
« Reply #1 on: April 29, 2014, 06:05:31 AM »
Hello,

the typical reason would be that there are no links pointing to those pages anymore and generator bot cannot find them as a result.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

VladUA

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 11
Re: Unclear reason of removing URLs from sitemap
« Reply #2 on: May 02, 2014, 10:42:25 PM »
Dear Oleg,

But these URLs was included into the block "Skipped URLs - crawled but not added in sitemap".
If I correctly understood (based on the block description) these pages was found/crawled at the last run of a sitemap script, but was skipped. And for the each URL after symbol "-" must be specified reason of exclusion ("404 Not Found", "text/xml", "lu", "301 Moved Permanently" or other).
If my point of view is not correct could you please explain difference between blocks "Removed URLs" and "Skipped URLs - crawled but not added in sitemap".

All my pages are static and on the next run of the sitemap script these pages was found and included into sitemap.xml (please see attached image #1).

I have faced with this problem again, but with other URL (please see attached image #2):
[external links are visible to admins only]

This URL included into the list of URLs of the page [external links are visible to admins only] (please see attached image #3)

URL [external links are visible to admins only] included into sitemap.xml (please see attached picture #4)

May be problem in my configuration of the Sitemap Generator?

Thank you in advance for your help,
Vlad

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Unclear reason of removing URLs from sitemap
« Reply #3 on: May 03, 2014, 02:20:17 PM »
Hello,

it means that generator bot got an error when trying to fetch those pages (one of the reasons could be that website server was overloaded at that moment and couldn't handle the request).
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

VladUA

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 11
Re: Unclear reason of removing URLs from sitemap
« Reply #4 on: May 03, 2014, 05:37:12 PM »
Dear Oleg,

Is there ability definitely receive reason of the error?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Unclear reason of removing URLs from sitemap
« Reply #5 on: May 05, 2014, 07:04:32 AM »
Hello,

you can try to check your server error logs for details. From the generator's crawler side it appears as "connection failed" attempt.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2