Sitemap Generator Forum

Hello,

I've always used this tool to create sitemaps without problems. My site has hundreds of links and this sitemap creator worked fine with it.

Now I've used again the sitemap creator to create again the sitemap and it only finds 25 links....

What can be happening? My page is [ External links are visible to forum administrators only ]

Hello,

what is an example link that is not included in sitemap and how it can be reached, starting from homepage?

Only 7 links are being indexed.

Some examples of pages that should be indexed and included in the sitemap and are accessible from the homepage are:

[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]

Thanks for your help

Hello,

looks like your site returns 403 forbidden response when there many incoming requests from generator crawler. You can use https://www.xml-sitemaps.com/indexadv.html page to define a delay after certain number of requests to slow down crawling and avoid the mentioned above issue.

I've asked my hosting tech support about this and this is their answer (I think it doesn't make any sense. What do you think?):

----------

Hello,

  I have checked and noticed that you have placed a robots.txt file which blocks search crawlers to certain folders in your home directory. The 403 errors are due this 'forbidden' rules in your robots.txt file. You may please check this with the the help of a website developer.

----------

What do you think?

Hello,

403 http code should not be related to robots.txt.