XML Sitemaps Generator

Author Topic: Very few links indexed  (Read 12677 times)

jawks

  • Approved member
  • *
  • Posts: 3
Very few links indexed
« on: February 22, 2012, 08:38:54 PM »
Hello,

I've always used this tool to create sitemaps without problems. My site has hundreds of links and this sitemap creator worked fine with it.

Now I've used again the sitemap creator to create again the sitemap and it only finds 25 links....

What can be happening? My page is [external links are visible to admins only]

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Very few links indexed
« Reply #1 on: February 23, 2012, 07:20:07 PM »
Hello,

what is an example link that is not included in sitemap and how it can be reached, starting from homepage?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

jawks

  • Approved member
  • *
  • Posts: 3
Re: Very few links indexed
« Reply #2 on: March 09, 2012, 02:02:12 PM »
Only 7 links are being indexed.

Some examples of pages that should be indexed and included in the sitemap and are accessible from the homepage are:

[external links are visible to admins only]
[external links are visible to admins only]
[external links are visible to admins only]

Thanks for your help


XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Very few links indexed
« Reply #3 on: March 09, 2012, 03:03:08 PM »
Hello,

looks like your site returns 403 forbidden response when there many incoming requests from generator crawler. You can use http://www.xml-sitemaps.com/indexadv.html page to define a delay after certain number of requests to slow down crawling and avoid the mentioned above issue.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

jawks

  • Approved member
  • *
  • Posts: 3
Re: Very few links indexed
« Reply #4 on: March 12, 2012, 10:28:37 AM »
I've asked my hosting tech support about this and this is their answer (I think it doesn't make any sense. What do you think?):

----------

Hello,

  I have checked and noticed that you have placed a robots.txt file which blocks search crawlers to certain folders in your home directory. The 403 errors are due this 'forbidden' rules in your robots.txt file. You may please check this with the the help of a website developer.

----------

What do you think?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Very few links indexed
« Reply #5 on: March 13, 2012, 07:28:39 AM »
Hello,

403 http code should not be related to robots.txt.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2