XML Sitemaps Generator

Author Topic: Google crawler is producing 400 and 404 errors  (Read 11654 times)

ecevikel

  • Approved member
  • *
  • Posts: 3
Google crawler is producing 400 and 404 errors
« on: October 20, 2009, 02:19:15 PM »
I ran xml-sitemap tool to generate sitemap.xml then load it to the root of my website [external links are visible to admins only]
I did configured Google+Site Configuration+Sitemaps to use sitemap.xml

Crawler Access is setup as:
User-agent: *
Disallow: /aspnet_client/
Disallow: /Bin/
Disallow: /App_Code/
Disallow: /Common/
Allow: /*
Sitemap: [external links are visible to admins only]
Sitemap: [external links are visible to admins only]
Sitemap: [external links are visible to admins only]
Sitemap: [external links are visible to admins only]
Sitemap: [external links are visible to admins only]

I m using Google's recommended crawle rate

There are 4 HTML (400 errors), 8 In Sitemap (404 errors), 14 Not Found (404 errors)

Thank you in advance for your help.

ecevikel

  • Approved member
  • *
  • Posts: 3
Re: Google crawler is producing 400 and 404 errors
« Reply #1 on: October 20, 2009, 09:19:59 PM »
[external links are visible to admins only]
I ran xml-sitemap tool to generate sitemap.xml then load it to the root of my website [external links are visible to admins only]
I did configured Google+Site Configuration+Sitemaps to use sitemap.xml

Crawler Access is setup as:
User-agent: *
Disallow: /aspnet_client/
Disallow: /Bin/
Disallow: /App_Code/
Disallow: /Common/
Allow: /*
Sitemap: [external links are visible to admins only]
Sitemap: [external links are visible to admins only]
Sitemap: [external links are visible to admins only]
Sitemap: [external links are visible to admins only]
Sitemap: [external links are visible to admins only]

I m using Google's recommended crawle rate

There are 4 HTML (400 errors), 8 In Sitemap (404 errors), 14 Not Found (404 errors)

Thank you in advance for your help.


XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Google crawler is producing 400 and 404 errors
« Reply #2 on: October 20, 2009, 10:21:23 PM »
Hello,

you should submit only sitemap.xml to Google.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

ecevikel

  • Approved member
  • *
  • Posts: 3
Re: Google crawler is producing 400 and 404 errors
« Reply #3 on: October 23, 2009, 01:45:54 PM »
As you suggessted, I changed it to have only sitemap.xml as shown below.  But nothing has changed. Still, there are 4 HTML (400 errors), 8 In Sitemap (404 errors), 14 Not Found (404 errors).  And, all of these error links are working when I click on them.

User-agent: *
Disallow: /aspnet_client/
Disallow: /Bin/
Disallow: /App_Code/
Disallow: /Common/
Allow: /*
Sitemap: [external links are visible to admins only]

Thank you for all your help.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Google crawler is producing 400 and 404 errors
« Reply #4 on: October 25, 2009, 08:04:31 AM »
Hello,

make sure that you check errors in webmaster account->Site configuration->Sitemaps->Click opn sitemap name. Issues reported in other areas of google webmaster account are not related to sitemap files.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

DenT44

  • Guest
Re: Google crawler is producing 400 and 404 errors
« Reply #5 on: November 06, 2009, 12:24:05 AM »
Yeah, this could be totally unrelated. Funny, I sometimes get 404 errors from mozilla bot.

landdesignguy

  • Guest
Re: Google crawler is producing 400 and 404 errors
« Reply #6 on: November 06, 2009, 01:11:38 AM »
lots of error it seems

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2