Competitor Analysis Tool: Analyze rival websites and gain valuable SEO and PPC insights
« previous next »
Pages: 1

ecevikel

*
  • *
  • 3
Google crawler is producing 400 and 404 errors
« on: October 20, 2009, 03:19:15 PM »
I ran xml-sitemap tool to generate sitemap.xml then load it to the root of my website [ External links are visible to forum administrators only ]
I did configured Google+Site Configuration+Sitemaps to use sitemap.xml

Crawler Access is setup as:
User-agent: *
Disallow: /aspnet_client/
Disallow: /Bin/
Disallow: /App_Code/
Disallow: /Common/
Allow: /*
Sitemap: [ External links are visible to forum administrators only ]
Sitemap: [ External links are visible to forum administrators only ]
Sitemap: [ External links are visible to forum administrators only ]
Sitemap: [ External links are visible to forum administrators only ]
Sitemap: [ External links are visible to forum administrators only ]

I m using Google's recommended crawle rate

There are 4 HTML (400 errors), 8 In Sitemap (404 errors), 14 Not Found (404 errors)

Thank you in advance for your help.
Logged

ecevikel

*
  • *
  • 3
Re: Google crawler is producing 400 and 404 errors
« Reply #1 on: October 20, 2009, 10:19:59 PM »
Quote from: ecevikel on October 20, 2009, 03:19:15 PM
I ran xml-sitemap tool to generate sitemap.xml then load it to the root of my website [ External links are visible to forum administrators only ]
I did configured Google+Site Configuration+Sitemaps to use sitemap.xml

Crawler Access is setup as:
User-agent: *
Disallow: /aspnet_client/
Disallow: /Bin/
Disallow: /App_Code/
Disallow: /Common/
Allow: /*
Sitemap: [ External links are visible to forum administrators only ]
Sitemap: [ External links are visible to forum administrators only ]
Sitemap: [ External links are visible to forum administrators only ]
Sitemap: [ External links are visible to forum administrators only ]
Sitemap: [ External links are visible to forum administrators only ]

I m using Google's recommended crawle rate

There are 4 HTML (400 errors), 8 In Sitemap (404 errors), 14 Not Found (404 errors)

Thank you in advance for your help.

Logged

XML-Sitemaps Support

*
  • *****
  • 11780
Re: Google crawler is producing 400 and 404 errors
« Reply #2 on: October 20, 2009, 11:21:23 PM »
Hello,

you should submit only sitemap.xml to Google.
Logged

ecevikel

*
  • *
  • 3
Re: Google crawler is producing 400 and 404 errors
« Reply #3 on: October 23, 2009, 02:45:54 PM »
As you suggessted, I changed it to have only sitemap.xml as shown below.  But nothing has changed. Still, there are 4 HTML (400 errors), 8 In Sitemap (404 errors), 14 Not Found (404 errors).  And, all of these error links are working when I click on them.

User-agent: *
Disallow: /aspnet_client/
Disallow: /Bin/
Disallow: /App_Code/
Disallow: /Common/
Allow: /*
Sitemap: [ External links are visible to forum administrators only ]

Thank you for all your help.
Logged

XML-Sitemaps Support

*
  • *****
  • 11780
Re: Google crawler is producing 400 and 404 errors
« Reply #4 on: October 25, 2009, 08:04:31 AM »
Hello,

make sure that you check errors in webmaster account->Site configuration->Sitemaps->Click opn sitemap name. Issues reported in other areas of google webmaster account are not related to sitemap files.
Logged

DenT44

*
Re: Google crawler is producing 400 and 404 errors
« Reply #5 on: November 06, 2009, 12:24:05 AM »
Yeah, this could be totally unrelated. Funny, I sometimes get 404 errors from mozilla bot.
Logged

landdesignguy

*
Re: Google crawler is producing 400 and 404 errors
« Reply #6 on: November 06, 2009, 01:11:38 AM »
lots of error it seems
Logged
Pages: 1
« previous next »