Site not getting crawled
« on: March 07, 2008, 07:58:12 PM »
I have a site that has had authentication for approximately 2 months. Naturally, google bot will not be able to crawl it. So in the google webmaster tools, it saids 401 error. After removing authentication and submitted sitemap. I still have not been re-crawled. It has been approximately 6 days since I removed authentication and it has been approximately 1 or 2 days since my sitemap has been submitted.
Re: Site not getting crawled
« Reply #1 on: March 07, 2008, 11:09:26 PM »
If you had your site closed for a long period, I would expect search engines to be on a slow side when re-crawling your site now (until they "figure out" that it's open for indexing now).