• Welcome to Sitemap Generator Forum.
 

how to scan password protected sites

Started by ohadziv, May 25, 2009, 02:14:32 PM

Previous topic - Next topic

ohadziv

Hi
I have a site for members only.
after you type the url of my site you have to insert login details (.htaccess)
How can I scan it

10x

XML-Sitemaps Support

Hello,

you should allow accessing your website without login/password, since even if you will create a sitemap, Google will still NOT index your pages if thir bot is unable to open your pages.

sales499

I have same issue for [ External links are visible to forum administrators only ]
Only few pages have free access, all others are under user/password access.

Is it possible to provide the master user/password to your script so it can index the pages?

XML-Sitemaps Support

Please check the post above - even if you will include those links in sitemap, they still will NOT be indexed by search engines and their both will not be able to open them.