Only part of site indexed
« on: February 22, 2008, 08:31:51 PM »
Hello,

When I try the demo on two of my sites, only parts of the site are indexed (a very small part).

The sites are:
http://www.health-anxiety-stress.com
http://www.backpainfixes.com

both sites start out with two pages. One links to subdomains and the other subfolders.  Each subfolder has many pages.

Will I have the same problem with the script?
Re: Only part of site indexed
« Reply #1 on: February 23, 2008, 12:19:56 AM »
Hello,

according to the xml sitemap protocol, it's not allowed to include URLs from other (sub)domains in sitemap, so the crawler skips them.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Only part of site indexed
« Reply #2 on: February 23, 2008, 06:01:56 PM »
ok, so if it won't index, say http://back.backpainfixes.com, why won't it index http://health-anxiety-stress.com/anxiety/ which is under http://www.health-anxiety-stress.com?  Is it because the www is missing?
Re: Only part of site indexed
« Reply #3 on: February 24, 2008, 08:52:20 PM »
Yes, that's right - "www" prefix means another domain name in this case.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Only part of site indexed
« Reply #4 on: February 25, 2008, 03:53:57 PM »
I still don't understand why the folders at http://www.health-anxiety-stress.com are not being indexed - only 17 pages are indexed but each folder has many articles.  Here are the links:

            <table width="100%"  border="0" align="center" cellpadding="2" cellspacing="0">
              <tr valign="top" class="style2">
                <td class="style3"><a href="http://www.health-anxiety-stress.com/anxiety">Anxiety</a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/stress">Stress</a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/medications">Stress and Anxiety Medications </a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/herbal">Herbal Medications</a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/alternative">Alternative Medicines and Therapies</a></td>
              </tr>
              <tr valign="top" class="style2">
                <td class="style3"><a href="http://www.health-anxiety-stress.com/social">Social Anxiety </a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/disorders">Panic Disorders</a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/depression">Depression</a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/attacks">Panic Attacks </a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/psychologists">Psychologist</a></td>
              </tr>
              <tr valign="top" class="style2">
                <td class="style3"><a href="http://www.health-anxiety-stress.com/management">Stress Management</a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/behavior">Behavior Modifications </a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/symptoms">Anxiety Symptoms and Stress Signs</a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/mental">Mental Health</a></td>
                <td class="style3"><a href="http://www.health-anxiety-stress.com/relief">Stress and Anxiety Relief</a></td>
              </tr>
              <tr valign="top" class="style2">
                <td colspan="5" class="style3"><a href="http://www.health-anxiety-stress.com/blog">Stress & Anxiety Blog</a></td>
            </tr>
            </table>

Am I missing something in the way the links are set up?
Re: Only part of site indexed
« Reply #5 on: February 25, 2008, 11:40:08 PM »
Hello,

the "www. yourdomain .com/anxiety" link is redirected to "yourdomain .com/anxiety" for me.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Only part of site indexed
« Reply #6 on: February 26, 2008, 10:29:27 PM »
I still have the same result if I start spidering at http://health-anxiety-stress.com instead of http://www.health-anxiety-stress.com.
Re: Only part of site indexed
« Reply #7 on: February 27, 2008, 12:20:21 AM »
The links are pointing to www domain on your pages (although they are redirected to non-www then). Make sure that ALL links (both on-page and landing URLs after redirect) reside  on exactly the same domain (non-www or www domain).
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Only part of site indexed
« Reply #8 on: February 27, 2008, 04:01:58 PM »
Thank you!

laugh555

*
Re: Only part of site indexed
« Reply #9 on: April 22, 2008, 05:20:36 PM »
Posted by: AndrewTraub
I still have the same result if I start spidering at http://health-anxiety-stress.com instead of http://www.health-anxiety-stress.com.

Thank you
Re: Only part of site indexed
« Reply #10 on: April 22, 2008, 10:23:43 PM »
You *should* use www domain as Starting URL since your links are pointing to it.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.