XML Sitemaps Generator

Author Topic: Only part of site indexed  (Read 24191 times)

AndrewTraub

  • Approved member
  • *
  • Posts: 5
Only part of site indexed
« on: February 22, 2008, 08:31:51 PM »
Hello,

When I try the demo on two of my sites, only parts of the site are indexed (a very small part).

The sites are:
[external links are visible to admins only]
[external links are visible to admins only]

both sites start out with two pages. One links to subdomains and the other subfolders.  Each subfolder has many pages.

Will I have the same problem with the script?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Only part of site indexed
« Reply #1 on: February 23, 2008, 12:19:56 AM »
Hello,

according to the xml sitemap protocol, it's not allowed to include URLs from other (sub)domains in sitemap, so the crawler skips them.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

AndrewTraub

  • Approved member
  • *
  • Posts: 5
Re: Only part of site indexed
« Reply #2 on: February 23, 2008, 06:01:56 PM »
ok, so if it won't index, say [external links are visible to admins only], why won't it index [external links are visible to admins only] which is under [external links are visible to admins only]  Is it because the www is missing?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Only part of site indexed
« Reply #3 on: February 24, 2008, 08:52:20 PM »
Yes, that's right - "www" prefix means another domain name in this case.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

AndrewTraub

  • Approved member
  • *
  • Posts: 5
Re: Only part of site indexed
« Reply #4 on: February 25, 2008, 03:53:57 PM »
I still don't understand why the folders at [external links are visible to admins only] are not being indexed - only 17 pages are indexed but each folder has many articles.  Here are the links:

            <table width="100%"  border="0" align="center" cellpadding="2" cellspacing="0">
              <tr valign="top" class="style2">
                <td class="style3"><a href="[external links are visible to admins only]">Anxiety</a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Stress</a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Stress and Anxiety Medications </a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Herbal Medications</a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Alternative Medicines and Therapies</a></td>
              </tr>
              <tr valign="top" class="style2">
                <td class="style3"><a href="[external links are visible to admins only]">Social Anxiety </a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Panic Disorders</a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Depression</a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Panic Attacks </a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Psychologist</a></td>
              </tr>
              <tr valign="top" class="style2">
                <td class="style3"><a href="[external links are visible to admins only]">Stress Management</a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Behavior Modifications </a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Anxiety Symptoms and Stress Signs</a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Mental Health</a></td>
                <td class="style3"><a href="[external links are visible to admins only]">Stress and Anxiety Relief</a></td>
              </tr>
              <tr valign="top" class="style2">
                <td colspan="5" class="style3"><a href="[external links are visible to admins only]">Stress & Anxiety Blog</a></td>
            </tr>
            </table>

Am I missing something in the way the links are set up?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Only part of site indexed
« Reply #5 on: February 25, 2008, 11:40:08 PM »
Hello,

the "www. yourdomain .com/anxiety" link is redirected to "yourdomain .com/anxiety" for me.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

AndrewTraub

  • Approved member
  • *
  • Posts: 5
Re: Only part of site indexed
« Reply #6 on: February 26, 2008, 10:29:27 PM »
I still have the same result if I start spidering at [external links are visible to admins only] instead of [external links are visible to admins only].

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Only part of site indexed
« Reply #7 on: February 27, 2008, 12:20:21 AM »
The links are pointing to www domain on your pages (although they are redirected to non-www then). Make sure that ALL links (both on-page and landing URLs after redirect) reside  on exactly the same domain (non-www or www domain).
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

AndrewTraub

  • Approved member
  • *
  • Posts: 5
Re: Only part of site indexed
« Reply #8 on: February 27, 2008, 04:01:58 PM »
Thank you!

laugh555

  • Guest
Re: Only part of site indexed
« Reply #9 on: April 22, 2008, 04:20:36 PM »
Posted by: AndrewTraub
I still have the same result if I start spidering at http://health-anxiety-stress.com instead of http://www.health-anxiety-stress.com.

Thank you

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Only part of site indexed
« Reply #10 on: April 22, 2008, 09:23:43 PM »
You *should* use www domain as Starting URL since your links are pointing to it.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2