XML Sitemaps Generator

Author Topic: Site went from over 400 pages down to 7  (Read 24850 times)

mnw2000

  • Approved member
  • *
  • Posts: 1
Site went from over 400 pages down to 7
« on: August 17, 2006, 10:50:49 PM »
I generated a site map for my site [external links are visible to admins only] (and [external links are visible to admins only]) and it did a excellent job of getting every link (by going down three levels) resulting in over 400 pages.

However, when I attempted to generate a new site map after adding a few new pages, the site map stops at 7 pages (just the top level). The key file is "/webpages.php"  with various arguments such as "?id=794". However, the generator does not create any entries for these links.

Is there something I need to do to make sure it looks down three levels and works with a framed index.html file.

Thanks.

Marc

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Site went from over 400 pages down to 7
« Reply #1 on: August 18, 2006, 07:42:20 PM »
Hello Marc,

you have the following in robots.txt file:
Quote
Disallow: /webpages/listing.php
that effectively disallows crawling all of your listing pages.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2