XML Sitemaps Generator

    Advanced search
Sitemap Generator Forum
July 04, 2008, 10:57:12 PM
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
   Home   Help Search Login Register  
Sitemap software 2.9 released - Email notifications, html sitemap customizing and more
6614 Posts in 1634 Topics by Members
Latest Member: info589
Pages: [1]
  Print  
Author Topic: Site went from over 400 pages down to 7  (Read 7736 times)
mnw2000
Newbie
*
Posts: 1


View Profile
« on: August 17, 2006, 11:50:49 PM »

I generated a site map for my site [external links are visible to admins only] (and [external links are visible to admins only]) and it did a excellent job of getting every link (by going down three levels) resulting in over 400 pages.

However, when I attempted to generate a new site map after adding a few new pages, the site map stops at 7 pages (just the top level). The key file is "/webpages.php"  with various arguments such as "?id=794". However, the generator does not create any entries for these links.

Is there something I need to do to make sure it looks down three levels and works with a framed index.html file.

Thanks.

Marc
Logged
admin
Administrator
Hero Member
*****
Posts: 2755


View Profile
« Reply #1 on: August 18, 2006, 08:42:20 PM »

Hello Marc,

you have the following in robots.txt file:
Quote
Disallow: /webpages/listing.php
that effectively disallows crawling all of your listing pages.
Logged

Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.5 | SMF © 2006, Simple Machines LLC Valid XHTML 1.0! Valid CSS!