crawler has indexed 6,160 pages while my site has over 20,000...
« on: September 03, 2009, 10:45:29 AM »
hello,

I've created this site map [ External links are visible to forum administrators only ] with Standalone Sitemap Generator.
Despite the fact that my site contains a business directory which has more than 10,000 businesses - each diplayed in two separate pages (in different languages) in addition to other content pages like news, articles etc. the crawler has indexed only 6160 pages.

when i look at the site map i can see that many categories in my directory are partly indexed.
for expmale:
here [ External links are visible to forum administrators only ]
under Supermarkets category there is only one page titled "bar kol" although you can see that this category in the site  [ External links are visible to forum administrators only ] has over 50 businesses

 
Re: crawler has indexed 6,160 pages while my site has over 20,000...
« Reply #1 on: September 03, 2009, 10:03:21 PM »
Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached starting  from homepage?