Crawling only menu links
« on: April 15, 2007, 12:23:16 PM »

I've just purchased the stand alone sitemap generator and have found it invaluable for finding broken links - it's worth the money just for that facility alone. I have a number of sites and have an issue with just one of them. On this one site ([ External links are visible to forum administrators only ]) the crawl is only finding menu links - it doesn't crawl the links on the pages and as such is only returning 6 pages on a site of 200 plus. It's obviously something in the code on the pages but there are no no-follow tags and as far as I can see no other reasons for limiting the crawl.

Can you take a look and advise please


Re: Crawling only menu links
« Reply #1 on: April 16, 2007, 03:47:30 AM »

thank you for your kind words.

U just tried to crawl your site and generator has found many pages successfully.