not crawling whole site
« on: January 14, 2009, 07:42:55 PM »
Just bought the paid version but it doesn't crawl my entire site:

[ External links are visible to forum administrators only ]

Example of URL not being picked up:
[ External links are visible to forum administrators only ]

Please help

Re: not crawling whole site
« Reply #2 on: January 15, 2009, 12:45:41 AM »
Your email didn't come through, but I figured out that the sub-folders must be located through the main site linking...which they are not. Is there a way to enable this?
Re: not crawling whole site
« Reply #3 on: January 15, 2009, 02:38:01 PM »
Hello,

sitemap generator has a built-in crawler that visits your site and finds pages similar to regular visitors, that's why there must be a way to reach those pages starting from the URL you specified in options. You can create a separate html page that will include links to your homepage and those subfolders and then specify it as "Starting URL" for generator, so it will be able to crawl your site completely.