XML Sitemaps Generator

Author Topic: not crawling whole site  (Read 17411 times)

info778

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
not crawling whole site
« on: January 14, 2009, 07:42:55 PM »
Just bought the paid version but it doesn't crawl my entire site:

[external links are visible to admins only]

Example of URL not being picked up:
[external links are visible to admins only]

Please help


XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: not crawling whole site
« Reply #1 on: January 15, 2009, 12:36:14 AM »
Hello,

replied to your email.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

info778

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: not crawling whole site
« Reply #2 on: January 15, 2009, 12:45:41 AM »
Your email didn't come through, but I figured out that the sub-folders must be located through the main site linking...which they are not. Is there a way to enable this?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: not crawling whole site
« Reply #3 on: January 15, 2009, 02:38:01 PM »
Hello,

sitemap generator has a built-in crawler that visits your site and finds pages similar to regular visitors, that's why there must be a way to reach those pages starting from the URL you specified in options. You can create a separate html page that will include links to your homepage and those subfolders and then specify it as "Starting URL" for generator, so it will be able to crawl your site completely.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2