XML Sitemaps Generator

Author Topic: Will not crawl subdirectories...  (Read 6958 times)

cs24

  • Registered Customer
  • Approved member
  • *
  • Posts: 8
Will not crawl subdirectories...
« on: December 17, 2013, 08:33:21 AM »
Hello,

I am using 6.1 unlimited sitemap generator. It is crawling [external links are visible to admins only] but it will not crawl the URLs listed on that page and add to the sitemap. Any ideas?

Thanks!

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Will not crawl subdirectories...
« Reply #1 on: December 17, 2013, 11:50:43 AM »
Hello,

what is an example URL that is not added?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

cs24

  • Registered Customer
  • Approved member
  • *
  • Posts: 8
Re: Will not crawl subdirectories...
« Reply #2 on: December 17, 2013, 01:04:56 PM »
Here is an example of one of the URLs that was not added: [external links are visible to admins only] . Right now I have 3 URLs but soon (within 12 hours) I will have 10,000 + URLs like this one that will need to be added to the sitemap.

Thanks!
« Last Edit: December 17, 2013, 01:07:31 PM by cs24 »

cs24

  • Registered Customer
  • Approved member
  • *
  • Posts: 8
Re: Will not crawl subdirectories...
« Reply #3 on: December 18, 2013, 06:43:11 PM »
Any ideas...? I need to resolve this as soon as possible, please!

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Will not crawl subdirectories...
« Reply #4 on: December 18, 2013, 08:46:24 PM »
Hello,

in case if starting URL is domain.org/table/centers/ then only URLs in that folder (table/centers/) are indexed. You need to set domain.org as starting URL.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

cs24

  • Registered Customer
  • Approved member
  • *
  • Posts: 8
Re: Will not crawl subdirectories...
« Reply #5 on: December 18, 2013, 08:49:09 PM »
Hello,

It is set like this: Starting URL: [external links are visible to admins only]


XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10622
Re: Will not crawl subdirectories...
« Reply #6 on: December 19, 2013, 01:57:28 PM »
Hello,

please let me know your generator URL/login in private message to check this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2