Will not crawl subdirectories...
« on: December 17, 2013, 08:33:21 AM »
Hello,

I am using 6.1 unlimited sitemap generator. It is crawling http://www.my-domain.org/table/centers/ but it will not crawl the URLs listed on that page and add to the sitemap. Any ideas?

Thanks!
Re: Will not crawl subdirectories...
« Reply #1 on: December 17, 2013, 11:50:43 AM »
Hello,

what is an example URL that is not added?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Will not crawl subdirectories...
« Reply #2 on: December 17, 2013, 01:04:56 PM »
Here is an example of one of the URLs that was not added: http://std-test.org/centers/std-testing-abbott.html . Right now I have 3 URLs but soon (within 12 hours) I will have 10,000 + URLs like this one that will need to be added to the sitemap.

Thanks!
« Last Edit: December 17, 2013, 01:07:31 PM by cs24 »
Re: Will not crawl subdirectories...
« Reply #3 on: December 18, 2013, 06:43:11 PM »
Any ideas...? I need to resolve this as soon as possible, please!
Re: Will not crawl subdirectories...
« Reply #4 on: December 18, 2013, 08:46:24 PM »
Hello,

in case if starting URL is domain.org/table/centers/ then only URLs in that folder (table/centers/) are indexed. You need to set domain.org as starting URL.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.
Re: Will not crawl subdirectories...
« Reply #5 on: December 18, 2013, 08:49:09 PM »
Hello,

It is set like this: Starting URL: http://www.MyDomain.org

Re: Will not crawl subdirectories...
« Reply #6 on: December 19, 2013, 01:57:28 PM »
Hello,

please let me know your generator URL/login in private message to check this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.