sm

*
  • *
  • 12
I have a subdirectory at:
[ External links are visible to forum administrators only ]

domain.com/subdirectory
starting URL: domain.com/subdirectory

I would like to set this up with only crawling this directory and not other subdirectories and root files.

Is this possible without adding the EACH of the files name and other subdirectories to "Exclude URLs:" plus having the subdirectory placed at ""Include ONLY" URLs:" as the subdirectoy.

A different way to look at this is having one sitemap in the root, and a seconded one has a sitemap generator too. Then having 2 sitemaps that don't crawl each other.
Hello,

if you specify Starting URL as domain.com/subdirectory/ (with ending slash), it would only include URLs in subdirectory/.

sm

*
  • *
  • 12
sometimes it amazes me how a slash could make such a big difference.

Thanks!!

mine was defaulting domain.com/sub

now domain.com/sub/ is working