• Welcome to Sitemap Generator Forum.
 

How to restrict the sitemap to pull only specific urls.

Started by jenny5, February 04, 2014, 10:42:24 PM

Previous topic - Next topic

jenny5

I want to restrict the crawl to specific pages like [ External links are visible to forum administrators only ] and then I want it to pull up any page in the /zip/directory and crawl each page.. meaing if the page is full of urls leading to an mls page I want to crawl all the urls on every page that is in the /zip/ directory.. does this make any sense?
Or if I crawl the pages and get the urls.. can a huge text file of urls be fed to the sitemap generator to format and place theim all in a sitemap..

XML-Sitemaps Support

Hello,

you can use [ External links are visible to logged in users only ] as Starting URL to crawl/index only URLs including zip/ part.