How to restrict the sitemap to pull only specific urls.
« on: February 04, 2014, 10:42:24 PM »
I want to restrict the crawl to specific pages like and then I want it to pull up any page in the /zip/directory and crawl each page.. meaing if the page is full of urls leading to an mls page I want to crawl all the urls on every page that is in the /zip/ directory.. does this make any sense?
Or if I crawl the pages and get the urls.. can a huge text file of urls be fed to the sitemap generator to format and place theim all in a sitemap..