XML Sitemaps Generator

Author Topic: How to Limit Crawl To Specific URL String  (Read 23464 times)


  • Registered Customer
  • Approved member
  • *
  • Posts: 2
How to Limit Crawl To Specific URL String
« on: October 11, 2006, 07:32:15 PM »
How do I set the crawler to only crawl urls on my site that contain the following in the URL string: itemType=CATEGORY in the url string?

Currently the crawler is taking days to get through our site and if I can limit the sitemap to only crawl our extensive combination of categories it should bring the crawl to a more manageable crawl time.


XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10599
Re: How to Limit Crawl To Specific URL String
« Reply #1 on: October 13, 2006, 12:29:26 AM »
Hello Jason,

currently there is no option for "Crawl only" URLs in sitemap generator, but you can use "Do not parse" option to exclude certain parts of your site.
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.


SMF 2.0.12 | SMF © 2014, Simple Machines