how to not crawl dynamic pages?
« on: January 28, 2011, 07:28:33 AM »
How do you not crawl dynamic pages created by website?
Re: how to not crawl dynamic pages?
« Reply #1 on: January 28, 2011, 10:47:16 AM »
Hello,

you should just enter your homepage link as Starting URL and further dynamic pages will be found by generator crawler as long as they linked somewhere on your site.
Re: how to not crawl dynamic pages?
« Reply #2 on: January 28, 2011, 11:05:20 AM »
My question was to NOT crawl dynamic pages.
[ External links are visible to forum administrators only ]]=1
Re: how to not crawl dynamic pages?
« Reply #3 on: January 28, 2011, 11:18:56 AM »
Sorry, I misunderstood the request. You should add a question mark "?" in "Exclude URLs" setting, and all links with it will be excluded.
Re: how to not crawl dynamic pages?
« Reply #4 on: January 28, 2011, 11:20:43 AM »
Thanks worked perfectly, do I have to buy the standalone version for each site i own?
Re: how to not crawl dynamic pages?
« Reply #5 on: January 28, 2011, 11:09:09 PM »
You can use the same license for more than one domain as long as you are the owner of those websites: https://www.xml-sitemaps.com/license.html