Exclude Urls, Do not parse urls
« on: April 20, 2009, 05:50:16 AM »
Hello, I have a website [ External links are visible to forum administrators only ]
I  don't want to crawl and put links in sitemaps from directory [ External links are visible to forum administrators only ]
I write in configuration in field EXCLUDE URLS "[ External links are visible to forum administrators only ]" and in DO NOT PARSE field also "[ External links are visible to forum administrators only ]". But when the script start to crawl they crawl my directory DIR and put links to sitemap.
I don't want to put this dir to sitemap, how can I do that?

Thanks!
Re: Exclude Urls, Do not parse urls
« Reply #1 on: April 21, 2009, 01:31:50 AM »
You should define it without domain name, just:
Code: [ External links are visible to forum administrators only ]
dir/
Re: Exclude Urls, Do not parse urls
« Reply #2 on: April 28, 2009, 09:10:56 PM »
I have Disallow: /*?quicktabs and Disallow: /*?sort= in the robot.txt and google webmaster tool shows as if blocked, but the sitemap generator includes the urls with these variables.

[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]

Would I just place the following in "Exclude URLs:" or how can I not include it into the sitemap.xml file?
/*?quicktabs
/*?sort=
Re: Exclude Urls, Do not parse urls
« Reply #3 on: April 29, 2009, 08:06:40 PM »
Hello,

in "Exclude URLs" option you should add them as:
Code: [ External links are visible to forum administrators only ]
?quicktabs
?sort=