• Welcome to Sitemap Generator Forum.
 

Exclude Urls, Do not parse urls

Started by freesn, April 20, 2009, 05:50:16 AM

Previous topic - Next topic

freesn

Hello, I have a website [ External links are visible to forum administrators only ]
I  don't want to crawl and put links in sitemaps from directory [ External links are visible to forum administrators only ]
I write in configuration in field EXCLUDE URLS "www.domain.com/dir/" and in DO NOT PARSE field also "www.domain.com/dir/". But when the script start to crawl they crawl my directory DIR and put links to sitemap.
I don't want to put this dir to sitemap, how can I do that?

Thanks!


giggler

I have Disallow: /*?quicktabs and Disallow: /*?sort= in the robot.txt and google webmaster tool shows as if blocked, but the sitemap generator includes the urls with these variables.

[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]

Would I just place the following in "Exclude URLs:" or how can I not include it into the sitemap.xml file?
/*?quicktabs
/*?sort=