• Welcome to Sitemap Generator Forum.
 

Can't exclude urls

Started by lagunacat, December 13, 2006, 05:05:56 AM

Previous topic - Next topic

lagunacat

I have added ext. to my Exclude URLs: box but they are still being crawled and added to my sitemap.html and urlstxt files.

I have tried different ways ie. blah.html, /blah.html , but still won't exclude.

also getting alot of urls like these in both of these files
[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]

XML-Sitemaps Support

Hello,

in order to exclude [ External links are visible to logged in users only ] URLs, you should add this in bot "Do not parse" and "Exclude URLs":

this=

lagunacat

Ok thanks for the help on excluding these type of links - [ External links are visible to forum administrators only ].

What about the first part of my question, excluding normal links that end in html?

Like this link - blah.html. I have put this in the exclude box in many different ways, ie:

blah.html
/blah.html
folder/blah.html

Anyway I put it it still ends being crawled and placed in the sitemap.

lagunacat

Ok I have added this link - [ External links are visible to forum administrators only ] - to the Do not Parse  and Exclude Urls bot and it is still being added to my sitemap. I have tried many configurations like;
[ External links are visible to forum administrators only ]
[ External links are visible to forum administrators only ]
/?this=1
?this=1
/?this=
?this=

They are not excluded

There are 46 of these numbered sequentially. I don't even know what these links are, they don't exist on my web site.

XML-Sitemaps Support

QuoteLike this link - blah.html. I have put this in the exclude box in many different ways, ie:

blah.html
Use this exclusion string:
.html

QuoteOk I have added this link - [ External links are visible to logged in users only ] - to the Do not Parse  and Exclude Urls bot and it is still being added to my sitemap. I have tried many configurations like;
You should add the following for that:
this=1

lagunacat


Use this exclusion string:
.html

That isn't a solution. if I add that then all my links ending in .html are excluded. I only want to exclude certain links.

QuoteOk I have added this link - [ External links are visible to forum administrators only ] - to the Do not Parse  and Exclude Urls bot and it is still being added to my sitemap. I have tried many configurations like;
You should add the following for that:
this=1
[/quote]

This solution didn't work either. What are these links? ( this=1, this=2. etc ). Like I said they don't exist in my website and are being generated by the Sitemap Generator.

XML-Sitemaps Support

Quote
This solution didn't work either. What are these links? ( this=1, this=2. etc ). Like I said they don't exist in my website and are being generated by the Sitemap Generator.
Sitemap Generator doesn't create ANY links at your website, it only finds existing ones.