XML Sitemaps Generator

    Advanced search
Sitemap Generator Forum
July 05, 2008, 04:48:33 AM
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
   Home   Help Search Login Register  
Sitemap software 2.9 released - Email notifications, html sitemap customizing and more
6618 Posts in 1634 Topics by Members
Latest Member: mariebenz
Pages: [1]
  Print  
Author Topic: Can't exclude urls  (Read 9050 times)
lagunacat
Registered Customer
Newbie
*
Posts: 9


View Profile
« on: December 13, 2006, 05:05:56 AM »

I have added ext. to my Exclude URLs: box but they are still being crawled and added to my sitemap.html and urlstxt files.

I have tried different ways ie. blah.html, /blah.html , but still won't exclude.

also getting alot of urls like these in both of these files
[external links are visible to admins only]
[external links are visible to admins only]
[external links are visible to admins only]
Logged
admin
Administrator
Hero Member
*****
Posts: 2755


View Profile
« Reply #1 on: December 13, 2006, 10:49:11 PM »

Hello,

in order to exclude http://www.xxxxxxxxx.com/?this=XX URLs, you should add this in bot "Do not parse" and "Exclude URLs":

Code:
this=
Logged

lagunacat
Registered Customer
Newbie
*
Posts: 9


View Profile
« Reply #2 on: December 14, 2006, 01:26:33 AM »

Ok thanks for the help on excluding these type of links - [external links are visible to admins only].

What about the first part of my question, excluding normal links that end in html?

Like this link - blah.html. I have put this in the exclude box in many different ways, ie:

blah.html
/blah.html
folder/blah.html

Anyway I put it it still ends being crawled and placed in the sitemap.
Logged
lagunacat
Registered Customer
Newbie
*
Posts: 9


View Profile
« Reply #3 on: December 14, 2006, 05:21:27 PM »

Ok I have added this link - [external links are visible to admins only] - to the Do not Parse  and Exclude Urls bot and it is still being added to my sitemap. I have tried many configurations like;
[external links are visible to admins only]
[external links are visible to admins only]
/?this=1
?this=1
/?this=
?this=

They are not excluded

There are 46 of these numbered sequentially. I don't even know what these links are, they don't exist on my web site.
Logged
admin
Administrator
Hero Member
*****
Posts: 2755


View Profile
« Reply #4 on: December 14, 2006, 11:05:10 PM »

Quote
Like this link - blah.html. I have put this in the exclude box in many different ways, ie:

blah.html
Use this exclusion string:
Code:
.html

Quote
Ok I have added this link - http://www.xxxxxxxxx.com/?this=1 - to the Do not Parse  and Exclude Urls bot and it is still being added to my sitemap. I have tried many configurations like;
You should add the following for that:
Code:
this=1
Logged

lagunacat
Registered Customer
Newbie
*
Posts: 9


View Profile
« Reply #5 on: December 15, 2006, 06:48:03 PM »


Use this exclusion string:
Code:
.html

That isn't a solution. if I add that then all my links ending in .html are excluded. I only want to exclude certain links.

Quote
Ok I have added this link - [external links are visible to admins only] - to the Do not Parse  and Exclude Urls bot and it is still being added to my sitemap. I have tried many configurations like;
You should add the following for that:
Code:
this=1
[/quote]

This solution didn't work either. What are these links? ( this=1, this=2. etc ). Like I said they don't exist in my website and are being generated by the Sitemap Generator.
Logged
admin
Administrator
Hero Member
*****
Posts: 2755


View Profile
« Reply #6 on: December 15, 2006, 07:36:04 PM »

Quote
This solution didn't work either. What are these links? ( this=1, this=2. etc ). Like I said they don't exist in my website and are being generated by the Sitemap Generator.
Sitemap Generator doesn't create ANY links at your website, it only finds existing ones.
Logged

Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.5 | SMF © 2006, Simple Machines LLC Valid XHTML 1.0! Valid CSS!