Help!
« on: September 12, 2006, 06:07:50 PM »
I have 100s of pages I do not want crawled. I am having a hard time figuring out how to set this up to work properly. Here are some examples of what I want crawled and below are the some I dont. Please help.

Want crawled:

Base URL/idirectory/inspectors.php?st=va
Base URL/idirectory/inspectors.php?st=md

And all the rest of the states



Dont want crawled

All the listings below exsist for each member of our site. That is 100s Each listing pulls the information out of the database.

Base URL/idirectory/view.php?id=142&page=0&cat=1

Base URL/idirectory/out.php?ID=142

Based URL/idirectory/mail2.php?id=142&cat=1&page=0

Base URL/idirectory/review.php?id=142&cat=1&page=0

Re: Help!
« Reply #1 on: September 13, 2006, 02:16:26 AM »
Hello,

you can add the following in "Exclude URLs" option:
Quote
view.php
out.php
mail2.php
review.php