Toggle navigation
PRO Sitemaps
Standalone Generator
About Sitemaps
News
SEO Tools
Sitemap Generator Forum
Guest
Login
Register
Menu
Home
Help
Search
Login
Register
XML Sitemaps Discussions Category
/
Unlimited PHP Sitemap Generator
/
Blocking specific pages/directories from getting crawled
Gain New Customers!
Be #1 in Google for your keywords.
Gain lots of quality website visitors.
« previous
next »
Pages:
1
yacosta
1
Blocking specific pages/directories from getting crawled
«
on:
July 25, 2007, 06:55:42 PM »
Hi is there a way to block specific pages/directories from being crawled?
Logged
XML-Sitemaps Support
11520
Re: Blocking specific pages/directories from getting crawled
«
Reply #1 on:
July 26, 2007, 01:24:58 AM »
Hello,
yes, you can block them using robots.txt or just define them in "Exclude pages" option in Sitemap generator configuration.
Logged
Oleg Ignatiuk
https://www.xml-sitemaps.com
Send me a Private Message
Gain New Customers Today! Click here
.
Pages:
1
« previous
next »