• Welcome to Sitemap Generator Forum.
 
Tools for Any SEO Challenge: Get more traffic to your website with versatile solutions for every step of your SEO journey.

Blocking specific pages/directories from getting crawled

Started by yacosta, July 25, 2007, 06:55:42 PM

Previous topic - Next topic
Print
Go Down Pages1
User actions

yacosta

July 25, 2007, 06:55:42 PM
Hi is there a way to block specific pages/directories from being crawled?

XML-Sitemaps Support

#1
July 26, 2007, 01:24:58 AM
Hello,

yes, you can block them using robots.txt or just define them in "Exclude pages" option in Sitemap generator configuration.
Print
Go Up Pages1
User actions