XML Sitemaps Generator

Author Topic: robots.txt 'Allow:' command  (Read 6375 times)


  • Registered Customer
  • Approved member
  • *
  • Posts: 1
robots.txt 'Allow:' command
« on: February 04, 2013, 01:47:47 PM »
Hi, does Standalone Sitemap Generator respect the robots.txt 'Allow:' command for bots like google [external links are visible to admins only]

I have an image folder under a disallowed directory but have used the 'Allow:' command to allow it. Not sure if i have done this correct but no images are included in my images site map.

Disallow: /components/
Allow: /components/product/images/
Disallow: /components/product/images/thumb/
Disallow: /includes/

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10599
Re: robots.txt 'Allow:' command
« Reply #1 on: February 05, 2013, 07:46:36 AM »

no, only "disallow" is used.
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.


SMF 2.0.12 | SMF © 2014, Simple Machines