XML Sitemaps Generator

Author Topic: Exclusion preset for IPB 3.x.x  (Read 11140 times)

spagman_72

  • Registered Customer
  • Approved member
  • *
  • Posts: 1
Exclusion preset for IPB 3.x.x
« on: February 28, 2010, 01:12:12 AM »
Hello, Im a new happy customer of XML Sitemaps, just got this yesterday and created my first sitemap today.
Everything went just fine but I have a couple of questions.

after the scan I reviewed the broken/404 links and most of them are not really "broken" I believe the robots.txt file is blocking them, because I can open them when I click on them.

I ran XML Sitemap on my IPB site, and I wanted to know if there are exclusions presets for IPB, I searched the forums but didnt see any.

if not, this is my robots.txt, can I create them from the robot.txt so they get excluded and dont show up as broken links?
Thank You for the great product!

Code: [external links are visible to admins only]
User-agent: *

Disallow: /admin/

Disallow: /public/js/

Disallow: /public/style_captcha/

Disallow: /public/style_css/

Disallow: /index.php?app=core&module=task

Disallow: /index.php?app=core&module=usercp&tab=forums&area=forumsubs

Disallow: /index.php?app=core&module=usercp&tab=forums&area=watch&watch=topic

Disallow: /index.php?app=forums&module=extras&section=forward

Disallow: /index.php?app=members&module=messaging

Disallow: /index.php?app=members&module=chat

Disallow: /index.php?app=members&module=search

Disallow: /index.php?app=members&module=search&do=active

Disallow: /index.php?&unlockUserAgent=1

Disallow: /*app=core&module=usercp

Disallow: /*app=core&module=usercp

Disallow: /*app=members&module=messaging

Disallow: /*&p=

Disallow: /*&pid=

Disallow: /*&hl=

Disallow: /*&start=

Disallow: /*page__*$

Disallow: /*view__*$

Disallow: /*&view=*$

Disallow: /calendar/

Sitemap: http:// mysite/sitemap_index.xml.gz

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10625
Re: Exclusion preset for IPB 3.x.x
« Reply #1 on: March 01, 2010, 01:15:21 PM »
Hello,

generator reads the robots.txt file, so as long as you have them disallows in robots file, no need to define them in Exclusions list.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2