kid

*
  • *
  • 3
What to include
« on: October 14, 2010, 03:00:37 PM »
I just finished crawling my forum a few times, using differant exclusions. My question is, is it better to include as much as possible, or I should I exclude all but the most relevant topics. Like most forums, I have an anything goes section, that sees a majority of the posting. The topics are not relevant to my site content, so I excluded those topics. After doing this, I went from over 600 pages indexed, down to 44. I imagine a bigger footprint in Google will get better traffic?

Site info:
[ External links are visible to forum administrators only ]
runs on SMF
Re: What to include
« Reply #1 on: October 15, 2010, 01:08:55 PM »
Hello,

I would include all topics in sitemap, it won't hurt your other pages. But I'd recommend to apply SMF URLs exclusion preset (that is built-in in generator) to avoid crawling of noise content pages.

kid

*
  • *
  • 3
Re: What to include
« Reply #2 on: October 15, 2010, 01:14:23 PM »
Thanks. I found the filter for SMF, and it knocked about 4ooo useless URL's from list(login pages and whatnot) Is there anyway to block certain words from showing in a site map? Being a forum, my top crawled keywords are post, reply, quote, etc. Obviously I use revelent keywords in my meta tags, but I believe Google gives a lot of weight to waht is crawled.

Hello,

I would include all topics in sitemap, it won't hurt your other pages. But I'd recommend to apply SMF URLs exclusion preset (that is built-in in generator) to avoid crawling of noise content pages.

Re: What to include
« Reply #3 on: October 15, 2010, 03:08:38 PM »
Hello,

you can only filter the indexed pages by URL, i.e. enter the common part of the link in "Exclude URLs" setting and it will be skipped when crawling.

kid

*
  • *
  • 3
Re: What to include
« Reply #4 on: October 15, 2010, 03:13:41 PM »
Okay, that's what I thought. Thanks!

Hello,

you can only filter the indexed pages by URL, i.e. enter the common part of the link in "Exclude URLs" setting and it will be skipped when crawling.