XML Sitemaps Generator

Author Topic: sitemap not crawling all forum links  (Read 7586 times)

paypal346

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 29
sitemap not crawling all forum links
« on: August 27, 2011, 02:40:00 AM »
Using a vbulletin forum and used the settings to not crawl specific pages, e.g. profiles. But only a fraction of the actual forum pages are crawled. Here is the sitemap: [external links are visible to admins only]  There should be far more threads indexed.

paypal346

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 29
Re: sitemap not crawling all forum links
« Reply #1 on: August 27, 2011, 02:44:35 AM »
It's also not crawling wordpress blogs properly. If you look at the end of the sitemap, there are a whole bunch of links that are individual links but are actually part of one blog. it isn't recognizing them properly even if they are in their own directory (I used sort by directory.)

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: sitemap not crawling all forum links
« Reply #2 on: August 27, 2011, 08:22:27 PM »
Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached starting from homepage?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2