Keir

*
  • *
  • 10
Giant Sitemap with Forum
« on: November 08, 2010, 05:22:10 PM »
Hi

I have phpBB3 installed and when I run the site-map for it I get over 3000 links and the site-map takes at least 15 minutes to generate. I'd like it to be quick and not have over 3000 links! I tried the method in the Crazed Sitemap topic which was to add Parse-Only URL's etc. I would like it just to generate the sitemap for my index, faq, members, topics and announcements

Thanks
« Last Edit: November 08, 2010, 05:27:17 PM by Keir »

Keir

*
  • *
  • 10
Re: Giant Sitemap with Forum
« Reply #1 on: November 08, 2010, 07:05:26 PM »
Okay I just realised since giving it the parseonly etc it is about 5 minutes faster but how can I make it so it's super quick?

You could make it so that once it does a full scan then the next 5 times it only does topics inside the active threads search then after 5 times do a full scan?

Or is there an easier way?
Re: Giant Sitemap with Forum
« Reply #2 on: November 08, 2010, 09:35:21 PM »
Hello,

sitemap generator bot needs to scan all pages, otherwise it won't be able to find new ones.
Make sure that you use URL exclusion preset for phpBB, it will be very fast (including topics in sitemap directly, without parsing them).

Keir

*
  • *
  • 10
Re: Giant Sitemap with Forum
« Reply #3 on: November 09, 2010, 04:57:13 PM »
Hello,

sitemap generator bot needs to scan all pages, otherwise it won't be able to find new ones.
Make sure that you use URL exclusion preset for phpBB, it will be very fast (including topics in sitemap directly, without parsing them).

Okay what else can I put in? This is what I have

Exclude URL's:
Code: [Select]
&pid=
&view=
&do=reply
&do=new
act=Post
act=Login
act=usercp
req=sendentry
op=reportcomment
show=next
req=syndicate
req=showarchive

Do Not parse URL's:

Code: [Select]
showuser=
&user=
do=stats
req=printentry
show=comment
showtopic=
showentry=
lofiversion/index.php/t
&img=

Sorry I got "parse-only urls" mixed up with "do not parse urls lol"

Thanks
Re: Giant Sitemap with Forum
« Reply #4 on: November 09, 2010, 07:05:42 PM »
It looks ok - topics are added directly due to "Do not parse" setting, assuming that you have default dynamic links for IPB (index.php?showtopic=XX)

Keir

*
  • *
  • 10
Re: Giant Sitemap with Forum
« Reply #5 on: November 14, 2010, 05:13:16 PM »
It looks ok - topics are added directly due to "Do not parse" setting, assuming that you have default dynamic links for IPB (index.php?showtopic=XX)

Okay Thanks

Keir

*
  • *
  • 10
Re: Giant Sitemap with Forum
« Reply #6 on: January 31, 2011, 05:10:59 PM »
Sorry to bump this thread but what are the "URL exclusion preset for phpBB"???? Thanks
Re: Giant Sitemap with Forum
« Reply #7 on: January 31, 2011, 08:22:34 PM »
On sitemap generator config page you can find this:
Narrow Indexed Pages Set
->
Exclusion preset

In that dropdown select "phpBB" if you are using phpbb forum.

Keir

*
  • *
  • 10
Re: Giant Sitemap with Forum
« Reply #8 on: February 02, 2011, 06:08:38 PM »
Thanks! Trying that now!