My Forum is too big?
« on: March 12, 2014, 11:24:37 AM »
I talk to the ADMIN: some week ago you wrote to me but i can't found the email sorry...
I talk to all people: Seem that my forum is too much big... After 2 hours the crawler don't map all forum and the file take 113 Mb of ram and the list is above 20.000 links...

It is normal?
My technicians tell me that if he modify too much the php possibility the server become instable so he tell to me: " stop the crowl at the second level link deep. But i would like craw all my forum...
What can i do?
The Online service is more stable?
Let me know...

Nwatson99

*
  • *
  • 20
  • AR15 Owner and Operator
Re: My Forum is too big?
« Reply #1 on: March 14, 2014, 06:56:35 AM »
I have over 2 million posts on my forum and the sitemap generator works great.
Now I will advise you two things:
1. DO NOT let it crawl the calendar, the sitemap generator will not honor the allotted time frame set for the calendar, so I would turn the calendar off to guests and search bots.
2. Set up and run the cron job, it will generate the sitemap in less than an hour vs running the generator through the browser will take days or at least it did for me.

Just FYI, my site generates 6 sitemaps.
Re: My Forum is too big?
« Reply #2 on: March 14, 2014, 07:36:49 AM »
I use Woltlab. How i can avoid to skan something in my forum? For example i don't want scan the members list because are 3000...
At the time for limit the crawl i choose max deep level 2 because if i choose 3 the crawl crash....
Re: My Forum is too big?
« Reply #3 on: March 14, 2014, 02:00:10 PM »
Hello,

please let me know your generator URL/login in private message to check this.

Nwatson99

*
  • *
  • 20
  • AR15 Owner and Operator
Re: My Forum is too big?
« Reply #4 on: March 14, 2014, 02:39:22 PM »
I use Woltlab. How i can avoid to skan something in my forum? For example i don't want scan the members list because are 3000...
At the time for limit the crawl i choose max deep level 2 because if i choose 3 the crawl crash....
I know that software, looks like it is built off of old IPB.
Anyway you should not have to make changes to your sitemap generator, you board is small and still growing and the sitemap will run just fine.
Oleg is here he will take care of you and make it right.