Using XML-Sitemaps behind a load balancer?
« on: May 01, 2014, 10:52:21 AM »

We're running the tool behind a load balancer, and experiencing strange effects.

1. When I log into the tool, it shows no sitemaps have been generated. I click to generate one, which shows that c212 pages were crawled, it stops at "Creating sitemaps" and seemingly doesnt complete.

2. I edit settings, including adding a "notify when complete" email address. i switch to another tab and then back, and the email address has gone (even though I saved it).

3. I then get an email from the tool saying the sitemap completed, but saying it has 139 pages.

4. Also, one session in the tool will show "No sitemaps have been generated", whilst another will show one was created, but that it contains 139 pages?

I'm starting to think this is down to the load balancer - i.e connecting to almost a "seperate instance" on each side of the LB - one with the changed settings, one without?

I also think the 212 vs 139 pages crawled relates to the LB, i.e 212 pages served from one side of the lb, 139 served from the other?

Any advice apreciated,


Re: Using XML-Sitemaps behind a load balancer?
« Reply #1 on: May 02, 2014, 06:57:31 AM »

replied to your email.
in case of multi-server setup begind the load balancer I would recommend to run generator in command line instead to make sure it's on the same server. When sitemap is created you'd need to sync files across servers.
Re: Using XML-Sitemaps behind a load balancer?
« Reply #2 on: May 02, 2014, 09:14:33 AM »
Thanks - will pass that on to my devs.

Thanks for coming back to me :)