XML Sitemaps Generator

Author Topic: Form > Crawler doesn't follow select dropdown links  (Read 208 times)

guy10

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Form > Crawler doesn't follow select dropdown links
« on: September 10, 2017, 06:35:27 AM »
Hi,

A question here:
I've got a SELECT dropdown & form on my homepage visitors need to use in order to select a town. The problem is the XML-sitemap doesn't crawl the different select options (all unique url's) but it should do that. I'm only getting back some general pages (links in footer / header menu).

Site: [external links are visible to admins only]
OR in English: [external links are visible to admins only]

Sitemap: [external links are visible to admins only]

How can I make sure the sitemap is crawling through all the subpages...
The selection a user makes is stored in a cookie so he/she only needs to select a city once. Maybe that's a problem (but I don't see ANY cities in my xml sitemap)...

Anyone to the rescue?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10896
Re: Form > Crawler doesn't follow select dropdown links
« Reply #1 on: September 11, 2017, 05:44:24 AM »
Hello,

all pages must be accessible via <a href=""> links and should be working with javascript disabled in the browser to allow crawler bots to find them.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

guy10

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Re: Form > Crawler doesn't follow select dropdown links
« Reply #2 on: September 11, 2017, 07:59:29 AM »
That did the trick indeed. Thanks for the quick reply.
Should all these links be visible or can I "display:none" them so they are only visible for bots?
Or will SEO punish me for this kind of trics.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10896
Re: Form > Crawler doesn't follow select dropdown links
« Reply #3 on: September 11, 2017, 08:36:35 PM »
Those can be invisible for the sitemap generator bot, although it's not recommended for search engines bots.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.14 | SMF © 2017, Simple Machines
XHTML RSS WAP2