XML Sitemaps Generator

Author Topic: Crawler reading Calandar links  (Read 2216 times)

ajgilbertson

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Crawler reading Calandar links
« on: May 01, 2016, 05:45:03 PM »
How do I go about stopping the crawling of calendar years that are not even here yet?
Example 2020, 2022. I am showing a link depth of 91 currently "Current page: calendar/2023/09/21/"

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Crawler reading Calandar links
« Reply #1 on: May 03, 2016, 08:47:17 AM »
Hello,

you can add them in Exclude URLs setting:
calendar/2022/
calendar/2023/



or
calendar/202
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

ajgilbertson

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Re: Crawler reading Calendar links
« Reply #2 on: May 03, 2016, 12:37:10 PM »
Thanks -
I will give this a try.
The way it is going I may want to exclude the whole Calendar. Crawling the year 1923 now.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2