Crawler reading Calandar links
« on: May 01, 2016, 06:45:03 PM »
How do I go about stopping the crawling of calendar years that are not even here yet?
Example 2020, 2022. I am showing a link depth of 91 currently "Current page: calendar/2023/09/21/"
Re: Crawler reading Calandar links
« Reply #1 on: May 03, 2016, 09:47:17 AM »

you can add them in Exclude URLs setting:

Re: Crawler reading Calendar links
« Reply #2 on: May 03, 2016, 01:37:10 PM »
Thanks -
I will give this a try.
The way it is going I may want to exclude the whole Calendar. Crawling the year 1923 now.