Hi, I have a cron job that is set to run on the hour and half hour which runs fine. However, for some strange reason the crawler runs several times a day at times that are not programmed in my cron job. Generally around 15 before or 15 after the hour.
In itself this is odd, but when the crawler runs on these odd times it puts old pages on the top of my rss feed that are not the most recent pages.
Not sure why this is happening but it just seems strange that crawler starts by itself. These odd time crawls run so long that my scheduled cron jobs do not run.
Anyway, hope you can come up with a reason.
Thank you,