We need to stop the crawling process, please advise how to do it.
We have a site with 000s of labels attached to the site pages. Apparently the crawler is visiting on each labels and this is bringing the site to its knees. We can't make it stop.
We are running Linux Ubuntu.