Limited crawling
« on: December 11, 2025, 06:23:49 PM »
Hello,

I've run the script several times, but it always stops at the same number of links: 667, even though I've set it to 100,000.

More than three-quarters of the pages/links aren't being crawled.

What could be causing this?

How can I find out?
Re: Limited crawling
« Reply #1 on: December 11, 2025, 07:00:19 PM »
Hello,

could you please provide an example URL that is not included in a sitemap and how it can be reached starting from homepage?