My site has over 14000 pages and the sitemap generated shows only ~300+ links in it. I noticed that the pages are being skipped. When the crawler is executing, I can see all those pages being crawled, however they are not getting added in the sitemap.
I want the crawler to ignore a certain type of query. I have tried blocking Disallow: /? and also tried the generator exclude url. Both works fine. However, the crawled URL's are now even less.
Would appreciate if you could help me fix/ look into what is going wrong in my case.