I'm at a loss to explain why this is so. There are thousands of pages scanned but not added. robots.txt and the meta tags are not interfering with the indexing of these pages. Google has no problem crawling these pages. I'm not sure what information you want. I looked at other threads that had the same problems but there are no solutions; only private messages. Using the latest version that I just downloaded today.
Here is a typical page that is not getting added: [external links are visible to admins only]
My robots.txt file contains this: