so... there was a faster way to crawl(parse).
« on: May 19, 2017, 02:48:11 AM »
I am getting 130000 pages in 23 minus. Used to having 130000 pages in days.
Hint was in Narrow Indexed Pages Set. Try Exclusion preset. I am now using this software without a hassle.