I've read the posts about trying to improve performance - I run a website that has a million or so pages. Has anyone been able to successfully create a sitemap of this size running this software?
I've been running this thing for a couple days now, I've changed the configuration and added some info into the Do Not Parse section. I have two entries, is the syntax below correct? The first line - I'd like to not parse any of my forum posts - they all have a different topic id. The second line is a directory. Are those correct?
Next question, adding these into the Do Not Parse did not improve my performance. I'm running in the background but had to resume the process after it stopped for some reason. Is it using my new configuration? Do I need to start over? How do I get the crawl to use the updated configuration?
Even though the program is running in one browser (with the background option), when I go to the Crawling section in another browser, the second browser asks if I'd like to continue the interupted session. However, I see the numbers still updating in the first browser, so it appears to be running. Why is this happening?
How do I stop the crawl if it is running in the background?
Thank you for your help - I haven't given up on this quite yet!