I purchased the Sitemap generator, installed and ran it. Over 40,000 pages were crawled.
Now my question is, when I run the crawl again, will it go over the 40,000 pages again?
Because those 40,000 + are permanent, they won't be changed. I have a folder called "news_of_2010" and that is the only folder updated daily with new links.
Will Google still crawl the old pages? Will Sitemap run over the same pages over and over?
How do I make it only crawl the new folde rthat's updated daily, since the other 40,000 + files are permanent and non-editable.
Thanks in advance.