I am opening up a new section on the website with a calendar using canendar software. As this is a calendar it is infinate.
Is it possible to limit the amount of pages that it crawls in that folder.
The main reason i use this software is to create the sitemap.xml file that google and the other search engines look for.
I tried stoping the crawl by loggin out as there was no other way to stop the crawl and i do not know if the sitemap.xml file has been updated and can not look at the list as it does not update when you stop the crawl this way.
Can someone help. Other wise the crawl will go on forever...