limit pages crawled in folder
« on: August 18, 2008, 04:58:20 PM »
Hi

I am opening up a new section on the website with a calendar using canendar software. As this is a calendar it is infinate.

Is it possible to limit the amount of pages that it crawls in that folder. ::)

The main reason i use this software is to create the sitemap.xml file that google and the other search engines look for.

I tried stoping the crawl by loggin out as there was no other way to stop the crawl and i do not know if the sitemap.xml file has been updated and can not look at the list as it does not update when you stop the crawl this way.

Can someone help. Other wise the crawl will go on forever... :'(

Craig
Re: limit pages crawled in folder
« Reply #1 on: August 18, 2008, 05:03:01 PM »
Hi

also because this folder go's o forever can not be sure that all the other folders are not indexed.

craig
Re: limit pages crawled in folder
« Reply #2 on: August 18, 2008, 09:53:16 PM »
Hello,

you can limit the maximum number of URLs in sitemap generator configuration ("Maximum pages" option).