XML Sitemaps Generator

Author Topic: limit pages crawled in folder  (Read 14944 times)

craig5

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
limit pages crawled in folder
« on: August 18, 2008, 03:58:20 PM »
Hi

I am opening up a new section on the website with a calendar using canendar software. As this is a calendar it is infinate.

Is it possible to limit the amount of pages that it crawls in that folder. ::)

The main reason i use this software is to create the sitemap.xml file that google and the other search engines look for.

I tried stoping the crawl by loggin out as there was no other way to stop the crawl and i do not know if the sitemap.xml file has been updated and can not look at the list as it does not update when you stop the crawl this way.

Can someone help. Other wise the crawl will go on forever... :'(

Craig

craig5

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Re: limit pages crawled in folder
« Reply #1 on: August 18, 2008, 04:03:01 PM »
Hi

also because this folder go's o forever can not be sure that all the other folders are not indexed.

craig

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: limit pages crawled in folder
« Reply #2 on: August 18, 2008, 08:53:16 PM »
Hello,

you can limit the maximum number of URLs in sitemap generator configuration ("Maximum pages" option).
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2