spidering limit problem
« on: July 07, 2006, 10:57:31 AM »
Dear XML sitemap

First, what a fantastic product. By far the best generation tool i have discovered on the internet.

My problem is that our company are running our website on a red hat server [as per recommendations of the developer] and internal skills are all MS. Due to a series of issues, we neither have the capability nor confidnece that the unlimited sitemap generator will cause problems.

The difficulty in for us to develop sitemaps is that we have a bespoke file extension - .ice [ External links are visible to forum administrators only ] which many generation tools are sadly unable to spider further than the home page.

Your online tool is the first to spider properly, but unfortuantely as your limit is 500 pages - which is totally understandable - i don't really know how to proceed from here.

Many thanks

Re: spidering limit problem
« Reply #1 on: July 07, 2006, 11:03:29 PM »

in case if Online Sitemap Generator is crawling your website successfully, the Standalone Generator will do as well.
You can find the system requirements for standalone version here:
Re: spidering limit problem
« Reply #2 on: July 08, 2006, 09:24:41 AM »
Thnaks for yuor reply.
unfortunately, our problem is that as a company there are no skills red hat/apache/python etc, and our server is blocked by the develop preventing external access to third parties.

do you have any explanation why your sgerantor works of our site, and the majority cannot spider beyond the homepage?
Re: spidering limit problem
« Reply #3 on: July 08, 2006, 10:02:33 PM »

I believe that you should contact authors of other generators to find out why they are not working :)