• Welcome to Sitemap Generator Forum.
 

Crawling only 6 pages

Started by bernard, September 24, 2008, 03:45:35 PM

Previous topic - Next topic

bernard

Hi

I've just installed the software and when run it will only index 6 pages before writing the necessary files.  The site ( [ External links are visible to forum administrators only ] ) is crawlable which I confirmed by using your free online version where it indexed the 500 page limit.
Could you please advise me further.

Regards


bernard

Hi
I've overcome that problem by adding www into the site url.  However, I have a further problem as the server does not allow an HTTP request to be sent to the same server to avoid scripts potentially running in infinite loops.  This means that I can't index pages within the same site as the software is installed on.  I can index pages on another server but then I can't save the sitemap on the scanned site.  This situation of course does not allow me to run a cron job to automate updates.

Do you have any thoughts on how I can overcome this problem.

Regards

XML-Sitemaps Support

Hello,

depending on server configuration, possibly there is a proxy service running on the server that allows to open local pages for scripts?
Also, you can setup sitemap generator on external site and then create another cron task on main site just to download resulting sitemap files.