Sitemap Generator Forum

I have a site with about 4900 pages and 775,000 links on those pages.

I never had troubles until recently. Now the site map generator stalls on the 3rd level. My server provider indicates that the sitemap generator was using 100% of the cpu and had to be stopped.

This problem is consistent and does not changes from attemt to attempt.

The only change I have made has been to add about 700 pages (which is included in the 4900 number above). When the task is aborted, it is not crawling one of those pages.

Tech Support (Lexiconn server) seems to think that there is a bug in the sitemap generator cauksing it to go into a loop.

I'm not sure how to go about debugging this further.

Hello,

I suggest you to use "Make delay after each X request" option to reduce the server load while crawling. In case if it interrupts the process, you can resume generation from that point.

 I tried a 1 sec delay after 5 accesses  and 1 sec delay after 2 access…server shut us down both times..

 

Does that happen quickly ? How many URLs are scanned to that point?

sorry for the delay in getting back to you..

server killing site map was fixed...software issue..

problem now is that your generator stops at level 3...indicates it is done...



any ideas?

Please install the latest version of Sitemap Generator. Replied to your email.