I have a site with about 4900 pages and 775,000 links on those pages.
I never had troubles until recently. Now the site map generator stalls on the 3rd level. My server provider indicates that the sitemap generator was using 100% of the cpu and had to be stopped.
This problem is consistent and does not changes from attemt to attempt.
The only change I have made has been to add about 700 pages (which is included in the 4900 number above). When the task is aborted, it is not crawling one of those pages.
Tech Support (Lexiconn server) seems to think that there is a bug in the sitemap generator cauksing it to go into a loop.
I'm not sure how to go about debugging this further.