Strange crawling problem
« on: January 14, 2008, 09:03:53 PM »
Hello,

I recently purchased a Sitemap Generator copy. I first had some problems with the crawling but when i increased the max_execution_time setting it worked fine and crawled about 9000 pages everytime i started crawling.

Since I update my site every day, I just wanted to start crawling again and then all of a sudden it only crawled 84 pages. I've tried it about 20 times now but it always only crawls 84 pages. What is going on? There really seems to be something wrong, the crawl page also always shows me that the link depth value is 1, even though i set it to 2.

What should I do now? Re-install the script? I need some help here please.
Re: Strange crawling problem
« Reply #1 on: January 15, 2008, 12:19:01 AM »
Hello,

please let me know your generator URL and an example URL that is not included in sitemap via private message.
Re: Strange crawling problem
« Reply #2 on: January 15, 2008, 06:37:29 PM »
sent you a pm, please let me know if you haven't got it