1500 pages more than yesterday !!
« on: September 30, 2011, 09:18:17 PM »
hi, i installed this a few days ago, i have run the crawl and it gave me something like 580 pages, i have ran it again today and now it tells me i have 2800 pages and i haven't done any updates to the website?


Re: 1500 pages more than yesterday !!
« Reply #1 on: October 01, 2011, 10:27:53 AM »
Possibly the website is restricting the access from the same IP, disallowing to get too many pages in a short time, so it failed the first time.
Re: 1500 pages more than yesterday !!
« Reply #2 on: October 01, 2011, 10:42:01 AM »
Hi thanks for the reply, i dont think thats the issue as there are no errors as far as i can see.

i have uninstalled it and ran it again and i am getting 2800 pages, i have now deleted the xml sitemap until i can sort it out.
regards
steve