Sitemap won't crawl a specific folder
« on: March 26, 2009, 11:25:44 AM »
I have installed the sitemap generator and it crawls all the pages on my site except the ones in a specific directory.

I have looked to see if there is any no follows and robot.txt but everything seems to be in order.

This is the most important directory on my site as it contains 20,000+ of my products.

http://www.easycrane.co.uk
WON@T CRAWL - http://www.easycrane.co.uk/office-supplies/


do you have an suggestions of why this would happen?

thanks
Re: Sitemap won't crawl a specific folder
« Reply #1 on: March 26, 2009, 03:09:47 PM »
problem solved. don't know how but its now working.

next problem.....

crawling stops at 1804 pages and won't go no further. all very frustrating!!!!!!!!!!!! :-\
Re: Sitemap won't crawl a specific folder
« Reply #2 on: March 27, 2009, 01:02:39 AM »
Hello,

please PM me your generator URL/login.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.