Only crawls one URL - the homepage
« on: September 04, 2008, 02:29:50 PM »
Hello,

I have installed the latest version (2.9) on a site.  I have run the crawl and it only crawls the homepage.

I have searched these forums but not found an answer.

Can someone please explain? 

There should be thousands of URLs.  I bought this software because the CoffeCup software is limited to a few thousand URLs and I needed something capable of crawling large sites.

Thanks.
Re: Only crawls one URL - the homepage
« Reply #1 on: September 04, 2008, 08:04:50 PM »
Hello,

please check your site with search engine robot simulator tool to make sure that your pages are crawlable: https://www.xml-sitemaps.com/se-bot-simulator.html