Not picking up files
« on: March 07, 2009, 01:41:14 PM »
Hi we have just purchased site generator and the setup is fine however it is only picking up two files when we crawl and cannot understand why it is not reading other files?

With other free site generators it picked up 99 files for google please can you help?

thanks David
Re: Not picking up files
« Reply #1 on: March 07, 2009, 05:15:00 PM »
Hello,

please try to change starting URL from www.domain.com to domain.com or vice versa.

Also, please try our SE bot simulator tool to see if links are crawlable: https://www.xml-sitemaps.com/se-bot-simulator.html