• Welcome to Sitemap Generator Forum.
 

crawler wont scan SMF forum directory?

Started by QuizToon, February 01, 2011, 02:11:29 PM

Previous topic - Next topic

QuizToon

Hi,

Our generator crawls the main website fine and produces XML and ROR sitemaps without problem, however I have noticed it only covers 48 links

I'm not sure why but it seems to miss out our forum installation in folder /smf

When I run a manual 'free' scan from the [ External links are visible to forum administrators only ] website to [ External links are visible to forum administrators only ] it scans the folder fine and produces an output file too (albeit limited to 500 url's)

I've checked all permissions and everything seems fine (755), there is no exclusion of this folder in the config.

Please help

KR