Only one directory is found.
I have set the starting URL to my domain's root e.g. [external links are visible to admins only]
The crawler is completing with no errors as if all it could find was that one directory.
I ran the online version and everything works as expected, e.g., crawled the entire site.
Thanks for your help,
Its working now that I removed the www in the URL.
modified the URL to [external links are visible to admins only]