Standalone Generator Not Crawling Subdirectories?
« on: August 21, 2021, 06:38:24 PM »
I'm trying to get my standalone php generator to create a sitemap of my website but it's only listing pages from the directory I start it in. I have have various pages organised into subdirectories and I cannot find anywhere that tells me how to make it crawl those directories. All i can find in the documentation is "The crawler will explore only the URLs within the starting directory" but I do not believe this can be true. It crawls and lists images stored in sub directories but not .htm pages.  Can you help, I'm probably missing something really simple. Thanks!
Re: Standalone Generator Not Crawling Subdirectories?
« Reply #1 on: August 24, 2021, 05:35:41 AM »

there must be a way to reach any page by clicking links, starting from homepage, so that generator's crawler can find them.