• Welcome to Sitemap Generator Forum.
 

script not crawling deeply enough

Started by jr5, May 17, 2019, 05:15:12 AM

Previous topic - Next topic

jr5

I have a page that has potentially 50,000 listings on it, and I want to be able to sitemap and index them all.  I had some sub-pages with a lot of links set on the home page, and it was crawling those and I would have a sitemap of about 1500 pages.  So to help get a sitemap with all of the pages, I experimented with a footer link with links of links to all the pages.  Since doing this, I have only gotten less than 300 pages on a crawl.

I checked the configuration and it says "maximum depth level", and it is set to 0, which says "unlimited".  However, it is not doing this.

How do I get it to crawl every single link on every single page?

XML-Sitemaps Support

Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached starting from homepage?
 

jr5

Hi, I sent the PM's to you last Friday, but I haven't heard back yet.  Can you please look at it and respond to me?  Thank you.

jr5

Hi sorry my email didn't notify me that you had privately responded already, I see the message, thank you for your help.