Only first page is being scanned...?
« on: February 11, 2013, 05:37:21 PM »
My website I am creating the sitemap on has content with multiple pages that the user must click Next page on to view. The crawling process does not recognize the next page and therefore only scans the first page of content. This leaves my sitemap missing literally thousands of objects that should be in the sitemap.

How do I configure the generator to recognize that there are multiple pages and scan appropriately?

Thanks

Bernard
Re: Only first page is being scanned...?
« Reply #1 on: February 11, 2013, 07:53:28 PM »
just some more info: The original page might have a URL [ External links are visible to forum administrators only ] . The next page will have [ External links are visible to forum administrators only ]

I need the page 2 and as many pages it will take to display all items to be craweled
Re: Only first page is being scanned...?
« Reply #2 on: February 11, 2013, 09:40:06 PM »
Hello,

please make sure that pagination links are included in html source code as <a href=""> tags to make them visible to bots. You can check if links are "crawlable" using this page:
https://www.xml-sitemaps.com/se-bot-simulator.html