Unlimited Sitemap Generator v8.0 doesn't crawl all pages

[vaniacpsoares]

Hi!

First I want to say that I love your product! But I'm facing a problem:

Generator v8.0
Use server's response
Number of URLs: 40000
URLs per file: 10
Maximum pages: 0
Maximum depth level: 150
Maximum execution time, seconds: 0
Maximum memory usage, MB: 512
Use robots.txt file: yes

My sitemap:
[ External links are visible to forum administrators only ]

The sitemap generates with no errors, but doesn't do a ton of pages, I don't understand why, it always stops at 2787 links, for example he completely ignores this link:

[ External links are visible to forum administrators only ]

You can search the sitemap for 5607322395156 and it's not there, please help, thanks!

[XML-Sitemaps Support]

Hello,
please let me know how that page can be reached, starting from homepage.

[vaniacpsoares]

Just like others that are included in the sitemap, that's the strange part, anyway:

[ External links are visible to forum administrators only ] (left side categories, click "Placas de derivados de madeira e outros")
[ External links are visible to forum administrators only ] (you have several pages)
[ External links are visible to forum administrators only ] (is at page 15, enter it)
[ External links are visible to forum administrators only ] (done!)

The strange part is that the sitemap has products from this category, just search for "placas":

[ External links are visible to forum administrators only ]

Link 456, 457, 458 and others, I don't get it! Thanks for reply and please help!

[XML-Sitemaps Support]

Hello,

the  pagination links have rel=nofollow attribute on your website:

Code Select Expand

<a rel="nofollow" href="/21-placas-de-derivados-de-madeira-e-outros?p=2">

[vaniacpsoares]

I'm going to investigate asap, will reply when I have more news, thanks.

[vaniacpsoares]

Here is my report.

I contacted my provider and after a couple of tests we came to the conclusion that unfortunately your generator doesn't work properly with Prestashop (not sure for other online stores)

It works and adds all links, you should consider updating your script, thanks again.

[XML-Sitemaps Support]

Hello,

as long as rel=nofollow attribute is removed from links on your website it should work fine.

Also, you can disable "Support robots" setting in generator configuration to ignore those robots attributes.

[vaniacpsoares]

I can't change source code of Prestashop, it's not practical, and like I said, the other crawler does all the pages.

I'm still gonna use both, but you guys should consider revising your code, that said can you please check your inbox? Thanks again!

[XML-Sitemaps Support]

You can disable the setting as described above if you want to instruct generator bot to ignore the directive specified in the link on your website.