not all pages are in sitemap.xml
« on: March 04, 2014, 06:45:06 PM »
Hello

we have tried to use sitemap generator but more than 100K pages are not indexed and i 'm not sure why.


i have a lot of error in debug.log

({skipped actor/actorsprofile.php?ad_id=44169 - 402 Payment Required})

but when i access the page is just fine.
Please advice.
« Last Edit: March 04, 2014, 07:45:16 PM by support304 »
Re: not all pages are in sitemap.xml
« Reply #1 on: March 06, 2014, 04:52:32 AM »
Hello,

it's possible that the server couldn't handle the number of requests generator crawler is sending. You can try to use "Make delay" setting to slow down crawling and avoid overloading server.
Re: not all pages are in sitemap.xml
« Reply #2 on: March 07, 2014, 11:46:38 AM »
i had tried with a 2 seconds  delay between each requests  but still is failing.
Re: not all pages are in sitemap.xml
« Reply #3 on: March 10, 2014, 07:04:43 AM »
Hello,

please contact your site developer to find out why "402 Payment Required" response is returned by the site, there might be other reasons (possibly site blocks generator crawler after a while).
Re: not all pages are in sitemap.xml
« Reply #4 on: March 21, 2014, 01:24:42 PM »
ok we resolve that 402 but now we get

*** *** [ External links are visible to forum administrators only ]
 # RETRY - 400 Bad Request - 400 - #

all pages like
[ External links are visible to forum administrators only ] we get it .

Re: not all pages are in sitemap.xml
« Reply #5 on: March 23, 2014, 04:12:01 PM »
Hello,

this error message is caused by the same reason (server overloaded).
Re: not all pages are in sitemap.xml
« Reply #6 on: March 27, 2014, 07:23:28 AM »
Error loading stylesheet: An unknown error has occurred (805303f4)https://www.talentpages.com/generator/pages/mods/sitemap.xsl

what does it means ? Where is problem ?