Change Generator from v8.0 to v7.2 complete the Sitemap
« on: July 10, 2018, 01:20:23 PM »
Hello,

I could not complete the sitemap with v8.0. Some of the issues I was having is that the generator will:

  • Skipped a number of URL when crawling large amount
  • When it goes to depth 5 or 6 (around 10K quere), it will always reset to depth 1.

I should of copy and paste the error it was getting. It was referring to an error on generator/page/(not sure which files) but it was line 165. It wasn't allow to post something duplicate, but I do not have any duplicate urls.

Another error is that the script will not saved at a certain depth. After I resume, the crawler will skipped the remaining files and go to the next level or restart to depth 1.

The pages are html. I never have the issue before. Since, I'm using a shared web hosting, I downgrade from v8.0 to v7.2 to complete the crawl. I have to remove the php ini and use cron job to finish.
« Last Edit: July 10, 2018, 01:27:49 PM by procardshannonw »
Re: Change Generator from v8.0 to v7.2 complete the Sitemap
« Reply #1 on: January 21, 2019, 09:43:07 PM »
Still no reply
Re: Change Generator from v8.0 to v7.2 complete the Sitemap
« Reply #2 on: January 22, 2019, 04:28:50 PM »
Hello,

v8.0 includes new updates that improve it but it could also increase resource usage.