Change Generator from v8.0 to v7.2 complete the Sitemap
« on: July 10, 2018, 01:20:23 PM »

I could not complete the sitemap with v8.0. Some of the issues I was having is that the generator will:

  • Skipped a number of URL when crawling large amount
  • When it goes to depth 5 or 6 (around 10K quere), it will always reset to depth 1.

I should of copy and paste the error it was getting. It was referring to an error on generator/page/(not sure which files) but it was line 165. It wasn't allow to post something duplicate, but I do not have any duplicate urls.

Another error is that the script will not saved at a certain depth. After I resume, the crawler will skipped the remaining files and go to the next level or restart to depth 1.

The pages are html. I never have the issue before. Since, I'm using a shared web hosting, I downgrade from v8.0 to v7.2 to complete the crawl. I have to remove the php ini and use cron job to finish.
« Last Edit: July 10, 2018, 01:27:49 PM by procardshannonw »