Hello,
I could not complete the sitemap with v8.0. Some of the issues I was having is that the generator will:
- Skipped a number of URL when crawling large amount
- When it goes to depth 5 or 6 (around 10K quere), it will always reset to depth 1.
I should of copy and paste the error it was getting. It was referring to an error on generator/page/(not sure which files) but it was line 165. It wasn't allow to post something duplicate, but I do not have any duplicate urls.
Another error is that the script will not saved at a certain depth. After I resume, the crawler will skipped the remaining files and go to the next level or restart to depth 1.
The pages are html. I never have the issue before. Since, I'm using a shared web hosting, I downgrade from v8.0 to v7.2 to complete the crawl. I have to remove the php ini and use cron job to finish.
Guest