Generator stuck on the same page
« on: February 23, 2018, 07:15:44 AM »
These are my first runs of the generator.  Initially it did not appear to be getting very far and so I excluded a particular type of link.  The excluded link was to about 650,000 generated pages.  This would leave a remaining estimated 400,000 links.

After the generator had been running for an hour+, I looked to see how far it had gone by pressing "view sitemap" button and then the "crawling" button.  I have repeated this a number of times.  Each time it reports that it has crawled 1040 and added 1032 in sitemap and the same current page:

Updated on 2018-02-23 06:58:00 (232 seconds ago) , Time elapsed: 0:00:29,
Pages crawled: 1040 (1032 added in sitemap), Queued: 413, Depth level: 2
Current page: public/cemetery.php?cemetery=2394 (0.0)

I resume by clicking the "Continue the interrupted session" box and pressing the "run" button.

The website being crawled is on its own 950 dedicated server.
Re: Generator stuck on the same page
« Reply #1 on: February 23, 2018, 05:13:26 PM »
Hello,  in this case I would recommend to run generator in command line if you have ssh access to your server.
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.