Crawling never finished
« on: July 24, 2016, 02:46:50 AM »
I bought Unlimited Sitemap Generator two day ago and installed and start crawling but until now (nearly 2 days) the job doesn't finish yet. The processing stops every 5 minutes and I hit button Run to resume the last session for many times.
I send the message that appears every 5 minutes:

"Links depth: 14
Current page: advancedphoto/battle/w_6739/l_5209/mode_/
Pages added to sitemap: 106776
Pages scanned: 108557 (3,810,113.5 KB)
Pages left: 1793 (+ 55091 queued for the next depth level)
Time passed: 9:52:35
Time left: 0:09:47
Memory usage: 121,506.8 Kb
Resuming the last session (last updated: 2016-07-24 01:25:11)
Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 35087104 bytes) in /home/lybach/public_html/generator/pages/class.utils.inc.php on line 102"

I set PHP.INI :
max_execution_time = 9000
memory_limit = 1024M

I tried many hits and this message appeared:
There was an error while retrieving the URL specified: [ External links are visible to forum administrators only ]
HTTP Code:

HTTP headers:
x_csize: 0

HTTP output:


Please help me!
Mike N
« Last Edit: July 24, 2016, 03:13:47 AM by adserver »
Re: Crawling never finished
« Reply #1 on: July 24, 2016, 05:33:57 AM »
Hello,

you also need to increase "Maximum memory" setting in generator configuration (256 is default setting).
Re: Crawling never finished
« Reply #2 on: July 24, 2016, 06:24:17 AM »
I follow everything as you instruct but nothing better. When I hit the button Run, this message appears:
An error occured
There was an error while retrieving the URL specified: [ External links are visible to forum administrators only ]
HTTP Code:

HTTP headers:
x_csize: 0

HTTP output:

I don't know what wrong?
Re: Crawling never finished
« Reply #3 on: July 24, 2016, 05:14:39 PM »
Probably there is a configuration problem - it looks like your server doesn't allow local network connections via port 80 (http) - as a result sitemap generator is not able to crawl the site. This is usually related to firewall installed at the host - could you please contact your hosting support regarding this?