XML Sitemaps Generator

Author Topic: Crawling never finished  (Read 1464 times)

adserver

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Crawling never finished
« on: July 24, 2016, 01:46:50 AM »
I bought Unlimited Sitemap Generator two day ago and installed and start crawling but until now (nearly 2 days) the job doesn't finish yet. The processing stops every 5 minutes and I hit button Run to resume the last session for many times.
I send the message that appears every 5 minutes:

"Links depth: 14
Current page: advancedphoto/battle/w_6739/l_5209/mode_/
Pages added to sitemap: 106776
Pages scanned: 108557 (3,810,113.5 KB)
Pages left: 1793 (+ 55091 queued for the next depth level)
Time passed: 9:52:35
Time left: 0:09:47
Memory usage: 121,506.8 Kb
Resuming the last session (last updated: 2016-07-24 01:25:11)
Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 35087104 bytes) in /home/lybach/public_html/generator/pages/class.utils.inc.php on line 102"

I set PHP.INI :
max_execution_time = 9000
memory_limit = 1024M

I tried many hits and this message appeared:
There was an error while retrieving the URL specified: [external links are visible to admins only]
HTTP Code:

HTTP headers:
x_csize: 0

HTTP output:


Please help me!
Mike N
« Last Edit: July 24, 2016, 02:13:47 AM by adserver »

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10621
Re: Crawling never finished
« Reply #1 on: July 24, 2016, 04:33:57 AM »
Hello,

you also need to increase "Maximum memory" setting in generator configuration (256 is default setting).
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

adserver

  • Registered Customer
  • Approved member
  • *
  • Posts: 2
Re: Crawling never finished
« Reply #2 on: July 24, 2016, 05:24:17 AM »
I follow everything as you instruct but nothing better. When I hit the button Run, this message appears:
An error occured
There was an error while retrieving the URL specified: [external links are visible to admins only]
HTTP Code:

HTTP headers:
x_csize: 0

HTTP output:

I don't know what wrong?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10621
Re: Crawling never finished
« Reply #3 on: July 24, 2016, 04:14:39 PM »
Probably there is a configuration problem - it looks like your server doesn't allow local network connections via port 80 (http) - as a result sitemap generator is not able to crawl the site. This is usually related to firewall installed at the host - could you please contact your hosting support regarding this?
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2