XML Sitemaps Generator

Author Topic: Problem with run crawling  (Read 6436 times)

info2635

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Problem with run crawling
« on: October 02, 2014, 10:26:40 AM »
Hi
Excuseme but i don't speak english
I have a web site from 950000 url
I have buy this software and installa folder generator with permession (view tutorial)

I went to create the page url (Crawling --> RUN) but after 20 it stops scanning

I contacted my hosting (aruba.it) and the php.ini file is configured in this way
memory_limit 128 MB
max_execution_time 120

Can i help me?  ::)

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Problem with run crawling
« Reply #1 on: October 02, 2014, 10:40:28 AM »
Hello,

replied to your email.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

info2635

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: Problem with run crawling
« Reply #2 on: October 02, 2014, 10:55:34 AM »
I reply your email
In this page there are url with:
Category --> Provincia --> City

for example: [external links are visible to admins only]

In italy there are 110 province and 8000 city and in the web site there are 70-80 category

In total there are 950000 url

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Problem with run crawling
« Reply #3 on: October 02, 2014, 11:33:14 AM »
Hello,

you need to separate that huge page on multiple pages, for instance one province listed per page.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

info2635

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: Problem with run crawling
« Reply #4 on: October 02, 2014, 12:58:53 PM »
Ok I'm editing the pages'll let you know

Thank you

info2635

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: Problem with run crawling
« Reply #5 on: October 02, 2014, 01:36:36 PM »
I have this error
 Links depth: 2
Current page: ...../......html
Pages added to sitemap: 2514
Pages scanned: 2515 (852,397.1 KB)
Pages left: 7941 (+ 176429 queued for the next depth level)
Time passed: 0:12:38
Time left: 0:39:53
Memory usage: 126,527.8 Kb
Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 26669050 bytes) in /web/htdocs/www.namesite.com/home/generator/pages/class.utils.inc.php on line 102

????

info2635

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: Problem with run crawling
« Reply #6 on: October 03, 2014, 12:55:38 PM »
I don't have sa sitemap

from 6 days i don't a sitemap

This software are erro for create 960.000 url in my site

I don't understand.
Can i help me?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Problem with run crawling
« Reply #7 on: October 04, 2014, 11:43:17 AM »
Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
 
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

info2635

  • Registered Customer
  • Approved member
  • *
  • Posts: 6
Re: Problem with run crawling
« Reply #8 on: October 04, 2014, 02:04:10 PM »
I have to buy a hosting I want to know what should be the minimum value  memory_limit  and max_execution_time to be able to run your software

Thamk you

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Problem with run crawling
« Reply #9 on: October 05, 2014, 08:20:26 AM »
Hello,

it depends on many factors, I'd try 512M.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2