XML Sitemaps Generator

Author Topic: Crawling Stops  (Read 22694 times)

martyn

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Crawling Stops
« on: April 29, 2007, 11:42:33 PM »
I have version v2.4, 2006-10-11 and can not find where to upgrade, I have a problem when I run the crawler it stops part way through crawling my forums and I keep having to put in Do not parse and exclude URL's.
I run for about 10 minutes before all stops, is there a way to stop the crawl failing and also where can I upgrade my version

Cheers


XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10621
Re: Crawling Stops
« Reply #1 on: April 30, 2007, 02:51:24 AM »
Hello,

you should increase max_execution_time and memory_limit settings in your php configuration for that.

The newest version of Sitemap Generator can be obtained using the same download link sent to you in activation email.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

martyn

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Crawling Stops
« Reply #2 on: April 30, 2007, 06:30:54 PM »
can you resend the link as I no longer have the mail that was sent, if you need to validate me I used the mail address the I use here

thanks

Martyn

martyn

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Crawling Stops
« Reply #3 on: April 30, 2007, 06:53:39 PM »
set all the defaults to unlimited and it still stops, any ideas

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10621
Re: Crawling Stops
« Reply #4 on: May 01, 2007, 09:45:26 PM »
Hello,

I've resent you the download link.

Please note that you should increase setting values in *php* configuration (php.ini), NOT in Sitemap Generator configuration.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

martyn

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Crawling Stops
« Reply #5 on: May 06, 2007, 05:26:53 PM »
Thanks for the link, when you say change php.ini do you mean on the server?

Also I have a new issue. can you help out please

Links depth: 6
Current page: portal/forum/showthread.php?goto=lastpost&t=6467
Pages added to sitemap: 7131
Pages scanned: 8580 (203,417.1 Kb)
Pages left: 9359 (+ 10182 queued for the next depth level)
Time passed: 59:15
Time left: 64:38
Memory usage: 15,560.7 Kb
Resuming the last session (last updated: 2007-05-03 23:15:12)
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 6291374 bytes) in D:\sitename\wwwroot\generator\pages\class.grab.inc.php(2) : eval()'d code on line 298

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10621
Re: Crawling Stops
« Reply #6 on: May 07, 2007, 01:26:51 PM »
Hello,

yes, I mean php configuration on your server (php.ini file).
Quote
Also I have a new issue. can you help out please
This is related to the same php config issue.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

martyn

  • Registered Customer
  • Approved member
  • *
  • Posts: 4
Re: Crawling Stops
« Reply #7 on: May 08, 2007, 08:02:43 PM »
its not my server i'm on a hosted server, how can I get this sorted

Thanks

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10621
Re: Crawling Stops
« Reply #8 on: May 09, 2007, 01:33:05 AM »
Please ask your host if they allow custom php config parameters definition.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2