XML Sitemaps Generator

Author Topic: Crawler stops at certain point  (Read 12179 times)

stickit

  • Registered Customer
  • Approved member
  • *
  • Posts: 5
Crawler stops at certain point
« on: September 21, 2010, 07:15:14 AM »
Hello !!

After I've changed my hosting I am not able to crawl my site fully. Crawler stops after scanning 420 pages:
-----------------------------------------------------
Links depth: 2
Current page: talladega_logo-NA-25.php
Pages added to sitemap: 379
Pages scanned: 420 (20,490.6 KB)
Pages left: 140 (+ 1959 queued for the next depth level)
Time passed: 0:04:54
Time left: 0:01:38
Memory usage: 2,005.3 Kb
-----------------------------------------------------

I've checked with hosting company and they have modified following two settings in php.ini for me but still the result is same:
mamory_limit = 512M (It was 256M before)
max_execution_time = 9000 (It was 300 before)

Whether I run in Background mode or normal mode, it's not going forward after scanning 420 pages.

Any help please !!

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Crawler stops at certain point
« Reply #1 on: September 21, 2010, 09:16:27 AM »
Hello,

please let me know your generator URL/login in private message to check this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

stickit

  • Registered Customer
  • Approved member
  • *
  • Posts: 5
Re: Crawler stops at certain point
« Reply #2 on: September 21, 2010, 01:28:38 PM »
Where do I send you my generator username/password? In this forum - isn't it open for public?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Crawler stops at certain point
« Reply #3 on: September 21, 2010, 01:29:32 PM »
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

stickit

  • Registered Customer
  • Approved member
  • *
  • Posts: 5
Re: Crawler stops at certain point
« Reply #4 on: September 21, 2010, 01:33:44 PM »
Ok, Done - Sent.

stickit

  • Registered Customer
  • Approved member
  • *
  • Posts: 5
Re: Crawler stops at certain point
« Reply #5 on: September 22, 2010, 06:48:31 PM »
Hi !!

Did you get username/password for my Generator? Any updates?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Crawler stops at certain point
« Reply #6 on: September 22, 2010, 08:28:31 PM »
Hello,

I've repliad to your PM yesterday.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

stickit

  • Registered Customer
  • Approved member
  • *
  • Posts: 5
Re: Crawler stops at certain point
« Reply #7 on: September 23, 2010, 05:22:36 PM »
Hi !!

I am really sorry to trouble you again but can you resend the mail as I think either I've not received it or deleted it by mistake.

Thank you in advance.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: Crawler stops at certain point
« Reply #8 on: September 23, 2010, 08:04:35 PM »
I've resent the message, you can find it here:
http://www.xml-sitemaps.com/forum/index.php?action=pm
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2