XML Sitemaps Generator

Author Topic: Crawler Reaches a Particular URL and Locks-up  (Read 8564 times)


  • Registered Customer
  • Approved member
  • *
  • Posts: 3
Crawler Reaches a Particular URL and Locks-up
« on: December 26, 2011, 04:04:59 PM »
The crawler has been crawling my site for the past 10 hours. Every time it reaches one particular URL it stalls, restarts, stalls, restarts. It has been doing this endlessly.

Here is the info displayed:

Links depth: 4
Current page: gallery/photo.php?exhibition=61&photo=3501&ee_lang=eng
Pages added to sitemap: 11695
Pages scanned: 11720 (171,319.6 KB)
Pages left: 48308 (+ 0 queued for the next depth level)
Time passed: 0:44:44
Time left: 3:04:26
Memory usage: 51,680.1 Kb

I checked the URL and it seems unremarkable. It's just another photo in a gallery app.



XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10599
Re: Crawler Reaches a Particular URL and Locks-up
« Reply #1 on: December 27, 2011, 11:04:45 AM »

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.


SMF 2.0.12 | SMF © 2014, Simple Machines