XML Sitemaps Generator

Author Topic: How to make crawling speed fast?  (Read 6928 times)


  • Registered Customer
  • Approved member
  • *
  • Posts: 1
How to make crawling speed fast?
« on: November 26, 2014, 03:21:32 PM »
Crawling speed of sitemap generation is very slow and always stucks after every few minutes.

I already increased the maximum execution time and memory limit in PHP configuration of my website :- [external links are visible to admins only]

But still I am facing issue. There might be something we can do in Crawl Settings in Sitemap Generation application itself that can make the crawling fast.

I want to configure settings in such a way that crawling process becomes fast bypassing additional settings that are not mandatory.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10900
Re: How to make crawling speed fast?
« Reply #1 on: November 27, 2014, 04:33:07 AM »

The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.

Oleg Ignatiuk
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.


SMF 2.0.14 | SMF © 2017, Simple Machines