• Welcome to Sitemap Generator Forum.
 

How to make crawling speed fast?

Started by info2673, November 26, 2014, 03:21:32 PM

Previous topic - Next topic

info2673

Crawling speed of sitemap generation is very slow and always stucks after every few minutes.

I already increased the maximum execution time and memory limit in PHP configuration of my website :- [ External links are visible to forum administrators only ]

But still I am facing issue. There might be something we can do in Crawl Settings in Sitemap Generation application itself that can make the crawling fast.

I want to configure settings in such a way that crawling process becomes fast bypassing additional settings that are not mandatory.


XML-Sitemaps Support

Hello,

The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.