script is to slow
« on: July 06, 2022, 02:00:31 AM »
i have a lot of urls to scan is their a way to make the script use more resources shush as CPU RAM
Re: script is to slow
« Reply #1 on: July 08, 2022, 11:45:27 AM »
Hello,
The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
Re: script is to slow
« Reply #2 on: July 31, 2022, 02:48:27 PM »
Is any method or setting can be make it faster for sitemap generator? because i have more then 3.5Lakhs URL in my website
Re: script is to slow
« Reply #3 on: August 02, 2022, 01:39:41 PM »
Hello,


The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.