XML Sitemaps Generator

Author Topic: crawler problem  (Read 5421 times)

saicom2

  • Registered Customer
  • Approved member
  • *
  • Posts: 3
crawler problem
« on: August 24, 2014, 08:45:59 AM »
Crawler is very slow

pages added in sitemap 2800 in 1 hour

can you help

Regards

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10621
Re: crawler problem
« Reply #1 on: August 24, 2014, 07:08:12 PM »
Hello,

The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2