XML Sitemaps Generator

Author Topic: My hosting will block my IP when I crawling my website after a while  (Read 6246 times)

hctodd

  • Registered Customer
  • Approved member
  • *
  • Posts: 9
I use XML-sitemaps crawling my website after a while, my server will blocking my IP automatically for about 30 minutes.
How can I resolve it?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: My hosting will block my IP when I crawling my website after a while
« Reply #1 on: June 24, 2013, 06:19:02 AM »
You need to contact your hosting support to white list your own server IP from blocking.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

hctodd

  • Registered Customer
  • Approved member
  • *
  • Posts: 9
Re: My hosting will block my IP when I crawling my website after a while
« Reply #2 on: June 25, 2013, 04:49:54 AM »
[external links are visible to admins only]
You need to contact your hosting support to white list your own server IP from blocking.

I'm using VPS, so can you teach me how to white list my IP in WHM?

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10624
Re: My hosting will block my IP when I crawling my website after a while
« Reply #3 on: June 28, 2013, 01:14:25 PM »
It depends on the server configuration, i.e. where exactly it is blocked - whitelisting should be done in the same place.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2