XML Sitemaps Generator

Author Topic: Crawling for other site  (Read 7912 times)

ngftlaudhosp

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 20
Crawling for other site
« on: June 17, 2011, 12:46:35 PM »
Can I install the sitemap generator at another account?

Its an addon domain. So im wondering if we are allowed to use the same generator program in two locations, two sites.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10623
Re: Crawling for other site
« Reply #1 on: June 19, 2011, 07:16:17 AM »
Hello,

multiple copies of the Software may be installed and used, but for internal use only, for your personal websites. It cannot be used to create sitemaps for 3rd party sites or provided a service to others (paid or free).
http://www.xml-sitemaps.com/license.html
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

ngftlaudhosp

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 20
Re: Crawling for other site
« Reply #2 on: June 22, 2011, 05:07:08 PM »
Thank you. I need support on something that happened today, the whole forums directory was getting a 500 internal server error. My host said:

----START MESSAGE----

I do see a slight spike in memory usage between 8 AM - 11 AM today, but it wasn't very large. It did drop down at 11:20 AM which seems to coincide with the processes being killed, but unfortunately the logs are not conclusive enough to show what was causing the problem and without the processes running it is difficult to say what was causing the issue.

From what I can see it's likely these two files:

/home/ab2168/public_html/generator/runcrawl.php
/home/ab2168/public_html/generator/index.php

----END MESSAGE----

Is there anything I can do so that it is less intensive on the server?

Ive deleted all the logs it stores in generator/data is there anything else I can do?
Thanks.

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10623
Re: Crawling for other site
« Reply #3 on: June 23, 2011, 11:31:32 AM »
Hello,

you can use "Make delay for X seconds after each X request" - that tells generator's crawler to "sleep" for some time between requests to slow down the crawling speed and reduce the server load as a result.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

ngftlaudhosp

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 20
Re: Crawling for other site
« Reply #4 on: June 23, 2011, 06:32:32 PM »
[external links are visible to admins only]
Hello,

you can use "Make delay for X seconds after each X request" - that tells generator's crawler to "sleep" for some time between requests to slow down the crawling speed and reduce the server load as a result.

Thanks, you can close this thread now if you like.

 

SMF 2.0.12 | SMF © 2014, Simple Machines
XHTML RSS WAP2