notifying host of crawling?
« on: September 10, 2006, 03:16:28 PM »
Hello All,

I am learning as I go. Thanks in advance if these questions are so simplistic that they make you yawn!
Does a person need to notify the webhost of adding this generator to my site as it relates to memory concerns? Will this generator cause the hosting company any problems that might jeopardize my relationship with them? I realize this tool will run on the hosting site server and be monitored from my local browser. Is the memory of my local machine (my pc at home) critical for this process. I have tried java based crawlers before and my local machine bombs out due to memory. (yes, I know I shouldn't be such a tight wad and I will eventually grade up). This product appears to be pretty cool and very well priced. What else could be a pitfall?

Regards,
SydneyG
Re: notifying host of crawling?
« Reply #1 on: September 11, 2006, 02:09:52 AM »
Hello SydneyG,

the amount of memory depends on the number of pages your site contains.
You will be able to limit the maximum number of pages in configuration first if you are concerned about this, look how it goes, and increase it then if everything is ok.
The script is running at the server side, so amount of memory on local PC doesn't matter in this case.