acnz

*
  • *
  • 14
can't find my domain
« on: August 24, 2008, 05:53:29 AM »
I have tried the latest version
The config saves okay, and the pathes look okay, but when I crawl it says it can't find the file (my domain)
That is so weird - what file?

acnz

*
  • *
  • 14
Re: can't find my domain
« Reply #1 on: August 24, 2008, 06:03:01 AM »
Oh the exact error message needs a very fast capture screen, cos it's only there for a moment -- but what I got was:

There was an error while retrieving the URL specified: [ External links are visible to forum administrators only ]
Error message: Error opening socket to [ External links are visible to forum administrators only ]
HTTP headers follow:
HTTP output:

The folder is here:
[ External links are visible to forum administrators only ]

Cheers!

Re: can't find my domain
« Reply #2 on: August 25, 2008, 12:21:14 AM »
Hello,

please try to specify your server's IP address in "Use IP address for crawling" option OR manually enable this setting in config.inc.php file:
'xs_usecurl'

acnz

*
  • *
  • 14
Re: can't find my domain
« Reply #3 on: August 26, 2008, 03:55:14 AM »
Hello there

Yes I tried that, manually setting the server's IP address, but it didn't help.

I just tried setting 'xs_usecurl' => '1', is that what you mean?? Um, no didn't help either

Thanks for your help

acnz

*
  • *
  • 14
Re: can't find my domain
« Reply #5 on: August 27, 2008, 03:52:14 AM »
Thanks Oleg
Tried that, but no difference.

The free online Stemap generator can index it okay, it's the uploaded version can't

Re: can't find my domain
« Reply #6 on: August 28, 2008, 09:57:04 PM »
Hello,

online generator is running from our server, that also confirms that your site is "crawlable".
Probably there is a configuration problem - it looks like your server doesn't allow local network connections via port 80 (http) - as a result sitemap generator is not able to crawl the site. This is usually related to firewall installed at the host - could you please contact your hosting support regarding this?
Also, in some cases this can be fixed with DNS Doctoring operation on the server.

acnz

*
  • *
  • 14
Re: can't find my domain
« Reply #7 on: August 29, 2008, 01:52:29 AM »
Thanks Oleg -- I will contact them

I might also try installing my sitemap programme somewhere else and seeing if it will scan remotely

Cheers!

acnz

*
  • *
  • 14
Re: can't find my domain
« Reply #8 on: September 01, 2008, 03:39:20 AM »
Hmmm, thanks Oleg -- I have been in touch with my provider, and what I didnt know was that part of the hosting package I have includes sharing the IP for a few websites. This domain I've been having trouble with was one of them, and a quick check shows the same error on all shared IP domains. So there's the problem - is your IP shared? In my case, yes. Thanks again
Re: can't find my domain
« Reply #9 on: September 01, 2008, 08:39:17 PM »
It doesn't matter really if IP is shared or not - since you and I and sitemap generator from our server are still able to access to your site, the problem is with DNS configuration on your server, which prevent sitemap generator script to connect to your domain *when installed on your host only*.

acnz

*
  • *
  • 14
Re: can't find my domain
« Reply #10 on: September 02, 2008, 01:22:45 PM »
Yes I just found that out
I went to extra expense, moved it to a dedicated IP and it still doesn't work

The support people are Russian and it just doesn't work. Complete waste of time

I will keep you updated.
Re: can't find my domain
« Reply #11 on: September 03, 2008, 08:07:53 AM »
May be they disallow outgoing http connections on the server at all?

acnz

*
  • *
  • 14
Re: can't find my domain
« Reply #12 on: September 03, 2008, 01:20:28 PM »
Hiya Oleg

Hmmmmm.

This is the help desk query they've just sent me.
It's only been 8 days *grin*

"Please try to contact with developers of this software and ask them what exactly should be done from the server side to fix problem with generating SiteMaps. Thanks."

Exactly what should be done? What sorta question is that?

May I ask them to email you direct?

Kind regards
Russell
Re: can't find my domain
« Reply #13 on: September 04, 2008, 07:59:30 PM »
To be specific, the server configuration should allow PHP code like:
Quote
$content = file('http://www.domain.com');
OR
Quote
$socket = fsockopen('www.domain.com', 80);
so that sitemap generator is able to connect to the site and crawl it.

Please note that this is *default* behaviour, so if it's disallowed on your host it would mean that they have applied something additional (like firewall or other system modification) to prevent it.

acnz

*
  • *
  • 14
Re: can't find my domain
« Reply #14 on: September 11, 2008, 02:56:53 AM »
Hi there Oleg

It looks like they are trying to solve their problem.

I have been asked to ask you,
"What does the program use to open the connect to the website to crawl? Does it use something like cURL or something else? As much information would be helpful."

Kind regards!
Russell