Sitemap Generator Forum

I have tried the latest version
The config saves okay, and the pathes look okay, but when I crawl it says it can't find the file (my domain)
That is so weird - what file?

Oh the exact error message needs a very fast capture screen, cos it's only there for a moment -- but what I got was:

There was an error while retrieving the URL specified: [ External links are visible to forum administrators only ]
Error message: Error opening socket to [ External links are visible to forum administrators only ]
HTTP headers follow:
HTTP output:

The folder is here:
[ External links are visible to forum administrators only ]

Cheers!

Hello,

please try to specify your server's IP address in "Use IP address for crawling" option OR manually enable this setting in config.inc.php file:
'xs_usecurl'

Hello there

Yes I tried that, manually setting the server's IP address, but it didn't help.

I just tried setting 'xs_usecurl' => '1', is that what you mean?? Um, no didn't help either

Thanks for your help

Please also try setting 127.0.0.1 as IP address.

Thanks Oleg
Tried that, but no difference.

The free online Stemap generator can index it okay, it's the uploaded version can't

Hello,

online generator is running from our server, that also confirms that your site is "crawlable".
Probably there is a configuration problem - it looks like your server doesn't allow local network connections via port 80 (http) - as a result sitemap generator is not able to crawl the site. This is usually related to firewall installed at the host - could you please contact your hosting support regarding this?
Also, in some cases this can be fixed with DNS Doctoring operation on the server.

Thanks Oleg -- I will contact them

I might also try installing my sitemap programme somewhere else and seeing if it will scan remotely

Cheers!

Hmmm, thanks Oleg -- I have been in touch with my provider, and what I didnt know was that part of the hosting package I have includes sharing the IP for a few websites. This domain I've been having trouble with was one of them, and a quick check shows the same error on all shared IP domains. So there's the problem - is your IP shared? In my case, yes. Thanks again

It doesn't matter really if IP is shared or not - since you and I and sitemap generator from our server are still able to access to your site, the problem is with DNS configuration on your server, which prevent sitemap generator script to connect to your domain *when installed on your host only*.

Yes I just found that out
I went to extra expense, moved it to a dedicated IP and it still doesn't work

The support people are Russian and it just doesn't work. Complete waste of time

I will keep you updated.

May be they disallow outgoing http connections on the server at all?

Hiya Oleg

Hmmmmm.

This is the help desk query they've just sent me.
It's only been 8 days *grin*

"Please try to contact with developers of this software and ask them what exactly should be done from the server side to fix problem with generating SiteMaps. Thanks."

Exactly what should be done? What sorta question is that?

May I ask them to email you direct?

Kind regards
Russell

To be specific, the server configuration should allow PHP code like:

$content = file('[ External links are visible to logged in users only ]');

OR

$socket = fsockopen('[ External links are visible to logged in users only ]', 80);

so that sitemap generator is able to connect to the site and crawl it.

Please note that this is *default* behaviour, so if it's disallowed on your host it would mean that they have applied something additional (like firewall or other system modification) to prevent it.

Hi there Oleg

It looks like they are trying to solve their problem.

I have been asked to ask you,
"What does the program use to open the connect to the website to crawl? Does it use something like cURL or something else? As much information would be helpful."

Kind regards!
Russell