There was an error while retrieving the URL specified...
« on: March 24, 2011, 10:25:57 PM »
This message keeps popping up when I try to start the crawler.  I set my IP to 127.0.0.1 and this error is still popping up.  I've also tried leaving off the [ External links are visible to forum administrators only ]  and the error is still there.  I am sitting here connected to my server and it's working just fine, and I know the URL itself is up and running.  Any other suggestions?

The full message reads:
There was an error while retrieving the URL specified: [ External links are visible to forum administrators only ]
HTTP headers follow:
content-type: text/html
date: Thu, 24 Mar 2011 22:15:25 GMT
connection: close
content-length: 34

HTTP output:
Bad Request (Invalid URL)
Re: There was an error while retrieving the URL specified...
« Reply #2 on: March 25, 2011, 04:25:43 PM »
Can you make this a private message?  I'd rather not have the link posted for everyone to see, but I don't have permission to send you a direct message.

[ External links are visible to forum administrators only ]

I did not set up a login yet so it will be whatever the default is.
Re: There was an error while retrieving the URL specified...
« Reply #3 on: March 27, 2011, 11:29:03 AM »
Probably there is a configuration problem - it looks like your server doesn't allow local network connections via port 80 (http) - as a result sitemap generator is not able to crawl the site. This is usually related to firewall installed at the host - could you please contact your hosting support regarding this?
Re: There was an error while retrieving the URL specified...
« Reply #4 on: March 28, 2011, 03:57:59 PM »
Our systems admin assures me this isn't the case.  Is there anything else I can try?  Right now it fluctuates between a "socket error" and "error while retrieving the URL specified".  This is getting very frustrating and I'm worried I wasted my money and should just try different software.
Re: There was an error while retrieving the URL specified...
« Reply #5 on: March 29, 2011, 09:40:29 PM »
Hello,

I just tried to change starting URL in your generator configuration to external website (google.com as an example), and crawler worked fine (limited it to 1 page for testing) - it means that the problem happens only when connecting to local site (same domain where generator is installed) and it's configured by an internal server firewall.
Re: There was an error while retrieving the URL specified...
« Reply #6 on: October 21, 2011, 05:23:42 PM »
Hi I had the same issue, it was because I had a temporary redirect from the home page, check your temp or perm redirects, also I was missing the "www" on the url portion ... once I changed both it worked like a charm!
Cheers!