There was an error while retrieving the URL specified
« on: March 16, 2016, 05:43:27 PM »
I cant crawl.
error message:

An error occured
There was an error while retrieving the URL specified: [ External links are visible to forum administrators only ]
HTTP Code:

HTTP headers:
x_csize: 0

HTTP output:
Re: There was an error while retrieving the URL specified
« Reply #2 on: March 17, 2016, 03:06:12 PM »
I did send it to you yesterday.
Re: There was an error while retrieving the URL specified
« Reply #4 on: March 20, 2016, 12:09:43 PM »
Hello,

I have moved our web site to HTTPS now and trying to crawl the site again with the Unlimited Sitemap Generator gives me the exact same error as specified in this post. I have also tried to crawl it by its public IP address, the same error appears. I had no issues crawling the web site when it was still on HTTP. Obviously, I am using now the updated site URL with HTTPS.

Please advise how to proceed further.
Re: There was an error while retrieving the URL specified
« Reply #6 on: March 21, 2016, 05:38:34 PM »
Hello,

Enabling this setting actually overcome the error that was seen before. However, after crawling starts the job is finished within 2 seconds and crawled pages are zero. I have no crawler limitations set such as page depth or maximum pages crawled ...

Urllist.txt is empty. Generated .log files have binary content, not sure how to decode those ...

What can I check next please?

Regards,
Re: There was an error while retrieving the URL specified
« Reply #8 on: March 22, 2016, 10:16:18 PM »
Hello,

Just sent the private message with the URL I am trying to crawl.

Regards,