Not crawling my site

[Renate Schedl]

Have installed the generator to 2 of my domains. Everything is exactly the same. On the one domain it is crawling without any dificulties. On the other it starts crawling and immediately ends with an error mesage, which I cannot read because it is only shown with a galnce. Disappears immediately.

Cannot find any difference in the installation of both sites. Can my domain be the reason for it? Can there be something which does not allow to crawl.

With this domain I as well have problems to get it crawled by Google. It still is not indexed yet. Have included a robot.txt file and google accepts the text of robot.txt.

Anybody here, who has an idea what can be wrong?

[XML-Sitemaps Support]

Hello,

please try to check your site with our search engine crawler simulator tool: https://www.xml-sitemaps.com/se-bot-simulator.html

[Renate Schedl]

I did, what you suggested. Got the answer:
There was an error while retrieving the URL specified

Now what. How can I find out, what the trouble is?

[Renate Schedl]

Can it have anything to do with the "follow" or "non follow". May be I made a mistake with a special plugin. Do not really understand this "follow" stuff.

[XML-Sitemaps Support]

It might be related, please let me know your website URL so that I can check.

[Renate Schedl]

[ External links are visible to forum administrators only ]

[Renate Schedl]

I found the mistake right now. Had the domain written in the configuration including www. after http://
Now I took the www. out and the site is being crawled.

Thank you for your help.