XML Sitemaps Generator

Author Topic: https nginx + Apache 0 pages crawled  (Read 1476 times)

capricorn

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
https nginx + Apache 0 pages crawled
« on: April 07, 2017, 03:08:45 PM »
Hello,

I'm running Generator v.7.2. My server setup is nginx on the front end as a back proxy for Apache.  Nginx is serving static content only and redirecting all http requests to https (443).

In my configuration the starting URL is https://mydomain.com. However after starting crawling the process immediately stops saying 0 pages were crawled and no xml sitemap is created. I never had a problem earlier with the http site and Apache only on my dedicated server.

I have read some related topics but couldn't find out the reason. Could you please advise where to look?

Best Regards

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10824
Re: https nginx + Apache 0 pages crawled
« Reply #1 on: April 08, 2017, 06:48:11 AM »
Hello,

please let me know your generator URL/login in private message to check this.
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

capricorn

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: https nginx + Apache 0 pages crawled
« Reply #2 on: April 08, 2017, 07:06:29 AM »
Done. If you need any other information pls let me know.

Thanks

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10824
Re: https nginx + Apache 0 pages crawled
« Reply #3 on: April 08, 2017, 03:50:09 PM »
Hello,

all pages are blocked with robots.txt
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

capricorn

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: https nginx + Apache 0 pages crawled
« Reply #4 on: April 08, 2017, 11:03:36 PM »
Thanks,

XML sitemap is created now after fixing robots.txt.

But why links to HTML and Text sitemaps on a "View Sitemap" tab are starting from http and not from https? It is not a problem since they are redirected to https by the server, however I would think that they should start from https as the link to XML Sitemap. Any suggestions?

BR

XML-Sitemaps Support

  • Administrator
  • Hero Member
  • *****
  • Posts: 10824
Re: https nginx + Apache 0 pages crawled
« Reply #5 on: April 09, 2017, 05:08:01 AM »
Hello,

you need to access generator using https://domain.com/generator/ as well (not via http://).
Oleg Ignatiuk
www.xml-sitemaps.com
Send me a Private Message

For maximum exposure and traffic for your web site check out our additional SEO Services.

capricorn

  • Registered Customer
  • Jr. Member
  • *
  • Posts: 16
Re: https nginx + Apache 0 pages crawled
« Reply #6 on: April 09, 2017, 06:07:47 AM »
I access generator by [external links are visible to admins only]

After crawling is finished the links to sitemaps are:

HTML SiteMap
[external links are visible to admins only] shown as [external links are visible to admins only]

Text SiteMap
[external links are visible to admins only] shown as [external links are visible to admins only]

1. XML SiteMap File
[external links are visible to admins only] shown as [external links are visible to admins only]

XML sitemap link and its text is ok. Two others need to look the same in my opinion - to avoid possible confusion.
« Last Edit: April 09, 2017, 06:14:49 AM by capricorn »

 

SMF 2.0.14 | SMF © 2017, Simple Machines
XHTML RSS WAP2