• Welcome to Sitemap Generator Forum.
 

How to configure the generator to crawl secure https pages?

Started by admin317, October 23, 2009, 01:40:38 PM

Previous topic - Next topic

admin317

I have just downloaded the latest version of  Unlimited PHP Sitemap Generator but cannot work out how to tell it to crawl the secure pages on my e-commerce website.
I have looked all over the documentation and the forums but seem to have missed the instructions.
Should I simply add 's' to the 'http' prefix in my site's URL?
Or is it something else?




admin317

Thank you, Admin, for your quick reply.
I have tried that method, however the Sitemap Generator only crawls one single page and then gives me the following message -
Request date:
23 October 2009, 16:02
Processing time:
0:00:01s
Pages indexed:
1
Sitemap files:
1
Pages size:
0.00Mb

I must have configured something wrongly, but I cannot work out what it is.
BTW, OpenSSL *is* enabled on my server


admin317

Just some additional information that might help solve the problem:

1) ERROR MESSAGE
An error occured: There was an error while retrieving the URL specified: [ External links are visible to forum administrators only ]
HTTP headers follow:
Sat, 24 Oct 2009 09:47:16 GMT
Apache/2.2.3 (CentOS)
[ External links are visible to forum administrators only ]

2) ERROR LOG
My server's error log generated the following error lists the last couple of times I tried to get Sitemap Generator to crawl my site:

[Sat Oct 24 01:51:52 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/webmasters, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 01:51:52 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/SiteExplorerService, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 01:51:52 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/ping, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 01:51:52 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/ping, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 01:51:52 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/ping.aspx, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 01:51:52 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/rpc, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 02:46:45 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/robots.txt, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 02:46:46 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/webmasters, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 02:46:46 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/SiteExplorerService, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 02:46:46 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/ping, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 02:46:46 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/ping, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 02:46:46 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/ping.aspx, referer: [ External links are visible to forum administrators only ]


[Sat Oct 24 02:46:46 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/rpc, referer: [ External links are visible to forum administrators only ]


XML-Sitemaps Support

Hello,

[ External links are visible to logged in users only ] shows hosting control panel login page for me, no site content there.

admin317

Thanks for your reply, Oleg!
My apologies, I overlooked that fact that I'm using a *shared* SSL certificate.
The URL should be  [ External links are visible to forum administrators only ]
But this URL still gives me the same problem - only one page gets crawled:

Request date:
25 October 2009, 22:02
Processing time:
0:00:02s
Pages indexed:
1
Sitemap files:
1
Pages size:
0.02Mb



admin317

Server Error Log Messages when I try to run the generator:

[Sun Oct 25 22:02:08 2009] [error] [client 220.237.137.184] File does not exist: /home/virtual/site199/fst/var/www/html/favicon.ico

[Sun Oct 25 22:02:11 2009] [error] [client 220.237.137.184] File does not exist: /home/virtual/site199/fst/var/www/html/favicon.ico

[Sun Oct 25 22:02:12 2009] [error] [client 220.237.137.184] File does not exist: /home/virtual/site199/fst/var/www/html/favicon.ico

[Sun Oct 25 22:02:29 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/webmasters, referer: [ External links are visible to forum administrators only ]

[Sun Oct 25 22:02:29 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/SiteExplorerService, referer: [ External links are visible to forum administrators only ]

[Sun Oct 25 22:02:29 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/ping, referer: [ External links are visible to forum administrators only ]

[Sun Oct 25 22:02:29 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/ping, referer: [ External links are visible to forum administrators only ]

[Sun Oct 25 22:02:29 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/ping.aspx, referer: [ External links are visible to forum administrators only ]

[Sun Oct 25 22:02:29 2009] [error] [client 209.151.28.209] File does not exist: /home/virtual/site199/fst/var/www/html/rpc, referer: [ External links are visible to forum administrators only ]
???


XML-Sitemaps Support

Hello,

it looks like all shop links from inner pages are pointing to your main domain anyway, so they are autoamtically excluded from sitemap (only pages from the same domain as Starting URL can be included in sitemap).

Entries from the error log do not seem to be related to sitemap generator.

admin317

Okay, thanks Oleg. I shall try to figure out how to fix this...

Best wishes.