README.sslcerts 10.1 KB
Newer Older
1
        Lynx SSL support for certificates - README.sslcerts file
2 3 4 5 6

BACKGROUND:

The original README.ssl document for lynx stated:

7 8
        Note that the server... may not have a valid certificate. Lynx will not
        complain, as it does not yet support certificates...
9

10 11 12 13 14
Such lack of support is no longer the case.  Lynx now features excellent
certificate management through the openssl project.  There is almost no
online documentation available regarding how to use openssl's certificate
management with other programs, so this will accompany lynx and hopefully
encourage good practical security for unix clients.
15 16

Lynx relies on openssl to not only encrypt connections over https, but also to
17 18
determine whether it should even accept a certificate and establish a secure
connection with a remote host. Because of this reliance upon openssl by lynx,
19
most of this tutorial deals with how to use openssl to "install" both
20 21
vendor-provided CA cert bundles as well as self-signed certs from trusted sources
and, most importantly, how to get them recognized by lynx.
22

23 24 25
While lynx on many systems will transparently accept valid certificates, not
all systems enjoy such functionality. Further, as noted above, older versions
of lynx do not perform any validity checks on a certificate.
26

27 28
There is also the common case of wanting to trust, use and install a
self-signed certificate from a known server source and have it be trusted by
29 30 31
client programs.

Briefly, the procedure will involve confirming the default system location for
32
certificates, setting values for SSL_CERT_DIR and SSL_CERT_FILE in
33
the environment, and converting and hashing the certificates using openssl
34 35 36 37 38 39 40 41 42 43 44
utilities to enable recognition.

THE CURRENT SITUATION:

Prior to lynx2.8.5dev9, lynx did not check at all for certificate validity.

Since lynx2.8.5.dev9, lynx has reported this openssl error:

SSL error:unable to get local issuer certificate-Continue? (y)

whenever an https connection was initiated and the certificate could not be
45
found, for whatever reason, by openssl, and therefore lynx.
46

47 48
This checking for a certificate is an enhancement to security, but rather
tediously generates errors at each https browser request.
49

50
The ability to turn off reporting of this error to the user was added to
51 52 53
lynx2.8.5dev16 as the FORCE_SSL_PROMPT setting in lynx.cfg as noted in the
CHANGELOG:

54 55
        This lets the user decide whether to ignore prompting for questionable
        aspects of an SSL connection.
56

57 58 59 60
While this is a convenient setting to employ when using lynx to script
https -dumps, it by definition ignores the issue of certificate validity
altogether. Those concerned with proper certificate management and
the maintenance of a store of updated CA certificates will be uncomfortable
61 62
with this relaxed security setting.

63 64 65
The ability to accept a 'wildcard' certificate, where the first character
is a '*' was added to lynx2.8.6dev18.

66 67
PRELIMINARY PROCEDURES:

68 69 70 71
It is assumed that openssl has been installed correctly, that the default
cert directory is /usr/local/ssl/certs, (it's often /etc/ssl/certs, but we
need a point of departure for the discussion) and that lynx has been compiled
--with-ssl.
72

73 74 75
The default location for certs on your system may be different, or there may not
be one. You will have to substitute that location for /usr/local/ssl/certs in
the following instructions, and/or set environment variables.
76

77 78
To determine the default location for certs on your system you may run the
following command:
79 80 81 82

strings libcrypto.a  | grep -in cert | less

Look in this output for SSL_CERT_DIR and SSL_CERT_FILE, and the lines just
83 84
above them. This is your default location, respectively, for certificates,
and the CA cert bundle, cert.pem. You will need to know where libcrypto.a is
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104
found of course.

Example output:

<snip>
7490:/etc/ssl/certs
7491:/etc/ssl/cert.pem
7492:SSL_CERT_DIR
7493:SSL_CERT_FILE
<snip>

Other possible example output:

<snip>
31555:/usr/local/ssl/certs
31556:/usr/local/ssl/cert.pem
31557:SSL_CERT_DIR
31558:SSL_CERT_FILE
<snip>

105 106
Note that when OpenSSL is installed, the c_rehash utility is installed in a
bin directory (default /usr/local/ssl/bin).  You will need to know where it
107 108
is on your system. The command:

109
whereis c_rehash
110 111 112

will probably give useful results.

113 114 115
Note also that there is no CA cert bundle distributed with OpenSSL. The
OpenSSL team specifically decided NOT to do that. Getting a set of trusted
certificates is left up to the installer.
116

117 118 119 120 121 122 123 124 125 126 127
It is no longer a fairly trivial procedure to pull the bundle of trusted root certs out
of a recent version of Internet Explorer. Multiple certificates are no longer 
exportable as a DER formatted file; extraction of a single certificate is the only
export for DER, and DER is what converts to PEM.

Users with access to Apple OS X can export all certificates from Keychain Access System Roots as
a .pem file. Place this in SSL_CERT_DIR and hash it and you're done.

The MirOS BSD project also provides them. The procedure to convert and install them 
is detailed later in this document, and if you simply need to have commercially provided 
certificates trusted by lynx, you can skip down a few lines to the INSTALLING OR UPDATING
128 129
THE CA BUNDLE section.

130 131 132 133 134
Extracted Mozilla cert bundles are available for download from the curl project,
http://curl.haxx.se/docs/caextract.html along with a script to extract from Mozilla
source.


135 136
INSTALLING A SELF-SIGNED CERTIFICATE:

137 138 139
When you would like to trust a self-signed (non-commercial) certificate you will
need to get hold of the actual file. If it's a cert local to your network you
can ask the sysadmin to make it available for download as a link on a webpage.
140

141 142
If such file is not human-readable it's probably DER formatted and will need to
be converted to PEM format to allow openssl to use it.
143 144 145 146 147 148 149 150

To convert DER formatted certificates into something openssl can deal with:

Save the cert as site_name.crt in a directory. In that directory, type:

openssl x509 -inform DER -in site_name.crt -outform PEM -out site_name.pem

You can now copy this individual cert into the directory for that, usually
151 152 153
/usr/local/ssl/certs.  The alternative is to concatenate the individual certs
to the cert.pem bundle in /usr/local/ssl. (Please see INSTALLING OR UPDATING
THE CA BUNDLE below).
154

155 156 157 158
The cert file will now be in an acceptable format to openssl, PEM encoded.
However, openssl, and by extension lynx, will not know about it until that
cert is symbolically linked to a file named after the hash value of that cert,
in the default directory /usr/local/ssl/certs.
159 160 161 162 163 164 165 166

So the next thing to do is to hash the cert using c_rehash.

INSTALLING OR UPDATING THE CA BUNDLE:

Now would be a good time to check to see if you have the bundle of CA certs
/usr/local/ssl/cert.pem, or to update them.

167
CA bundles are available in various places, such as the MirOS BSD distribution,
168 169 170 171
for those who want to take that route, or you can extract the current bundle
from a current version of Internet Explorer (export them all from IE and
transfer it onto your system).

172
From MirOS, a cert bundle is available at
173

174
http://caunter.ca/ssl.certs.shar
175

176
It includes the cacert.org certificate. Download the latest revision; read the
177 178 179
file to see how to get the certs out.

No hashing is necessary with this set of certs; it is already done; ignore
180 181
the c_rehash usage below for this bundle. Simply run `sh ssl.certs.shar` 
in SSL_CERT_DIR.
182

183
From IE 5.x certs extract as a PKCS7 file and need to be converted with something
184 185
like:

186
openssl pkcs7 -inform DER -in bundle.crt -outform PEM -out cert.pem \
187 188
-print_certs -text

189 190 191 192 193
The resulting cert.pem file should be copied to the default directory for
bundles (usually /usr/local/ssl) and renamed to "cert.pem", assuming that is
the SSL_CERT_FILE.

Individual certs can also process if added and hashed in /usr/local/ssl/certs.
194

195 196
We now have all of the individual certs we wish to trust in our certs
directory, and the most recent bundle of CA certs as well.
197

198 199
Confirm that you have the script c_rehash (See PRELIMINARY PROCEDURES; if it is
not found, a copy is usually located in the tools directory of the openssl
200 201 202 203 204 205 206
source tree. If you use this copy, it needs the execute bit set or it will not
run).

Run:

./c_rehash

207 208 209
The c_rehash utility is a perl script that runs openssl commands which creates
the files named after the hash values of the certs in the default directory
for certs.
210 211 212 213 214 215 216 217 218 219 220 221 222

Its output looks like this:

Doing /usr/local/ssl/certs
vsignss.pem => f73e89fd.0
vsign3.pem => 7651b327.0
...more output
<snip>

All pem encoded certs in /usr/local/ssl/certs will now be recognized.

SETTING AND EXPORTING ENVIRONMENT VARIABLES:

223
If lynx is still not recognizing certs, environment variables need
224 225
to be set; if on a sh type shell, the variables also need to be exported.

226
The environment variables SSL_CERT_DIR and SSL_CERT_FILE need to be set
227
if a non-default location is used for certificates, or if certs just can't be
228 229 230
found by lynx. They may be set as follows in /etc/profile, or a shell
initialization .profile or .*shrc, if we run a non csh type shell, according
to the results of the search for the default location for certs procedure
231 232 233 234 235 236 237 238 239 240
(See PRELIMINARY PROCEDURES):

SSL_CERT_DIR="/usr/local/ssl/certs"
SSL_CERT_FILE="/usr/local/ssl/cert.pem"
export SSL_CERT_DIR SSL_CERT_FILE

On csh type shells, you can use:
setenv SSL_CERT_DIR "/usr/local/ssl/certs"
setenv SSL_CERT_FILE "/usr/local/ssl/cert.pem"

241 242
Note that the environment variable SSL_CERT_FILE applies to the cert-bundle
if used outside of the default location (/usr/local/ssl/cert.pem) compiled
243
into OpenSSL. There are issues with SSL_CERT_FILE in 0.9.6x versions of openssl.
244

245 246 247 248 249
The configuration file lynx.cfg allows a system SSL_CERT_FILE variable to be set
which can simplify matters. 

SSL_CERT_FILE:/etc/ssl/certs/ca-certificates.crt

250 251 252 253 254 255 256 257 258
Make sure you have FORCE_SSL_PROMPT set to PROMPT in lynx.cfg like so:

FORCE_SSL_PROMPT:PROMPT

You will now connect without error to https servers with trusted certs, but
will still get this error for untrusted certs:

SSL error:self signed certificate-Continue? (y)

259
A quick check confirms that these procedures have the same effect with ssl
260 261
errors in the pine program.

262
2003 updated 2009
263
Stefan Caunter <stefan.caunter@mohawkcollege.ca>
264 265
Mohawk College Department of Computer Science
Hamilton Ontario Canada