Starting with OpenSSL 1.1.0, s_client will not offer TLS compression methods, even if OpenSSL is compiled with zlib support, unless the `-comp` flag is included in the command line.
This PR changes `run_crime()` to use `tls_sockets()` rather than failing if `$OPENSSL` lacks zlib support, unless `$SSL_NATIVE` is `true`.
At the moment, the ClientHello created by `socksend_tls_clienthello()` only specifies the NULL compression method. So, this PR adds a new parameter to `socksend_tls_clienthello()` and `tls_sockets()` to allow to caller to request that additional compression methods (DEFLATE and LZS) be specified in the ClientHello.
This PR makes another change to `run_crime()`. At the moment, if `$OPENSSL s_client` fails to connect to the server, `run_crime()` will report that the server is not vulnerable, since the output from `$OPENSSL s_client` includes the line "Compression: NONE" (see below). This PR changes that by checking whether the connection was successful, and reporting a "test failed (couldn't connect)" warning if it wasn't successful, rather than reporting "not vulnerable (OK)".
```
CONNECTED(00000003)
140338777061024:error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version:s23_clnt.c:769:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 389 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1483645971
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
```
This PR fixes a few bugs in `sslv2_sockets()`. The main issue is that a server may not send the entire ServerHello in a single packet. If it doesn't and the full response is being parsed (i.e., certificate and list of ciphers), then `parse_sslv2_serverhello()` will encounter errors, since it assumes that it has the entire ServerHello. This PR compares the length of the response to the length of the ServerHello as specified in the first two bytes of the response and requests more data from the server if the response appears incomplete.
This PR also modifies `parse_sslv2_serverhello()` to check for more errors. It compares the length of the response it has been provided to the specified length (`$v2_hello_length`) and returns an error if the response is shorter than `$v2_hello_length` and the full response is supposed to be parsed. It will also check whether there was an error in converting the certificate from DER to PEM format and will return an error if there was (and it will suppress the error message).
If `determine_tls_extensions()` does not create a temporary file (`$TEMPDIR/$NODEIP.determine_tls_extensions.txt`) then `run_server_defaults()` will display error messages when an attempt is made to copy this file or to search (grep) it. This may happen if `$OPTIMAL_PROTO` is `-ssl2` or if `determine_tls_extensions()` uses sockets and `parse_tls_serverhello()` encountered an error and did not create a temporary file (`$TEMPDIR/$NODEIP.parse_tls_serverhello.txt`). This PR fixes this by only trying to copy and search `$TEMPDIR/$NODEIP.determine_tls_extensions.txt` is `$OPTIMAL_PROTO` is not `-ssl2` and `determine_tls_extensions()` was successful (return value 0).
This PR changes `run_http2()` so that it uses `tls_sockets()` rather than failing, if `$OPENSSL` does not support the `-alpn` option. If `$OPENSSL` supports the `-alpn` option (or if `$SSL_NATIVE` is true), then this PR has no effect.
In a few places testssl.sh tries to determine $OPENSSL s_client's capabilities by calling `$OPENSSL s_client` without specifying a host to which to connect. For example:
```
$OPENSSL s_client -no_ssl2 2>&1
```
This idea is that `$OPENSSL s_client` should reveal something about its capabilities without actually trying to connect to a host.
This works in most cases. However, the manual pages for s_client states:
```
-connect host:port
This specifies the host and optional port to connect to. If not specified then an attempt is made to connect to the local host on port 4433.
```
So, the above call is actually trying to connect to the local host on port 4433. If the local host is running `$OPENSSL s_server`, then `$OPENSSL s_server` will by default be listening on port 4433, and the connection attempt will most likely succeed. Since the `OPENSSL s_client` command does not include a `< /dev/null`, the `OPENSSL s_client` will just hang waiting for additional input.
Adding `-connect x` to the `$OPENSSL s_client` prevents $OPENSSL from trying to connect to a host, but seems to still provide the necessary information about OpenSSL's capabilities.
This PR adds ",exp" to the bits column when `run_rc4()` is run in the "--wide" mode and the cipher is an export cipher. This makes the wide mode of `run_rc4()` align with other functions, such as `run_allciphers()`.
When `test_just_one()` uses `neat_list()` with a cipher that is not available and that uses DH for key exchange, the columns do not line up correctly. `test_just_one()` adds "TBD" in gray to "DH", and while `neat_list()` tries to adjust for the presence of color codes, it doesn't seem to correctly handle the gray color code here.
Rather than try to fix this in `neat_list()`, I propose to just remove the "TBD". Adding it is inconsistent with other functions (like `run_allciphers()`), and it seems inappropriate, since there is nothing "to be determined," as the cipher suite isn't supported by the server.
If adding "TBD" were appropriate anywhere, it would seem to be in cases in which the server does support the cipher, but the number of bits in the ephemeral key couldn't be determined because the version of OpenSSL being used can't show DH/ECDH bits. (Not that I'm proposing this. I think the one-line warning, "(Your $OPENSSL cannot show DH/ECDH bits)", is enough.
Here is an example of `test_just_one()` with some ciphers not supported by the server that use DH key exchange:
```
Testing single cipher with word pattern "CAMELLIA" (ignore case)
Hexcode Cipher Suite Name (OpenSSL) KeyExch. Encryption Bits Cipher Suite Name (RFC)
---------------------------------------------------------------------------------------------------------------------------
xc077 ECDHE-RSA-CAMELLIA256-SHA384 ECDH TBD Camellia 256 TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384 not a/v
xc073 ECDHE-ECDSA-CAMELLIA256-SHA384 ECDH TBD Camellia 256 TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384 not a/v
xc4 DHE-RSA-CAMELLIA256-SHA256 DH TBD Camellia 256 TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA256 not a/v
xc3 DHE-DSS-CAMELLIA256-SHA256 DH TBD Camellia 256 TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA256 not a/v
xc2 DH-RSA-CAMELLIA256-SHA256 DH/RSA Camellia 256 TLS_DH_RSA_WITH_CAMELLIA_256_CBC_SHA256 not a/v
xc1 DH-DSS-CAMELLIA256-SHA256 DH/DSS Camellia 256 TLS_DH_DSS_WITH_CAMELLIA_256_CBC_SHA256 not a/v
x88 DHE-RSA-CAMELLIA256-SHA DH 2048 Camellia 256 TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA available
x87 DHE-DSS-CAMELLIA256-SHA DH TBD Camellia 256 TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA not a/v
x86 DH-RSA-CAMELLIA256-SHA DH/RSA Camellia 256 TLS_DH_RSA_WITH_CAMELLIA_256_CBC_SHA not a/v
x85 DH-DSS-CAMELLIA256-SHA DH/DSS Camellia 256 TLS_DH_DSS_WITH_CAMELLIA_256_CBC_SHA not a/v
xc5 ADH-CAMELLIA256-SHA256 DH TBD Camellia 256 TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA256 not a/v
x89 ADH-CAMELLIA256-SHA DH TBD Camellia 256 TLS_DH_anon_WITH_CAMELLIA_256_CBC_SHA not a/v
xc079 ECDH-RSA-CAMELLIA256-SHA384 ECDH/RSA Camellia 256 TLS_ECDH_RSA_WITH_CAMELLIA_256_CBC_SHA384 not a/v
xc075 ECDH-ECDSA-CAMELLIA256-SHA384 ECDH/ECDSA Camellia 256 TLS_ECDH_ECDSA_WITH_CAMELLIA_256_CBC_SHA384 not a/v
xc0 CAMELLIA256-SHA256 RSA Camellia 256 TLS_RSA_WITH_CAMELLIA_256_CBC_SHA256 not a/v
x84 CAMELLIA256-SHA RSA Camellia 256 TLS_RSA_WITH_CAMELLIA_256_CBC_SHA not a/v
xc076 ECDHE-RSA-CAMELLIA128-SHA256 ECDH TBD Camellia 128 TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256 not a/v
xc072 ECDHE-ECDSA-CAMELLIA128-SHA256 ECDH TBD Camellia 128 TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256 not a/v
xbe DHE-RSA-CAMELLIA128-SHA256 DH TBD Camellia 128 TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA256 not a/v
xbd DHE-DSS-CAMELLIA128-SHA256 DH TBD Camellia 128 TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA256 not a/v
xbc DH-RSA-CAMELLIA128-SHA256 DH/RSA Camellia 128 TLS_DH_RSA_WITH_CAMELLIA_128_CBC_SHA256 not a/v
xbb DH-DSS-CAMELLIA128-SHA256 DH/DSS Camellia 128 TLS_DH_DSS_WITH_CAMELLIA_128_CBC_SHA256 not a/v
x45 DHE-RSA-CAMELLIA128-SHA DH 2048 Camellia 128 TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA available
x44 DHE-DSS-CAMELLIA128-SHA DH TBD Camellia 128 TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA not a/v
x43 DH-RSA-CAMELLIA128-SHA DH/RSA Camellia 128 TLS_DH_RSA_WITH_CAMELLIA_128_CBC_SHA not a/v
x42 DH-DSS-CAMELLIA128-SHA DH/DSS Camellia 128 TLS_DH_DSS_WITH_CAMELLIA_128_CBC_SHA not a/v
xbf ADH-CAMELLIA128-SHA256 DH TBD Camellia 128 TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA256 not a/v
x46 ADH-CAMELLIA128-SHA DH TBD Camellia 128 TLS_DH_anon_WITH_CAMELLIA_128_CBC_SHA not a/v
xc078 ECDH-RSA-CAMELLIA128-SHA256 ECDH/RSA Camellia 128 TLS_ECDH_RSA_WITH_CAMELLIA_128_CBC_SHA256 not a/v
xc074 ECDH-ECDSA-CAMELLIA128-SHA256 ECDH/ECDSA Camellia 128 TLS_ECDH_ECDSA_WITH_CAMELLIA_128_CBC_SHA256 not a/v
xba CAMELLIA128-SHA256 RSA Camellia 128 TLS_RSA_WITH_CAMELLIA_128_CBC_SHA256 not a/v
x41 CAMELLIA128-SHA RSA Camellia 128 TLS_RSA_WITH_CAMELLIA_128_CBC_SHA not a/v
```
This PR implements `run_pfs()` in a manner similar to `run_allciphers()`. It uses OpenSSL followed by `tls_sockets()` to test for both supported PFS cipher suites as well as elliptic curves offered.
I made an attempt at addressing #548 by using different colors to print the different curve names, depending on strength. The colors chosen are exactly the same as those that would be chosen by `read_dhbits_from_file()`:
```
# bits <= 163: pr_svrty_medium
163 < # bits <= 193: pr_svrty_minor
193 < # bits <= 224: out
# bits > 224: pr_done_good
```
I also added code for #464 to create a list of the DH groups from RFC 7919 that a server supports. However, since no servers seem to support this at the moment (except with TLS 1.3), I marked this code to only run if the $EXPERIMENTAL flag is set.
For several elliptic curves the number of bits, as indicated by OpenSSL, is slightly different than the name implies. For example, for sect239k1 OpenSSL outputs: `Server Temp Key: ECDH, sect239k1, 238 bits`.
This PR aligns the output created by `parse_tls_serverhello()` with OpenSSL.
When the cipher-mapping.txt file is read, the contents of the "Mac=..." column is placed in `TLS_CIPHER_EXPORT` rather than the contents of the "export" column. This PR fixes that.
This PR address a problem in `run_drown()` when the server does not support SSLv2, but does support multiple certificates or doesn't have an RSA certificate.
One example of the problem can be seen with www.facebook.com. If `run_server_preferences()` is run before `run_drown()`, then the results of `run_drown()` are:
```
DROWN (2016-0800, CVE-2016-0703) not vulnerable on this port (OK)
make sure you don't use this certificate elsewhere with SSLv2 enabled services
https://censys.io/ipv4?q=A626B154CC65634181250B810B1BD4C89EC277CEA08D785EEBE7E768BDA7BB00 SHA256 A3F474FB17509AE6C5B6BA5E46B79E0DE6AF1BF1EEAA040A6114676E714C9965 could help you to find out
```
If only `run_drown()` is performed, then the result is:
```
DROWN (2016-0800, CVE-2016-0703) not vulnerable on this port (OK)
make sure you don't use this certificate elsewhere with SSLv2 enabled services
https://censys.io/ipv4?q=A626B154CC65634181250B810B1BD4C89EC277CEA08D785EEBE7E768BDA7BB00 could help you to find out
```
However, A626B154CC65634181250B810B1BD4C89EC277CEA08D785EEBE7E768BDA7BB00 is the fingerprint of Facebook's ECDSA certificate, not its RSA certificate.
In addition, as noted in the "FIXME," `run_drown()` will display the warning "make sure you don't use this certificate elsewhere with SSLv2 enabled services" even if the server doesn't have an RSA certificate, even though SSLv2 can only use RSA certificates.
This PR fixes this issue by only showing the warning if the server has an RSA certificate and by ensuring that the `$cert_fingerprint_sha2` used to construct the "https://censys.io/ipv4?q=..." URL only contains a single SHA256 fingerprint and that it is the fingerprint of the server's RSA certificate.
This PR implements `run_rc4()` in a similar manner to `run_allciphers()` and `run_cipher_per_proto()` (in PR #541). The change doesn't seem to have much of an impact on speed, but when sockets are used it can detect ciphers that aren't locally supported by OpenSSL.
This PR fixes two minor bugs in run_allciphers():
* If `$SSL_NATIVE` or `$FAST` is `true`, then the cipher mapping file will not be used (unless `$OPENSSL ciphers` does not support the `-V` option), so there is no "fallback" to openssl, even if `[[ $TLS_NR_CIPHERS == 0 ]]`.
* If `$using_sockets` is `false` and `$SHOW_EACH_C` is `true`, then `ossl_supported` should be checked to see if the cipher was tested, not `TLS_CIPHER_OSSL_SUPPORTED`.
This function reorganizes `run_server_defaults()` based on the suggestion in #515.
The current `determine_tls_extensions()` is renamed to `get_server_certificate()`, and two changes are made to it:
*it no longer includes an extra call to `$OPENSSL s_client` to check for the ALPN extension; and
* rather than setting `$TLS_EXTENSIONS` to be the extensions found during this call to the function, it adds any newly found extensions to those already in `$TLS_EXTENSIONS`.
The PR then adds a new function, `determine_tls_extensions()`, which borrows some logic from the old `determine_tls_extensions()`, but this new `determine_tls_extensions()` only looks for additional TLS extensions, including ALPN.
`run_server_defaults()` makes multiple calls to `get_server_certificate()` (as it previously did to `determine_tls_extensions()`) in order to collect all of the server's certificates, and then it makes one call to `determine_tls_extensions()`, which checks for support for extensions that were not checked for by `get_server_certificate()` (e.g., ALPN, extended master secret, signed certificate timestamps).
The new `determine_tls_extensions()` will check for most of the extensions that are checked for by
`run_server_defaults()`, including the heartbeat extension, so the call to `determine_tls_extensions()` from `run_heartbleed()` will still work.
> The dh_bits are still not shown, maybe because of #531.
This PR fixes the issue of dh_bits not being shown if the cipher-mapping.txt file is missing. The problem is that the code in `parse_tls_serverhello()` that parses the ServerKeyExchange message assumes that `$rfc_cipher_suite` has the RFC version of the name the cipher suite. However, if the cipher-mapping.txt file is missing, `$rfc_cipher_suite` will have the OpenSSL name of the cipher suite. This PR changes the code to recognize either the RFC or OpenSSL names for ciphers with ephemeral DH or ECDH keys.
When `tls_sockets()` is used with the "full" option and the chosen cipher suite involves an ephemeral finite-field DH key (DH), this PR extracts the public key from the ServerKeyExchange message and adds it to `$TMPFILE`. In addition (and the primary reason for this PR), it compares the ephemeral public key's parameters to those specified in RFC 7919, and indicates whether one the groups from that RFC was used. This will allow `run_pfs()` to be extended to indicate which, if any, RFC 7919 DH groups a server supports.
This PR adds parsing of the CertificateStatus message to `parse_tls_serverhello()`. If the caller requests that the "full" response be parsed, then the CertificateStatus message is parsed, and the OCSP response is added to $TMPFILE, in a manner similar to the output of `$OPENSSL s_client` when the `-status` option is used.
The string "CamelliaGCM" is too long for the "Encryption" column printed by `neat_list()`. So, either "CamelliaGCM" needs to be shortened to "Camellia" (as this PR does), or the "Encryption" column needs to be made wider.
Client simulations can still use sockets even if the cipher mapping file is missing. If the cipher file is present, then `parse_tls_serverhello()` write the RFC name for the cipher and then `run_client_simulation()` converts that to the OpenSSL name (so that the output is the same as if OpenSSL were used). This PR changes `parse_tls_serverhello()` so that it writes the OpenSSL name for the cipher if the mapping file is missing, which `run_client_simulation()` can then just display.
This PR also unsets `ADD_RFC_STR` if the mapping file is missing, so that `neat_list()` won't try to display the RFC names for the ciphers.
This PR speeds up the implementation of `run_allciphers()` by introducing a number of changes:
* Rather than check for implemented ciphers in a hierarchical manner (as introduced in #326), this PR follows the approach of `cipher_pref_check()`. Testing a block of ciphers, marking the selected cipher as implemented, and then testing same block of ciphers, minus those that have previously been selected, until a test fails. Thus the number of calls to `$OPENSSL s_client` is just one more than the number of ciphers implemented. (Since some servers cannot handle ClientHellos with more than 128 messages, the tests are performed on blocks of 128 or few ciphers. So, if OpenSSL supports 197 ciphers, the number of calls to `$OPENSSL s_client` is 2 plus the number of ciphers supported by the server.
* If $using_sockets is true, then OpenSSL is used first to find all supported ciphers that OpenSSL supports (since OpenSSL is faster than `tls_sockets()`), and then `tls_sockets()` is only used to test those cipher suites that were not found to be supported by OpenSSL.
* The `prepare_debug()` function, which reads in `$CIPHERS_BY_STRENGTH_FILE` determines which ciphers are supported by the version of OpenSSL being used. If a version of OpenSSL older than 1.0 is being used, then this is used to determine which ciphers to test using OpenSSL rather than using `$OPENSSL ciphers -V`.
Following the approach of `cipher_pref_check()` reduces the number of queries to the server. Using OpenSSL before `tls_sockets()` reduces the number of calls to `tls_sockets()` to 3 plus the number of ciphers supported by the server that are not supported by OpenSSL, so the cost penalty over just using OpenSSL is fairly small.
The `tls_sockets()` and `sslv2_sockets()` use `get_pub_key_size()` to extract the size of the server's public key if the full response is being processed, and `get_pub_key_size()` uses `$OPENSSL pkey` to extract the server's public key from the certificate. However, OpenSSL 0.9.8 does not support the "pkey" command. This PR changes `get_pub_key_size()` to suppress the error message displayed by OpenSSL when the "pkey" command is not supported.
This PR adds parsing of the Certificate message to `parse_tls_serverhello()`. If the caller requests that the "full" response be parsed, then the Certificate message is parsed, the server's certificate is placed in $HOSTCERT and the intermediate certificates are placed in $TEMPDIR/intermediatecerts.pem. The certificates are also added to $TMPFILE, in a manner similar to the output of `$OPENSSL s_client` when the `-showcerts` option is used.
This PR uses `tls_sockets()` to determine whether a server supports certain extensions that may not be supported by `$OPENSSL`. At the moment it checks for max_fragment_length, client_certificate_url, truncated_hmac, ALPN, signed_certificate_timestamp, encrypt_then_mac, and extended_master_secret.
In https://github.com/dcooper16/testssl.sh/blob/extended_tls_sockets/testssl.sh, `run_server_defaults()` is re-written to use `tls_sockets()` instead of `$OPENSSL`, with just one call to `$OPENSSL s_client` to get the session ticket, which reduces the dependence on `$OPENSSL`, but this PR limits the number of calls to `tls_sockets()`, which is still slow.
Note: I included ALPN in the `tls_sockets()` ClientHello since a single call to `tls_sockets()` cannot test for both NPN and ALPN, and since support for NPN was added to OpenSSL before support for ALPN was added, I figured it was more likely that `determine_tls_extensions()` had already determined whether the server supported NPN.
This PR fixes the same issues as were fixed in PR #513, but also makes two changes to `parse_tls_serverhello()`:
* It changes the number of bits for curve X25519 from 256 to 253 to match OpenSSL.
* It removes the "ECDH, " from the "Server Temp Key: " line in order to match OpenSSL's output.
This PR fixes two issues related to curve X25519.
First, while OpenSSL 1.1.0 supports curve X25519, it is not included in the output of `$OPENSSL ecparam -list_curves`. I tried several versions of OpenSSL (and one version of LibreSSL), and every version output either "Error with command" or "unknown option" in response to `$OPENSSL s_client -curves $curve` if it either did not support the `-curves` option or did not support `$curve`. (When the `-curve` option was supported with `$curve`, a "connect" error was output.)
The second issue is that the "Server Temp Key" line in the output of `s_client` is different for curve X25519. For other elliptic curves, the output is
```
Server Temp Key: ECDH, P-256, 256 bits
```
For X25519 it is:
```
Server Temp Key: X25519, 253 bits
```
So, `read_dhbits_from_file()` needs to allow for `$what_dh` being "X25519" rather than "ECDH" and `run_pfs()` needs to allow for the possibility that the curve name will be the first field rather than the second.
The PR changes `run_allciphers()` to use `tls_sockets()` (and `sslv2_sockets()`)rather than `$OPENSSL` unless `$SSL_NATIVE` is set or `$STARTTLS` is non-empty. Using sockets allows `run_allciphers()` to test all ciphers, rather than just those supported by `$OPENSSL`.
Using sockets results in `run_allciphers()` running more slowly, partially since it is testing more ciphers, but mostly since `tls_sockets()` is currently slower than `$OPENSSL` (as noted in #413).
This PR makes similar changes to `run_client_simulation()` as were made to `tls_sockets()`, so that `run_client_simulation()` retrieves the entire server response, even if it is split across multiple packets, and it has `parse_tls_serverhello()` extract information about the server's ephemeral public key, if present.
The PR also changes `run_client_simulation()` to use information about the ephemeral public key. It includes the length of the public key in the output and, if it is a DH public key, checks that the size is within the acceptable range (`${minDhBits[i]} <= dh_bits <= ${maxDhBits[i]}`).
This PR adds initial parsing of the ServerKeyExchange message to `parse_tls_serverhello()`. For ephemeral DH keys, it extracts the length of the key. For ephemeral ECDH keys that are encoded using the named_curve option, it extracts the length of the key and the name of the curve.
This PR allows the caller to provide additional extensions to `tls_sockets()` to be included in the ClientHello. If the caller provides an extension that would have already been included in the ClientHello, then the caller's value for the extension is used rather than the default value.
This PR extended `parse_tls_serverhello()` in a few ways:
* If the "full" response is to be parsed, then additional checks are performed to verify that `$tls_hello_ascii` contains the entire response
* The extensions field is parsed and the list of extensions found is placed in `$TLS_EXTENSIONS` (if the "full" response is being parsed).
* Initial support for TLS 1.3 is added:
- Accounts for differences between TLS 1.2 ServerHello and TLS 1.3 ServerHello (as outlined in PR #499).
- Recognizes new alerts and handshake message types.
- Allows for server response to include message fragments of type "application data"
I forgot that `parse_tls_serverhello()` is also called by `client_simulation_sockets()`. Since PR #499 changed the input to `parse_tls_serverhello()`, the change needs to be made in `client_simulation_sockets()` as well.
Now that the mapping file is no longer used, `$ADD_RFC_STR` should not be unset just because the mapping file cannot be found.
In addition, since `show_rfc_style()` is now used in `parse_tls_serverhello()`, it cannot return an empty string just because the user set "--mapping no-rfc" on the command line. Instead, `neat_list()` should check the value of `$ADD_RFC_STR` and not call `show_rfc_style()` if it has been unset.
Finally, since `show_rfc_style()` no longer returns strings with extra spaces, there is no need to call `strip_spaces()`
In some cases the server's response to a ClientHello spans more than one packet. If the goal is just to determine whether the connection was successful and to extract a few pieces of information from the ServerHello message, then this is unlikely to be a problem. However, if there is a desire to extract the server's certificate chain (Certificate message) or to determine the type and size of the server's ephemeral public key (ServerKeyExchange message), then the entire response needs to be obtained, even if it spans multiple packets.
This PR adds a new function, `check_tls_serverhellodone()`, that checks whether the entire response has been received (e.g., whether the ServerHelloDone message has been received). If the response indicates that the response is incomplete, then `tls_sockets()` requests more data from the server until the response is complete or until the server doesn't provide any more data in response.
The PR only changes the behavior of `tls_sockets()` if the caller indicates that it wants to extract the ephemeral key or that it wants the entire response to be parsed. Otherwise, only the first packet returned by the server is sent to `parse_tls_serverhello()`. [The value of `$process_full` is not used at the moment, but will be in a subsequent PR that modifies `parse_tls_serverhello()`.]
This PR also changes `tls_sockets()` to send a close_notify to the server if the connection was successfully established.
PR #346 added a test for version tolerance to `run_protocols()`, but I think it may now be more appropriate to remove that test. Draft -16 of TLS 1.3, which was posted on September 22, changed the way that version negotiation is handled for TLS 1.3 and above. The current version tolerance test sends a ClientHello with the version field set to "03, 05", to represent a TLS 1.4 ClientHello. While this was consistent with RFC 5246 and with drafts of TLS 1.3 up to -15, draft -16 changed the version field to `legacy_version` and declared that its value should be "03, 03" for TLS 1.2 and above. (For TLS 1.3 and above a Supported Versions extension is included to inform the server which versions of TLS the client supports.) The change in draft -16 was made as a result of the problems with servers not handling version negotiation correctly.
Since the current draft suggests that a server should never be presented with a ClientHello with a version higher than "03, 03" (even for clients that support TLS versions higher than 1.2), it seems there is no reason to include the version tolerance test anymore.
For servers that do not support TLS 1.2, the additional checks that were added by PR #346 will already detect if the server cannot perform version negotiation correctly.
This PR adds the option for `parse_sslv2_serverhello()` to extract information from the ServerHello (server key size and cipher suites supported) and write the information to `$TMPFILE` as well as to write the server's certificate to `$HOSTCERT`.
The mapping file is now only used in `show_rfc_style()`. This PR changes `show_rfc_style()` to use the `$TLS_CIPHER_HEXCODE` and `$TLS_CIPHER_RFC_NAME` arrays.
Note that `get_install_dir()` still searches for the mapping-rfc.txt in order to determine `$INSTALL_DIR`. `$INSTALL_DIR` is only used to determine the location of the CA bundles in `determine_trust()`:
```
local ca_bundles="$INSTALL_DIR/etc/*.pem"
```
This PR changes `sslv2_sockets()` so that a list of ciphers may optionally be passed as an argument. This will support the use of `sslv2_sockets()` in some places where `$OPENSSL s_client` is currently used.
David Cooper