This PR adds parsing of the Certificate message to `parse_tls_serverhello()`. If the caller requests that the "full" response be parsed, then the Certificate message is parsed, the server's certificate is placed in $HOSTCERT and the intermediate certificates are placed in $TEMPDIR/intermediatecerts.pem. The certificates are also added to $TMPFILE, in a manner similar to the output of `$OPENSSL s_client` when the `-showcerts` option is used.
This PR uses `tls_sockets()` to determine whether a server supports certain extensions that may not be supported by `$OPENSSL`. At the moment it checks for max_fragment_length, client_certificate_url, truncated_hmac, ALPN, signed_certificate_timestamp, encrypt_then_mac, and extended_master_secret.
In https://github.com/dcooper16/testssl.sh/blob/extended_tls_sockets/testssl.sh, `run_server_defaults()` is re-written to use `tls_sockets()` instead of `$OPENSSL`, with just one call to `$OPENSSL s_client` to get the session ticket, which reduces the dependence on `$OPENSSL`, but this PR limits the number of calls to `tls_sockets()`, which is still slow.
Note: I included ALPN in the `tls_sockets()` ClientHello since a single call to `tls_sockets()` cannot test for both NPN and ALPN, and since support for NPN was added to OpenSSL before support for ALPN was added, I figured it was more likely that `determine_tls_extensions()` had already determined whether the server supported NPN.
This PR fixes the same issues as were fixed in PR #513, but also makes two changes to `parse_tls_serverhello()`:
* It changes the number of bits for curve X25519 from 256 to 253 to match OpenSSL.
* It removes the "ECDH, " from the "Server Temp Key: " line in order to match OpenSSL's output.
This PR fixes two issues related to curve X25519.
First, while OpenSSL 1.1.0 supports curve X25519, it is not included in the output of `$OPENSSL ecparam -list_curves`. I tried several versions of OpenSSL (and one version of LibreSSL), and every version output either "Error with command" or "unknown option" in response to `$OPENSSL s_client -curves $curve` if it either did not support the `-curves` option or did not support `$curve`. (When the `-curve` option was supported with `$curve`, a "connect" error was output.)
The second issue is that the "Server Temp Key" line in the output of `s_client` is different for curve X25519. For other elliptic curves, the output is
```
Server Temp Key: ECDH, P-256, 256 bits
```
For X25519 it is:
```
Server Temp Key: X25519, 253 bits
```
So, `read_dhbits_from_file()` needs to allow for `$what_dh` being "X25519" rather than "ECDH" and `run_pfs()` needs to allow for the possibility that the curve name will be the first field rather than the second.
The PR changes `run_allciphers()` to use `tls_sockets()` (and `sslv2_sockets()`)rather than `$OPENSSL` unless `$SSL_NATIVE` is set or `$STARTTLS` is non-empty. Using sockets allows `run_allciphers()` to test all ciphers, rather than just those supported by `$OPENSSL`.
Using sockets results in `run_allciphers()` running more slowly, partially since it is testing more ciphers, but mostly since `tls_sockets()` is currently slower than `$OPENSSL` (as noted in #413).
This PR makes similar changes to `run_client_simulation()` as were made to `tls_sockets()`, so that `run_client_simulation()` retrieves the entire server response, even if it is split across multiple packets, and it has `parse_tls_serverhello()` extract information about the server's ephemeral public key, if present.
The PR also changes `run_client_simulation()` to use information about the ephemeral public key. It includes the length of the public key in the output and, if it is a DH public key, checks that the size is within the acceptable range (`${minDhBits[i]} <= dh_bits <= ${maxDhBits[i]}`).
This PR adds initial parsing of the ServerKeyExchange message to `parse_tls_serverhello()`. For ephemeral DH keys, it extracts the length of the key. For ephemeral ECDH keys that are encoded using the named_curve option, it extracts the length of the key and the name of the curve.
This PR allows the caller to provide additional extensions to `tls_sockets()` to be included in the ClientHello. If the caller provides an extension that would have already been included in the ClientHello, then the caller's value for the extension is used rather than the default value.
This PR extended `parse_tls_serverhello()` in a few ways:
* If the "full" response is to be parsed, then additional checks are performed to verify that `$tls_hello_ascii` contains the entire response
* The extensions field is parsed and the list of extensions found is placed in `$TLS_EXTENSIONS` (if the "full" response is being parsed).
* Initial support for TLS 1.3 is added:
- Accounts for differences between TLS 1.2 ServerHello and TLS 1.3 ServerHello (as outlined in PR #499).
- Recognizes new alerts and handshake message types.
- Allows for server response to include message fragments of type "application data"
I forgot that `parse_tls_serverhello()` is also called by `client_simulation_sockets()`. Since PR #499 changed the input to `parse_tls_serverhello()`, the change needs to be made in `client_simulation_sockets()` as well.
Dirk
David Cooper
AlGreed