This PR modifies `cipher_pref_check()` to use `tls_sockets()`. As with similar PRs for `run_allciphers()`, `run_cipher_per_proto()`, and `run_rc4()`, it also makes use of `$OPENSSL s_client`, since `$OPENSSL s_client` is faster than `tls_sockets()`.
With this PR, `cipher_pref_check()` first uses `$OPENSSL s_client` to obtain an ordered list of ciphers. It then makes one call to `tls_sockets()` (or a few calls if proto is TLSv1.2 and `$SERVER_SIZE_LIMIT_BUG` is `true`) to find if the server supports any ciphers that are not detected by `$OPENSSL s_client`. If not, then it is done. If it finds one, then it throws out the previous results and starts over with `tls_sockets()`. [If proto is TLSv1.2 and `$SERVER_SIZE_LIMIT_BUG` is `true`, then it doesn't throw out the `$OPENSSL s_client` results. Instead, it continues with `tls_sockets()` to get the full list of supported ciphers, and then uses `tls_sockets()` to order that list.]
The result is that this PR works almost as fast as the current `cipher_pref_check()` if `$OPENSSL s_client` finds all of the supported ciphers, at the cost of a performance penalty when testing servers that support ciphers that would have otherwise been missed using just OpenSSL.
Note that in this PR I removed SSLv2 from the list of protocols tested. This is because https://community.qualys.com/thread/16255 states that "in SSLv2 the client selects the suite to use." It seems that in SSLv2, the client sends a list of ciphers that it supports, the server responds with a list of ciphers that the client and server have in common, and then "the client selects the suite to use." So, showing a cipher order for SSLv2 is a bit misleading.
As noted in #543, this PR does not modify the second part of `cipher_pref_check()`, which deals with NPN protocols.
This PR implements `run_rc4()` in a similar manner to `run_allciphers()` and `run_cipher_per_proto()` (in PR #541). The change doesn't seem to have much of an impact on speed, but when sockets are used it can detect ciphers that aren't locally supported by OpenSSL.
This PR fixes two minor bugs in run_allciphers():
* If `$SSL_NATIVE` or `$FAST` is `true`, then the cipher mapping file will not be used (unless `$OPENSSL ciphers` does not support the `-V` option), so there is no "fallback" to openssl, even if `[[ $TLS_NR_CIPHERS == 0 ]]`.
* If `$using_sockets` is `false` and `$SHOW_EACH_C` is `true`, then `ossl_supported` should be checked to see if the cipher was tested, not `TLS_CIPHER_OSSL_SUPPORTED`.
This function reorganizes `run_server_defaults()` based on the suggestion in #515.
The current `determine_tls_extensions()` is renamed to `get_server_certificate()`, and two changes are made to it:
*it no longer includes an extra call to `$OPENSSL s_client` to check for the ALPN extension; and
* rather than setting `$TLS_EXTENSIONS` to be the extensions found during this call to the function, it adds any newly found extensions to those already in `$TLS_EXTENSIONS`.
The PR then adds a new function, `determine_tls_extensions()`, which borrows some logic from the old `determine_tls_extensions()`, but this new `determine_tls_extensions()` only looks for additional TLS extensions, including ALPN.
`run_server_defaults()` makes multiple calls to `get_server_certificate()` (as it previously did to `determine_tls_extensions()`) in order to collect all of the server's certificates, and then it makes one call to `determine_tls_extensions()`, which checks for support for extensions that were not checked for by `get_server_certificate()` (e.g., ALPN, extended master secret, signed certificate timestamps).
The new `determine_tls_extensions()` will check for most of the extensions that are checked for by
`run_server_defaults()`, including the heartbeat extension, so the call to `determine_tls_extensions()` from `run_heartbleed()` will still work.
> The dh_bits are still not shown, maybe because of #531.
This PR fixes the issue of dh_bits not being shown if the cipher-mapping.txt file is missing. The problem is that the code in `parse_tls_serverhello()` that parses the ServerKeyExchange message assumes that `$rfc_cipher_suite` has the RFC version of the name the cipher suite. However, if the cipher-mapping.txt file is missing, `$rfc_cipher_suite` will have the OpenSSL name of the cipher suite. This PR changes the code to recognize either the RFC or OpenSSL names for ciphers with ephemeral DH or ECDH keys.
When `tls_sockets()` is used with the "full" option and the chosen cipher suite involves an ephemeral finite-field DH key (DH), this PR extracts the public key from the ServerKeyExchange message and adds it to `$TMPFILE`. In addition (and the primary reason for this PR), it compares the ephemeral public key's parameters to those specified in RFC 7919, and indicates whether one the groups from that RFC was used. This will allow `run_pfs()` to be extended to indicate which, if any, RFC 7919 DH groups a server supports.
This PR adds parsing of the CertificateStatus message to `parse_tls_serverhello()`. If the caller requests that the "full" response be parsed, then the CertificateStatus message is parsed, and the OCSP response is added to $TMPFILE, in a manner similar to the output of `$OPENSSL s_client` when the `-status` option is used.
The string "CamelliaGCM" is too long for the "Encryption" column printed by `neat_list()`. So, either "CamelliaGCM" needs to be shortened to "Camellia" (as this PR does), or the "Encryption" column needs to be made wider.
Client simulations can still use sockets even if the cipher mapping file is missing. If the cipher file is present, then `parse_tls_serverhello()` write the RFC name for the cipher and then `run_client_simulation()` converts that to the OpenSSL name (so that the output is the same as if OpenSSL were used). This PR changes `parse_tls_serverhello()` so that it writes the OpenSSL name for the cipher if the mapping file is missing, which `run_client_simulation()` can then just display.
This PR also unsets `ADD_RFC_STR` if the mapping file is missing, so that `neat_list()` won't try to display the RFC names for the ciphers.
This PR speeds up the implementation of `run_allciphers()` by introducing a number of changes:
* Rather than check for implemented ciphers in a hierarchical manner (as introduced in #326), this PR follows the approach of `cipher_pref_check()`. Testing a block of ciphers, marking the selected cipher as implemented, and then testing same block of ciphers, minus those that have previously been selected, until a test fails. Thus the number of calls to `$OPENSSL s_client` is just one more than the number of ciphers implemented. (Since some servers cannot handle ClientHellos with more than 128 messages, the tests are performed on blocks of 128 or few ciphers. So, if OpenSSL supports 197 ciphers, the number of calls to `$OPENSSL s_client` is 2 plus the number of ciphers supported by the server.
* If $using_sockets is true, then OpenSSL is used first to find all supported ciphers that OpenSSL supports (since OpenSSL is faster than `tls_sockets()`), and then `tls_sockets()` is only used to test those cipher suites that were not found to be supported by OpenSSL.
* The `prepare_debug()` function, which reads in `$CIPHERS_BY_STRENGTH_FILE` determines which ciphers are supported by the version of OpenSSL being used. If a version of OpenSSL older than 1.0 is being used, then this is used to determine which ciphers to test using OpenSSL rather than using `$OPENSSL ciphers -V`.
Following the approach of `cipher_pref_check()` reduces the number of queries to the server. Using OpenSSL before `tls_sockets()` reduces the number of calls to `tls_sockets()` to 3 plus the number of ciphers supported by the server that are not supported by OpenSSL, so the cost penalty over just using OpenSSL is fairly small.
AlGreed
David Cooper
Dirk Wetter
typingArtist