In cases in which the server offers a stapled OCSP response, this commit extracts the OCSP response and then checks the response for the status of the server's certificate. The check is performed in the same way as when the certificate includes an OCSP URI and the "--phone-out" option is set, except that the OCSP response is received from the TLS server rather than coming directly from the OCSP responder. Since this only involves additional processing of data that testssl.sh is already receiving, the check is performed whether or not the "--phone-out" flag is set.
This PR fixes#615 for the case in which tls_sockets() is used by splitting the list of CBC ciphers into two lists, each with fewer than 128 ciphers and then testing each list separately.
For the --ssl-native case, no changes were needed. Even though $cbc_ciphers contains 154 ciphers, no version of OpenSSL supports all of these ciphers, and so the actual ClientHello sent by every version of OpenSSL contains fewer than 128 ciphers.
I did, however, add the -no_ssl2 flag to the "$OPENSSL s_client" command to prevent OpenSSL from sending an SSLv2-compatible ClientHello. As is noted in a comment in run_server_preference(), "the supplied openssl will send an SSLv2 ClientHello if $SNI is empty and the -no_ssl2 isn't provided."
This PR fixes problems with check_revocation_crl() sometimes reporting that a certificate is revoked even when it isn't, and with check_revocation_ocsp() sometimes reporting "error querying OCSP responder" even if the OCSP responder provided a good response. The most common reason for this to happen is that OpenSSL cannot validate the server's certificate (even without status checking). PR #1051 attempted to get status checking to work even in cases in which the server's certificate could not be validated. This PR instead addresses the problem by not checking status if determine_trust() was unable to validate the server's certificate.
In some cases the server's certificate can be validated using some, but not all of the bundles of trusted certificates. For example, I have encountered some sites that can be validated using the Microsoft and Apple bundles, but not the Linux or Mozilla bundles.
This PR introduces GOOD_CA_BUNDLE to store a bundle that could be used to successfully validate the server's certificate. If there is no such bundle, then neither check_revocation_crl() nor check_revocation_ocsp() is run. When check_revocation_crl() and check_revocation_ocsp() are called, the status checks within them closely match the validation check in determine_trust(), which helps to ensure that if the check fails it is because of the status information.
As noted in #1057, at least one CA provides incorrect information when the CRL is downloaded, so validation could fail for a reason other than the certificate being revoked. So, this PR adds a check of the reason that validation failed and only reports "revoked" if the validation failed for that reason.
As noted in #1056, it is not possible to perform an OCSP query without access to the certificate issuer's public key. So, with this PR check_revocation_ocsp() is only called if the server's provided at least one intermediate certificate (i.e., the issuer's certificate, which contains the issuer's public key).
This PR improves the handling of error responses when checking status using OCSP. It can handle a few types of errors:
* When the responder just returns an error (e.g., "Responder error: unauthorized").
* When the response cannot be verified (e.g., invalid signature, expired certificate).
* When the response is valid ("Response verify OK"), but there is a problem with the response for the individual certificate (e.g., information is too old, or status is "unknown").
This PR fixes two issues with OCSP checking. First, the syntax for specifying a host header changed in OpenSSL between versions 1.0.2 and 1.1.0. With OpenSSL 1.0.2-chacha, 1.0.2o, and LibreSSL the syntax needs to be "-header HOST <hostname>". With OpenSSL 1.1.0h and 1.1.1 the syntax needs to be "-header HOST=<hostname>". I have not been able to test other versions of OpenSSL 1.0.2 or 1.1.0, but am assuming that all versions of OpenSSL 1.1.0 use the same syntax as 1.1.0h.
This PR also fixes a typo in the case of an error, which was causing $code to be set to "empty ocsp response" if the response was not empty rather than if it was empty.
The current code for extracting the validity dates for certificates assumes that the strings "Not Before" and "Not After" will appear exactly once in the pretty-print of the certificate. In most cases that works. However, there are a few server certificates that include the private key usage period extension, which also includes "Not Before" and "Not After" times. The result is that the current code does not correctly extract the start date and end date from any certificates that have private key usage period extensions.
This PR fixes the problem and also speeds up extraction of the dates by only using Bash internal functions.
The pretty-print of a certificate begins as follows:
Certificate:
Data:
Version: 3 (0x2)
Serial Number: ...
Signature Algorithm: ...
Issuer: ...
Validity
Not Before: ... GMT
Not After : ... GMT
...
The code in this PR extracts the start date by first removing from the certificate everything that comes before "Not Before: ". It looks for the shortest string that includes ""Not Before: " in order to ensure it is not getting the date from the private key usage period extension. After that, the longest string that begins with "GMT" is removed so that only the notBefore date remains.
The part that removes the string up to "Not Before: " actually looks for the first instance of "Not Before: " that comes after the "Validity". This is to protect against the unlikely possibility that the string "Not Before: " appears somewhere in the issuer's name.
The extraction of the notAfter date works similarly. It first looks for the first instance of "Not After :" that appears after both "Validity" and "Not Before: " and then takes the date string that appears immediately afterwards, with the assumption that the date string ends in "GMT".
In some cases the OCSP URI contains multiple components in the path (e.g., http://www.example.com/OCSP/myOCSPresponder).
This PR changes check_revocation_ocsp() to remove all components in the path, rather than just the final component, when extracting the host name from the URI for the host header.
