The server default run had several JSON objects which weren't, looking at just
the ID, either clear or contained a redundant explanation in "finding". Purely
certificate related JSON objects are now having the id "cert_<object>" like
cert_CN or cert_SAN.
This commit changes all this, also it avoids another colon in finding (see #830).
Also the implicit strategy "output for the screen s followed by only one output with
fileout" has been relaxed -- which results on more, better parsable JSON objects.
Some example of the changes:
Old:
----
{
"id" : "Server Certificate #1 fingerprint",
"severity" : "INFO",
"finding" : "Fingerprints / Serial: SHA1 2940BC13ECF7DAF30B9084CC734C3B971D73B3BB / 01BFD1DC15006E0ABBA7C670FF5E1101, SHA256 30BA61012FFE7CEAAF9A148A0CB0C5C852A9C04F4B1C27DB6
EFA9919C7F49CCF"
}
[..]
{
"id" : "Server Certificate #2 ocsp_stapling",
"severity" : "OK",
"finding" : "OCSP stapling : offered"
}
New:
----
{
"id" : "cert_key_size <cert#1>",
"severity" : "INFO",
"finding" : "Server keys 2048 bits"
},{
"id" : "cert_fingerprint_SHA1 <cert#1>",
"severity" : "INFO",
"finding" : "2940BC13ECF7DAF30B9084CC734C3B971D73B3BB"
},{
"id" : "cert_fingerprint_SHA256 <cert#1>",
"severity" : "INFO",
"finding" : "30BA61012FFE7CEAAF9A148A0CB0C5C852A9C04F4B1C27DB6EFA9919C7F49CCF"
},{
"id" : "cert_serial <cert#1>",
"severity" : "INFO",
"finding" : "01BFD1DC15006E0ABBA7C670FF5E1101"
}
[..]
{
"id" : "OCSP_stapling <cert#2>",
"severity" : "OK",
"finding" : "offered"
}
This PR also fixes the JSON output where for "OCSP must staple" the id was just
'id" : "OCSP must staple: ocsp_must_staple",' for multiple server
certificates without the certificate number.
As far as the code is concerned: $json_prefix should be a variable which is
used for the id object. If there was more then one certificates for a single
host detected, $json_postfix carries the certificate number.
Unit tests need to be fixed -- if possible.
Add fileout() to #965. This commit also contains a change which needs
to be commited before: separation of ``json_prefix`` from ``json_postfix``.
Open issue: sed in openssl x509 statments look GNUish ([ \t]). Needs clarification.
This commit prints the contents of the keyUsage and extended key usage extensions in certificates and checks the public keys in the certificates are not being used in a manner that is inconsistent with these extensions.
This PR is intended to improve the functionality of run_tls_fallback_scsv().
The original goal of this PR was to address servers that support TLSv1.3 when using OpenSSL 1.1.1. That does not seem to be an issue, as using `$OPENSSL s_client` with the `-no_tls1_2` flag results in a TLSv1.1 ClientHello, even if `$OPENSSL` supports TLSv1.3. However, if the server supports TLSv1.3, then a message that says "No fallback possible, TLS 1.2 is the only protocol (OK)" isn't entirely correct.
The main issue this PR fixes is some false positives in servers that do not support TLSv1.2. On a few servers that I tested, the current code incorrectly reports "Downgrade attack prevention NOT supported." Some of the servers only support TLSv1, so it should report that fallback is not possible. Another server supports TLSv1.1 and TLSv1, and it supports fallback protection. In both cases, the current code produces a false positive, since it assumes that TLSv1.2 is supported.
In three different places there is a line that is supposed to check whether the list of ciphers to be tested contains any TLSv1.3 ciphers. This check currently fails if there is only one TLSv1.3 cipher in the list and it is the first cipher in the list. This commit fixes the problem.
This commit adds support for draft 23, which contains 2 changes that are relevant for testssl.sh. It adds a few new values for the signature_algorithms extension and it changes to extension number for the key_share extension from 40 to 51.
With the change in the extension number, it is no longer possible to send a single ClientHello that works for all supported drafts of TLSv1.3. (I tried sending a ClientHello with two key share extensions, 40 and 51, but that didn't work.) So, this commit adds a test to determine_optimal_proto() to determine whether TLSv1.3 is supported and if so whether draft 23 is supported or only some earlier draft (18-22). In subsequent tests, the ClientHello uses the appropriate number for the key share (40 or 51) and specifies the appropriate draft version(s) in the supported_versions extension (either 23 or 18-22). In the case of run_protocols() the test for each draft version uses the appropriate key share extension number so that servers that support both draft 23 and an earlier draft can be detected.
This fixes a bug which prevented the script from running properly. Also
the commit restores writing a correct comment header. In addition it
adds TLS 1.3 ciphers.
support. If the server doesn't supply an session identifier the file
is just empty.
This commit fixes that by adding a separate case for OpenSSL 1.1.1
and an empty file. It is deliberately only changing this as this
was tested to work.
It is prelimary and a save-the-work-patch as it might be better
to catch this earlier.
For JSON pretty the host specific parameters target host + port
could be better placed in the scanResult object.
It is still under discussion as logically the parent object is deduced
the command line.
TLS_RSA_* which don't fall into the aleady mentioned
categories (CBC cipher, export, RC4 etc.) are now
a bit more more penalized. Those are the ones which have
an RSA key exchange AND a modern encryption.
pr_cipher_quality() needs to be redone after carefully
reconsidered which cipher should have which rating.
This PR is similar to #944. If using OpenSSL 1.1.1 to connect to a server that supports TLSv1.3, `run_crime()` will connect to the server using TLSv1.3, which does not support TLS-level compression. So, the server will be reported as "not vulnerable" even if would use compression for connections at TLSv1.2 and below.
I have not encountered any "live" servers that support both TLSv1.3 and TLS-level compression. I verified this problem by using OpenSSL 1.1.1 to create a server that supports both TLSv1.3 and TLS-level compression:
```
openssl111 s_server -cert cert.pem -key key.pem -accept 8443 -WWW -comp
```
I then tested the server using `testssl.sh --crime` with both openssl111 and OpenSSL 1.0.2-chacha.
run_renego() appears to produce a false positive if OpenSSL 1.1.1 is used and the server being tested supports TLSv1.3 (i.e., the server supports the same draft version of TLSv1.3 as the version of OpenSSL 1.1.1 being used does). This PR fixes the problem by telling calls to $OPENSSL s_client in run_renego() to not use TLSv1.3.
.. to check during the default run for server implemenation bugs
and run cipher per procol check instead of cipher check.
Please not that this option could disappear later.
In TLSv1.3 servers may send a supported_groups extension, which "SHOULD contain all groups the server supports, regardless of whether they are currently supported by the client."
This PR extracts the contents of the supported_groups extension, if `parse_tls_serverhello()` is to process "all" of the server's response. The contents of the extension are also displayed on the terminal if $DEBUG -ge 3.
In TLSv1.2 and below, servers respond to a status_request extension (a request for a stapled OCSP response) by returning an empty status_request extension and then including a CertificateStatus message, which follows the Certificate message. In TLSv1.3 the CertificateStatus response is included as the value of the status_request extension, which now appears as an extension within the Certificate message.
This PR extracts the contents of the status_request extension sent by the server so that it can later be processed in the same way as if it had sent in a TLSv1.2 or below response.
This PR adds code to decrypt the encrypted portion of the server's response for TLSv1.3 and to then process any certificates and encrypted extensions. This code supports all 5 TLSv1.3 cipher suites, and so any response can be decrypted as long as the session key can be derived (which requires OpenSSL to support the ephemeral key that was used - see #938).
For the symmetric decryption, the sym-decrypt() function uses OpenSSL when possible and internal Bash functions when needed.
For AES-GCM and AES-CCM ciphers sym-decrypt() normally uses internal Bash functions, which rely on using "$OPENSSL enc" in AES-ECB mode to generate the key stream and then Bash functionality to XOR the key stream with the ciphertext. With some version of OpenSSL the AES-GCM ciphers are decrypted using "$OPENSSL enc" in AES-GCM mode directly. On my system, however, both methods seem to work about equally fast.
For ChaCha20 ciphers, "$OPENSSL enc -chacha20" is used, if supported (OpenSSL 1.1.x only). and Bash internal functions (without any OpenSSL support) are used otherwise. In this case, if the Bash internal functions need to be used, decryption is very, very, very slow. Fortunately, in a typical run of testssl.sh there won't be many cases in which the connection will be TLSv1.3 with ChaCha20 and the entire response needs to be processed (requiring decryption). In most cases, even if the connection is TLSv1.3 with ChaCha20, will at most need the ephemeral key, which is available in plain text.
This is the first in a series of PRs to add support for processing the encrypted portions of the server's response in a TLSv1.3 handshake.
This PR adds the code to derive the handshake traffic key needed to decrypt the response (the next PR will add the code to perform the symmetric-key decryption of the encrypted portions of the response).
Since this PR does not make use of the traffic key that it derives, it doesn't yet add any new functionality.
Note that testssl.sh will not always be able to derive the session keys. If the version of OpenSSL that is bundled with testssl.sh is used and the server chooses to use an X25519 ephemeral key, OpenSSL will be unable to perform the shared secret in derive-handshake-traffic-secret(). (OpenSSL 1.1.0 supports X25519.) Since X25519 use a different encoding than ECDH keys, the lack of X25519 support will be discovered in parse_tls_serverhello() when $OPENSSL pkey is unable to convert the key from DER to PEM. So, in debugging mode, parse_tls_serverhello() now displays a warning if it receives a key share that $OPENSSL pkey cannot handle.
get_server_certificate() uses an awk script to extract the certificates from the output of OPENSSL s_client and it then uses the following line to determine how many certificates were found:
nrsaved=$(count_words "$(echo level?.crt 2>/dev/null)")
If $nrsaved is 0, then get_server_certificate() returns 1 (indicating failure); otherwise it returns 0 (indicating success).
However, the check for the number of certificates returned doesn't work if no certificates were found, as nrsaved will be set to 1 if no certificates were found:
> touch level0.crt
> echo level?.crt
level0.crt
> touch level1.crt
> echo level?.crt
level0.crt level1.crt
> rm level0.crt level1.crt
> echo level?.crt
level?.crt
This PR fixes the problem by first checking that level0.crt exists (-s is used instead of -e, since an empty file wouldn't have a certificate).
Dirk
Jack Stevens
David Cooper
Cédric Tissières