run_logjam() is only related to TLSv1.2 and earlier ciphers. So, run_pfs() should only update $DH_GROUP_OFFERED if a DH group was found using a non-TLSv1.3 cipher.
On the other side, if run_logjam() happened to have been run first, and it found an ffdhe cipher, then there is no need for run_pfs() to test for it.
In run_pfs(), when information about the finite field groups offered is printed, the color used is based on the length of the key. This information should also be conveyed to fileout() in the severity parameter.
For cipher suites that use ephemeral DH groups, run_pfs() currently only displays information about the group(s) used if the server complies with RFC 7919. In the case of TLSv1.3 this is appropriate, since server can only use the values from this RFC and only if they are offered by the client in the supported_groups extension.
For TLSv1.2 and earlier, however, servers are free to use whatever DH group they want, but run_pfs() only provides information about the group the server uses if the server complies with RFC 7919. (The information is, however, provided by run_logjam()). However, so far no servers comply with RFC 7919's requirement to refuse to negotiate a TLS_DHE cipher if the supported groups extension is present, included DH groups, but none that are supported by the server. There is also reason to believe that this will not change: https://www.ietf.org/mail-archive/web/tls/current/msg26378.html.
So, this PR proposes to change the way that run_pfs() searches for DH groups for TLSv1.2 and earlier. (Note that run_pfs() only checks for TLSv1.2 or earlier if the $EXPERIMENTAL flag is set to true.) First, it removes the test to see if the server will reject a ClientHello that only specifies TLS_DHE cipher suites if it includes a supported_groups extension that only specifies an unrecognized DH group. Instead, if the server supports TLS_DHE cipher suites (at TLSv1.2 or earlier) and the $EXPERIMENTAL flag is true, it will try to find out what group(s) the server uses. Second, it will report the group(s) found even if the server uses a group that does not come from RFC 7919.
The result is that if the server supports selecting groups from the supported_groups extension, it will print all of the groups that the server supports. If the server ignores the supported_groups extension and always uses the same group, it will print essentially the same information as is already printed by run_logjam().
One discrepancy, however, is that this code use pr_dh_quality() to determine how good a DH group is, based on the length of the prime, and pr_dh_quality() has differs from run_logjam() in terms of how it rates groups based on the lengths of their primes.
There are a couple of old SSLv2 ciphers which haben't been included in
etc/cipher-mapping.txt . This PR updates the file. Names were derived
from the (old) OpenSSL / SSLeay source code.
In addition TLS_NULL_WITH_NULL_NULL (>=SSLv3 cipher) was added.
ToDo: Review functions to be updated to use those ciphers.
Some SSLv2 ciphers were missing (see openssl/ssl/ssl2.h and
SSLeay (ssl.h + ssl_lib.c).
Also in this list security bit strength None were renamed to '0',
encryption None to Null.
.. otherwise we'll hit too soon the threshold: Logic: by specifying
a timeout a user indicates that there might be a problem.
Also fatal() now supports a hint which is printed in normal
text (to stderr)
Some Cyrus IMAD if configured with SSL_CTX_set_cipher_list(context, "!TLSv1")
and similar respond with a plaintext 'a002 NO Starttls negotiation failed"
when a not-supported protocol is detected, see #1082.
This PR fixes this by detecting (also) this downgrade. As a precaution
It still issues a warning as this is seems a special configuration.
Looking @ pending #1114 two improvements were done:
1) Keep the status of DH group detected (<name> or "Unknown DH group")
as well as the bit length
2) move the detection to a separate function get_common_prime()
There's still room for improvements when run_pfs() will take
over a part.
Also double code (my bad) from run_logjam() was move to a separate function.
As #1146 noted some installations miss hexdump. Better practice
is to check before what's needed albeit the error message when
a binary is missing does give the user a hint.
Currently the client simulation is based on the handshake data
from SSLlabs which is purely focussed on HTTP -- as SSLlabs does
HTTP only.
In #540 there was a PR addressing the fact that the data is not
what is claims to be -- the handshake of Android 7 seems to be
Chrome for Android and not Android itself.
This PR tries at least to modify the headline for client simulations.
This PR addresses the remaining TCP fragmentation by piping the line buffered
internal print through cat, see also #1130.
It extends 1b52834 which was the same doing for Linux and
OpenBSD.
This PR also consolidates the last remaining low level socket calls
in client_simulation_sockets() into socksend_clienthello().
An negative performance effect is barely measurable.
It also does a check whether the fd 5 is taken by a tty as
I see this while writing the commit message ;-). We might
want to make that line better instead of just echoing. :-)
At the beginning of run_server_preference(), if the attempt to connect to the server is unsuccessful, a message is printed listing all of the ciphers in $list_fwd and $tls13_list_fwd:
no matching cipher in this list found (pls report this): DES-CBC3-SHA:RC4-MD5:DES-CBC-SHA:RC4-SHA:AES128-SHA:AES128-SHA256:AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:ECDH-RSA-DES-CBC3-SHA:ECDH-RSA-AES128-SHA:ECDH-RSA-AES256-SHA:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-DSS-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:AES256-SHA256:ECDHE-RSA-DES-CBC3-SHA:ECDHE-RSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-SHA256:ADH-AES256-GCM-SHA384:AECDH-AES128-SHA:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-AES128-SHA:TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256
This message can be misleading. I tested a server that only supported TLSv1.3 using the provided OpenSSL 1.0.2-chacha. The server supported TLS_AES_256_GCM_SHA384, but OpenSSL didn't. However, the message implies that the server does not support TLS_AES_256_GCM_SHA384.
This PR changes the message (and the one included in CSV/JSON output) to only list those ciphers in $list_fwd and $tls13_list_fwd that are actually supported by $OPENSSL.
Note that even with this PR, some ciphers are listed that aren't really supported by $OPENSSL, since the `-s` option isn't used. But, that is #663.
* Put all low level socket related functions close to each other
* removed socksend2 as it was not used and outdated looking forward
* socksend_sslv2_clienthello() renamed to socksend_clienthello() as
it wasn't particular SSLv2 related
* removed the low level socket calls from socksend_tls_clienthello()
and called socksend_clienthello() instead
* renamed socksend_tls_clienthello() to prepare_tls_clienthello()
as it is not a low level function anymore
This PR is also based on #1139, but it addresses ECDH keys rather than DH keys. When run_pfs() prints the list of elliptic curves offered, it colors each curve according to its quality (based on key length). However, the severity level used when the list is sent to fileout() is always "INFO". This PR changes the call to fileout() to make the severity level be based on the quality of the shortest curve that the server offers.
This commit adds LMTP to the STARTTLS protocols
supported. It requires an openssl version which
supports this which is either openssl 1.1.1
or a backported version 1.0.2 (binary is in
process).
A check is in place whetrher the binary supports
this.
Furthermore some framework additions were made for
further STARTTLS protocols like IRC and NNTP.
For XMPP servers, when extracting the SRV-ID and XmppAddr names from the subjectAltName extension, need to take into account that the subjectAltName extension may be marked as critical, in which case there will be the DER encoding of TRUE (0101FF) between the DER encoding of the subjectAltName extension's OID (0603551D11) and the tag for OCTET STRING (04).
This PR is an attempt to address #1097. I have only been able to test it against jabber.topf.org and against locally created test certificates, so it needs more testing.
The main issue that this addresses is that testssl.sh currently checks against the wrong name for XMPP servers. According to RFC 6120, Section 13.7.2.1:
o The initiating entity sets the source domain of its reference
identifiers to the 'to' address it communicates in the initial
stream header; i.e., this is the identity it expects the receiving
entity to provide in a PKIX certificate.
So, if the --xmpphost option is provided, then the name provided in that option should be compared against the name in the certificate rather than the host name.
compare_server_name_to_cert() currently takes the server name to look for in the certificate as an parameter, but every call to compare_server_name_to_cert() uses $NODE as the argument. So, this PR removes the parameter and sets $servername to either $XMPP_HOST or $NODE as appropriate. This small change alone should fix the problem for most XMPP servers since the server's name SHOULD appear in the certificate encoded as a DNS name. That is the case for the one server I could test - jabber.topf.org.
The majority of the code in this PR is to address the other possibilities noted in RFC 6120, Section 13.7.1.2.1. This section notes that an XMPP server's identity name also appear in the subjectAltName extension as an otherName, either an SRV-ID or an XmppAddr identifier. Unfortunately, OpenSSL's certificate printer does not support otherName and just prints "othername:<unsupported>". So, this PR includes code to manually extract any SRV-ID or XmppAddr names from the certificate. This involves parsing the DER encoding of the certificate to look for the subjectAltName extension, looping through all of the names in the extension, and pulling out the names of these two name forms. This code is only run if the server is an XMPP server and the certificate does not have a matching DNS name. So, this code will rarely be executed.
This PR addresses one other issue. There is code in certificate_info() to set the variables $has_dns_sans and $has_dns_sans_nosni. These variables are needed to address the following requirement:
# Find out if the subjectAltName extension is present and contains
# a DNS name, since Section 6.3 of RFC 6125 says:
# Security Warning: A client MUST NOT seek a match for a reference
# identifier of CN-ID if the presented identifiers include a DNS-ID,
# SRV-ID, URI-ID, or any application-specific identifier types
# supported by the client.
While it is relatively easy to determine whether a certificate includes a DNS name in the subjectAltName extension, as noted above, it is not easy to determine whether it has an SRV-ID or an XmppAddr. So, this PR leverages the work compare_server_name_to_cert() does in parsing the subjectAltName extension by having compare_server_name_to_cert() set a global variable indicating whether the certificate has a subjectAltName extension with a relevant name form (DNS, SRV-ID, or XmppAddr for XMPP, or DNS for other servers). $has_dns_sans and $has_dns_sans_nosni are then just set to the value of this global variable.
Instead of checking via uname for Linux this commit does a check
whether the outcome for an external printf is what is expected. This
makes it more compatible e.g. with OpenBSD which surprisingly works
similar like the GNU counterpart.
Also it checks all external printfs installed wther it's the
"right one" to use. Previously it stopped just at the first one
and if this was "wrong", bash's printf was used.
Linux bash internal printf shortened the string
when using len2twobytes() with 3 chars, FreeBSD
e.g. did not.
It worked under both OS though when piping to
the socket with printf.
This commit makes sure that always 2+2 chars
are returned when a 3 char number is supplied.
Kudos @dcooper16
This commit basically reverses the previous commit 305eefc for
Linux only. Here the external printf is working fine, where as
the BSDish counterpart is not (see e.g. #1134).
For Linux is basically addresses #1130 / #1113.
A compatible solution for all OS needs to be found.
This PR removes the changes that were added in #1113. As noted in the conversation for #1113, it was eventually determined that the actual bug was related to the first message fragment being too short rather than the overall length of the message. So, the warning message that is displayed by the code is misleading. In addition, if changes are made to avoid TCP fragmentation, then this code will no longer even test for intolerance to short message fragments.
As @tomato42 pointed out in #1113 '\x0a' causes the printf buffer
to flush before all data was sent. As a result any '\x0a' in
a ClientHello causes a new TCP fragment.
This commit changes all TCP sockets write to use an external
printf if available which doesn't show this behaviour, see #1130.
It was checked against wireshark.
The external printf was available for Linux, FreeBSD 9 and OpenBSD,
so I do not expect any problems with MacOS X either.
There might be further solutions like 'stdbuf' or 'dd' which
are shown in #1130.
This commit removes the '0a' character from public keys used in the key_share extension. New key pairs were created by repeatedly generating new keys until one was found that had no '0a' characters in the public key.
As noted in #1130, the current implementation of socksend_tls_clienthello() results in packets being fragmented wherever a '0a' character appears in the message. This cannot be avoided, but there are a few places where a '0a' character appears in which the character could easily be replaced:
* In the session_id for a TLSv1.3 ClientHello.
* In the 32-byte client random value
* In any public key sent in the key_share extension
This PR removes those uses of the '0a' character. While this does not do much to address the problem, it does result in a slight reduction in the amount of fragmentation of messages.
This PR is an attempt to fix the problem identified in #1118.
Currently, get_cipher() and get_protocol() attempt the extract the cipher and protocol from the SSL-Session information printed by OpenSSL s_client. This does not always work for TLSv1.3, however, since OpenSSL 1.1.1 will only print SSL-Session information for a TLSv1.3 connection if the server sends New Session Ticket. If the server doesn't, then get_cipher() and get_protocol() return empty strings.
For TLSv1.3 connections in which the server does not send a New Session Ticket, there seems to be only one other source for this information. A line of the form:
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
[Note that "New" would be "Reused" if the connection were created via session resumption.]
The use of this line seems to be okay for extracting the negotiated cipher, but it cannot be used in general to extract the negotiated protocol. The reason is that this line is created as follows:
c = SSL_get_current_cipher(s);
BIO_printf(bio, "%s, Cipher is %s\n",
SSL_CIPHER_get_version(c), SSL_CIPHER_get_name(c));
While the cipher that is printed seems to be the negotiated cipher, the protocol that is printed is "the SSL/TLS protocol version that first defined the cipher." Since TLS 1.3 ciphers may only be used with TLS 1.3, protocol version printed on this line may be accepted as the negotiated protocol if and only if it is "TLSv1.3."
This PR addresses the problem by modifying get_cipher() and get_protocol() to check the "New, ..., Cipher is ..." line if lines from SSL-Session ("Cipher : ...", "Protocol : ...") cannot be found. In the case of get_protocol() the protocol on the "New, ..., Cipher is ..." will be accepted only if the protocol is "TLSv1.3" and the cipher is a TLSv1.3 cipher.
This PR also adds a check for the "New, ..., Cipher is ..." to sclient_connect_successful(). If this line is present, and the protocol and cipher are not "(NONE)", then this is accepted as an indication that the connection was successful even if the "Master-Key" line does not appear. It is not clear whether this extra test is needed, however, as sclient_connect_successful() will not even look at the text in the output of OpenSSL s_client if function's return value is 0, and OpenSSL s_client should return 0 if the connection was successful.
David Cooper
Dirk
Michael Koch
David Cooper