.. after some discussion. As TLS 1.3 is not tested here
any RFC 7919 primes using this protocol will not show
up (they in in run_pfs() though). To avoid misunderstandings
" DH key detected with <= TLS 1.2" is now being printed.
This PR provides an additional fix for the issue raised by #1159. It defines a third option for the degree of processing that should be performed by tls_sockets(): "all+". When "all+" is provided, the processing is exactly the same as for "all" with the exception of the creation of the supported_groups extension. For a TLSv1.3 ClientHello, curves that are not supported by $OPENSSL are omitted from the supported_groups extension rather than offering these curves as the least preferred option.
The "all+" option is used in run_server_defaults() where, unlike with almost every other call to tls_sockets(), a successful connection is of no use unless the response can be decrypted. This is also the case for run_alpn(), and so the call to tls_sockets() was also changed to "all+" there. But, the change has no effect at the moment, since run_alpn() sends a TLSv1.2 ClientHello.
As a result of #276, `run_server_defaults()` makes several attempts to find certificates that a server offers if the ClientHello is for TLSv1.2 and no SNI is offered. However, these tests are unnecessary if it is already known that the server does not support TLSv1.1.
This PR modifies `run_server_defaults()` so that the the TLSv1.1-only tests are skipped if the server is known to not support TLSv1.1.
This PR fixes#1159. If tls_sockets() connects to a server using TLSv1.3, it cannot be assumed that the server's certificate is available, as testssl.sh may not have been able to decrypt the server's response. This can happen, for example, if X25519 was used for the key exchange and `$OPENSSL` does not support X25519.
If the connection was successful, but the certificate could not be obtained, then this PR tries again using `$OPENSSL`. However, since `$OPENSSL` does not support TLSv1.3, this will only work if the server supports TLSv1.2 or earlier.
This commit addresses #179 and implements NNTP via STARTTLS. I did
a few tests and it did work so far.
However the binary support needs to be done. I backported in my
fork of @PeterMosmans tree the section from OpenSSL 1.1.1 -- but
it didn't work, see https://github.com/openssl/openssl/issues/7722.
I just tried to patch it as I suggested and it worked then. My
patch is pushed soon after to https://github.com/drwetter/openssl-1.0.2.bad,
however I'll better wait for the official OPenSSL 1.1.1 patch.
This commit finalizes #1139. It displays the DH groups
in both run_logjam() and run_pfs() in a simlilar manner
(except the FFDHE groups).
A common small function pr_dh() was introduced which prints
out the dh group and in round brackets colored DH bits.
This commit finally fixes#547 and makes XMPP handshakes at least
as fast as the other STARTTLS handshakes.
It utilizes dd to read from the file descriptor. In all tests
I ran so far it didn't cause any problems. There's a potential
problem though that dd might block.
This PR fixes#924 and does some foundation for #547. It's a
somewhat preliminary push of code and further work for #547 is required.
XMPP is now similar programmed as other STARTTLS handshakes with the exception
that it is not line based but stream based. That is still the catch here and
needs to be addressed: STARTTLS protocols like IMAP + SMTP use
starttls_full_read() which reads lines until the line is completely received or
the timeout was encountered.
The new function ``starttls_io()`` however does a wait (fixed value: 1 second)
as there's no lf or terminator.
The XMPP STARTTLS handshakes are now the same as in OpenSSL.
There are redundant functions in this code which will be removed later.
Also at some places a hint for lmtp was missing which was added.
The cipher suites names in the RFCs stem (mostly) from IANA, see
https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-4
This PR corrects that in places visible to the user. For backwards
compatibility the cmd line switches still work as before, but there's
a preference to IANA. The RFC naming is labeled as to be retired
in the future.
In addition to 7d36ba9a2e742e7d6bd2ae135ad94160eaca1d8d which
added new SSLv2 ciphers to the ciphers file this commit adds those
ciphers also to those functions where needed.
Also it does some housekeeping. [[ doesn't require strings on
the right hand side to be quoted, see bash hackers wiki.
PR #1114 brought #1139 a good step forward. This commit adds
a few tweaks to it:
* the groups in run_pfs() are now also italic, except FFDHE groups
* renaming FF groups to DH groups to provide consistency with the
remainder of testssl.sh
* JSON identifier was renamed from DHE_groups to DH_GROUPS
Open points:
* in run_logjam() there's no warning at all regarding e.g. dh512.badssl.com.
Reading the Logjam paper in section 3.5., first couple of paragraphs we
should warn at least against 512 bits here too.
* how do we treat/label 768 bit and 1024 bit in run_logjam() which comes from
unknown groups? Looks like the paper only was concerned about precompuation.
* In run_logjam() is the bit length not colored but in run_pfs() it is.
* Notation: when do we label FF groups / DH parameter ephemeral?
* Code in run_pfs() and run_logjam() can be merged more.
run_logjam() is only related to TLSv1.2 and earlier ciphers. So, run_pfs() should only update $DH_GROUP_OFFERED if a DH group was found using a non-TLSv1.3 cipher.
On the other side, if run_logjam() happened to have been run first, and it found an ffdhe cipher, then there is no need for run_pfs() to test for it.
In run_pfs(), when information about the finite field groups offered is printed, the color used is based on the length of the key. This information should also be conveyed to fileout() in the severity parameter.
For cipher suites that use ephemeral DH groups, run_pfs() currently only displays information about the group(s) used if the server complies with RFC 7919. In the case of TLSv1.3 this is appropriate, since server can only use the values from this RFC and only if they are offered by the client in the supported_groups extension.
For TLSv1.2 and earlier, however, servers are free to use whatever DH group they want, but run_pfs() only provides information about the group the server uses if the server complies with RFC 7919. (The information is, however, provided by run_logjam()). However, so far no servers comply with RFC 7919's requirement to refuse to negotiate a TLS_DHE cipher if the supported groups extension is present, included DH groups, but none that are supported by the server. There is also reason to believe that this will not change: https://www.ietf.org/mail-archive/web/tls/current/msg26378.html.
So, this PR proposes to change the way that run_pfs() searches for DH groups for TLSv1.2 and earlier. (Note that run_pfs() only checks for TLSv1.2 or earlier if the $EXPERIMENTAL flag is set to true.) First, it removes the test to see if the server will reject a ClientHello that only specifies TLS_DHE cipher suites if it includes a supported_groups extension that only specifies an unrecognized DH group. Instead, if the server supports TLS_DHE cipher suites (at TLSv1.2 or earlier) and the $EXPERIMENTAL flag is true, it will try to find out what group(s) the server uses. Second, it will report the group(s) found even if the server uses a group that does not come from RFC 7919.
The result is that if the server supports selecting groups from the supported_groups extension, it will print all of the groups that the server supports. If the server ignores the supported_groups extension and always uses the same group, it will print essentially the same information as is already printed by run_logjam().
One discrepancy, however, is that this code use pr_dh_quality() to determine how good a DH group is, based on the length of the prime, and pr_dh_quality() has differs from run_logjam() in terms of how it rates groups based on the lengths of their primes.
.. otherwise we'll hit too soon the threshold: Logic: by specifying
a timeout a user indicates that there might be a problem.
Also fatal() now supports a hint which is printed in normal
text (to stderr)
Some Cyrus IMAD if configured with SSL_CTX_set_cipher_list(context, "!TLSv1")
and similar respond with a plaintext 'a002 NO Starttls negotiation failed"
when a not-supported protocol is detected, see #1082.
This PR fixes this by detecting (also) this downgrade. As a precaution
It still issues a warning as this is seems a special configuration.
Looking @ pending #1114 two improvements were done:
1) Keep the status of DH group detected (<name> or "Unknown DH group")
as well as the bit length
2) move the detection to a separate function get_common_prime()
There's still room for improvements when run_pfs() will take
over a part.
Also double code (my bad) from run_logjam() was move to a separate function.
As #1146 noted some installations miss hexdump. Better practice
is to check before what's needed albeit the error message when
a binary is missing does give the user a hint.
Currently the client simulation is based on the handshake data
from SSLlabs which is purely focussed on HTTP -- as SSLlabs does
HTTP only.
In #540 there was a PR addressing the fact that the data is not
what is claims to be -- the handshake of Android 7 seems to be
Chrome for Android and not Android itself.
This PR tries at least to modify the headline for client simulations.
This PR addresses the remaining TCP fragmentation by piping the line buffered
internal print through cat, see also #1130.
It extends 1b52834 which was the same doing for Linux and
OpenBSD.
This PR also consolidates the last remaining low level socket calls
in client_simulation_sockets() into socksend_clienthello().
An negative performance effect is barely measurable.
It also does a check whether the fd 5 is taken by a tty as
I see this while writing the commit message ;-). We might
want to make that line better instead of just echoing. :-)
At the beginning of run_server_preference(), if the attempt to connect to the server is unsuccessful, a message is printed listing all of the ciphers in $list_fwd and $tls13_list_fwd:
no matching cipher in this list found (pls report this): DES-CBC3-SHA:RC4-MD5:DES-CBC-SHA:RC4-SHA:AES128-SHA:AES128-SHA256:AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:ECDH-RSA-DES-CBC3-SHA:ECDH-RSA-AES128-SHA:ECDH-RSA-AES256-SHA:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-DSS-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:AES256-SHA256:ECDHE-RSA-DES-CBC3-SHA:ECDHE-RSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-SHA256:ADH-AES256-GCM-SHA384:AECDH-AES128-SHA:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-AES128-SHA:TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256
This message can be misleading. I tested a server that only supported TLSv1.3 using the provided OpenSSL 1.0.2-chacha. The server supported TLS_AES_256_GCM_SHA384, but OpenSSL didn't. However, the message implies that the server does not support TLS_AES_256_GCM_SHA384.
This PR changes the message (and the one included in CSV/JSON output) to only list those ciphers in $list_fwd and $tls13_list_fwd that are actually supported by $OPENSSL.
Note that even with this PR, some ciphers are listed that aren't really supported by $OPENSSL, since the `-s` option isn't used. But, that is #663.
* Put all low level socket related functions close to each other
* removed socksend2 as it was not used and outdated looking forward
* socksend_sslv2_clienthello() renamed to socksend_clienthello() as
it wasn't particular SSLv2 related
* removed the low level socket calls from socksend_tls_clienthello()
and called socksend_clienthello() instead
* renamed socksend_tls_clienthello() to prepare_tls_clienthello()
as it is not a low level function anymore
This PR is also based on #1139, but it addresses ECDH keys rather than DH keys. When run_pfs() prints the list of elliptic curves offered, it colors each curve according to its quality (based on key length). However, the severity level used when the list is sent to fileout() is always "INFO". This PR changes the call to fileout() to make the severity level be based on the quality of the shortest curve that the server offers.
This commit adds LMTP to the STARTTLS protocols
supported. It requires an openssl version which
supports this which is either openssl 1.1.1
or a backported version 1.0.2 (binary is in
process).
A check is in place whetrher the binary supports
this.
Furthermore some framework additions were made for
further STARTTLS protocols like IRC and NNTP.
For XMPP servers, when extracting the SRV-ID and XmppAddr names from the subjectAltName extension, need to take into account that the subjectAltName extension may be marked as critical, in which case there will be the DER encoding of TRUE (0101FF) between the DER encoding of the subjectAltName extension's OID (0603551D11) and the tag for OCTET STRING (04).
This PR is an attempt to address #1097. I have only been able to test it against jabber.topf.org and against locally created test certificates, so it needs more testing.
The main issue that this addresses is that testssl.sh currently checks against the wrong name for XMPP servers. According to RFC 6120, Section 13.7.2.1:
o The initiating entity sets the source domain of its reference
identifiers to the 'to' address it communicates in the initial
stream header; i.e., this is the identity it expects the receiving
entity to provide in a PKIX certificate.
So, if the --xmpphost option is provided, then the name provided in that option should be compared against the name in the certificate rather than the host name.
compare_server_name_to_cert() currently takes the server name to look for in the certificate as an parameter, but every call to compare_server_name_to_cert() uses $NODE as the argument. So, this PR removes the parameter and sets $servername to either $XMPP_HOST or $NODE as appropriate. This small change alone should fix the problem for most XMPP servers since the server's name SHOULD appear in the certificate encoded as a DNS name. That is the case for the one server I could test - jabber.topf.org.
The majority of the code in this PR is to address the other possibilities noted in RFC 6120, Section 13.7.1.2.1. This section notes that an XMPP server's identity name also appear in the subjectAltName extension as an otherName, either an SRV-ID or an XmppAddr identifier. Unfortunately, OpenSSL's certificate printer does not support otherName and just prints "othername:<unsupported>". So, this PR includes code to manually extract any SRV-ID or XmppAddr names from the certificate. This involves parsing the DER encoding of the certificate to look for the subjectAltName extension, looping through all of the names in the extension, and pulling out the names of these two name forms. This code is only run if the server is an XMPP server and the certificate does not have a matching DNS name. So, this code will rarely be executed.
This PR addresses one other issue. There is code in certificate_info() to set the variables $has_dns_sans and $has_dns_sans_nosni. These variables are needed to address the following requirement:
# Find out if the subjectAltName extension is present and contains
# a DNS name, since Section 6.3 of RFC 6125 says:
# Security Warning: A client MUST NOT seek a match for a reference
# identifier of CN-ID if the presented identifiers include a DNS-ID,
# SRV-ID, URI-ID, or any application-specific identifier types
# supported by the client.
While it is relatively easy to determine whether a certificate includes a DNS name in the subjectAltName extension, as noted above, it is not easy to determine whether it has an SRV-ID or an XmppAddr. So, this PR leverages the work compare_server_name_to_cert() does in parsing the subjectAltName extension by having compare_server_name_to_cert() set a global variable indicating whether the certificate has a subjectAltName extension with a relevant name form (DNS, SRV-ID, or XmppAddr for XMPP, or DNS for other servers). $has_dns_sans and $has_dns_sans_nosni are then just set to the value of this global variable.
Instead of checking via uname for Linux this commit does a check
whether the outcome for an external printf is what is expected. This
makes it more compatible e.g. with OpenBSD which surprisingly works
similar like the GNU counterpart.
Also it checks all external printfs installed wther it's the
"right one" to use. Previously it stopped just at the first one
and if this was "wrong", bash's printf was used.
Linux bash internal printf shortened the string
when using len2twobytes() with 3 chars, FreeBSD
e.g. did not.
It worked under both OS though when piping to
the socket with printf.
This commit makes sure that always 2+2 chars
are returned when a 3 char number is supplied.
Kudos @dcooper16
