Commit Graph

3314 Commits

Author SHA1 Message Date
Dirk Wetter c0d04f7a84
Merge pull request #1198 from dcooper16/run_server_defaults_ssl2_only
Fix run_server_defaults() for SSLv2-only servers
2019-02-13 13:25:51 +01:00
David Cooper 1cf796fd36
Fix run_server_defaults() for SSLv2-only servers
This PR makes a few improvements to run_server_defaults() when run on an SSLv2-only server.

First, it uses sslv2_sockets() to test the server rather than $OPENSSL, so that it will work even if $OPENSSL does not support SSLv2.

Second, it changes run_server_defaults() to only call get_server_certificate() once if $OPTIMAL_PROTO is -ssl2, since calling more than once is a waste -  SSLv2 only supports ciphers that use RSA key exchange.

Finally, as some code assumes that $TEMPDIR/intermediatecerts.pem will exist, even if it is empty, this PR changes a couple of places that delete $TEMPDIR/intermediatecerts.pem to instead make the file empty.
2019-02-12 15:15:18 -05:00
David Cooper 90d36262ea
Fix run_server_preference() for SSLv2 only server
When run_server_preference() is run on a server that only supports SSLv2 it incorrectly reports that the server has a cipher order. The reason for this is that $list_fwd and $list_reverse only include one SSLv2 cipher.

In SSLv2 the server sends a list of all ciphers it supports in common with the client and the client chooses which cipher to use. As a result, the server cannot enforce a cipher order for SSLv2.

So, this PR fixes the problem in run_server_preference() by skipping the test for whether the server enforces a cipher order if $OPTIMAL_PROTO is -ssl2 and simply declares that the server does not enforce a cipher order.

Note that this PR is somewhat dependent on #1194, as #1194 needs to be applied in order for $OPTIMAL_PROTO to be set to -ssl2 when testing an SSLv2-only server.
2019-02-12 13:44:55 -05:00
David Cooper 8fdb388dc3
Reorganize run_cipher_per_proto()
This PR reorganizes run_cipher_per_proto(). Currently run_cipher_per_proto() runs a for loop, which loops over each protocol and prints the set of supported ciphers for each protocol. This PR simply places the body of the for loop in a separate function from the loop itself. This allows the body of the loop to be called for just a single protocol.

While this PR does not change the way that testssl.sh functions, it would allow for a future change in which run_server_preferences() called cipher_pref_check() for protocols in which the server enforces a cipher order and calls ciphers_by_strength() for protocols in which the server does not enforce a cipher order.
2019-02-12 12:50:24 -05:00
David Cooper 2a13643bb1
Reorganize cipher_pref_check()
This PR reorganizes cipher_pref_check(). Currently, cipher_pref_check() runs a for loop, which loops over each protocol and prints the set of supported ciphers for each protocol. This PR simply places the body of the for loop in a separate function from the loop itself. This allows cipher_pref_check() to be called for just a single protocol rather than for all protocols. Another PR will make a similar change to run_cipher_per_proto().

The reason for this change is that cipher_pref_check() was only intended to be used in cases in which the server enforces a cipher preference order. Some servers, however, enforce an order for some protocols, but not for others. The change in this PR will make it possible in the future to call cipher_pref_check() only for protocols in which the server enforces a cipher order.
2019-02-12 12:43:57 -05:00
David Cooper dd8ee06ca5
Fix determine_optimal_proto()
This PR fixes two bugs in determine_optimal_proto().

First, sslv2_sockets() returns 3 if the connection was successful.

Second, if all connection attempts using tls_sockets() were unsuccessful, it is possible that $TEMPDIR/$NODEIP.parse_tls_serverhello.txt will not exist, so copying it or grepping it will lead to an error. Checking that $proto is not 22 will fix this as $proto will be empty is $OPENSSL s_client was used and it will be 00, 01, 02, 03, or 04 if tls_sockets() was used and the connection was successful with some protocol higher than SSLv2.
2019-02-12 11:27:49 -05:00
David Cooper 7a62654aef
Fix get_cipher()
This PR fixes a bug in get_cipher() - one that also appears in sclient_connect_successful().

The code currently assumes that cipher names contain only uppercase letters and numbers. However, ciphers that do not provide authentication include "anon" in the name, which is written in lowercase.

This PR fixes the problem by allowing lowercase letters to appear in cipher names (except in the first portion of the name).

Note that no change was made to similar code in get_protocol(), since the line in get_protocol() only matches TLSv1.3 ciphers, which do not contain any lowercase letters.
2019-02-12 10:43:13 -05:00
Dirk Wetter 5d1109a582 Workarounds for missing curves in OpenSSL
In case where the OpenSSL version used cannot successfully do openssl s_client
connects there are a few problems, see #1087.

This PR partly addresses them by
* changing the logic of HTTP header failure: we don't terminate anymore but
  continue with a warning message
* we try to find out what the reason was: If it is a missing curve we signal
  it back to the user
* we keep track in a global variable KNOWN_OSSL_PROB. It's not being used yet
  on all connects as it has not been decided whether we do a connect despite
  we know if there's a problem or rather not.
* Give hints to the user for resumption tests, secure renegotiation, CRIME and BREACH.
  For the latter --assume-http needs to be supplied for any output.

Also: for finding the OPTIMAL_PROTO now (unless --ssl-native is being used)
sockets are the default which removes in cases where an openssl s_client
connect fails, the initial message 'doesn't seem to be a TLS/SSL enabled server'
and prompt 'Really proceed ? ("yes" to continue)'. For STARTTLS this needs
to be done as well.

Here a minor bug was fixed: when openssl s_client connect in determine_optimal_proto()
succeeded without a protocol supplied, OPTIMAL_PROTO wasn't set. A statement was
added but now it is only being used when --ssl-native was supplied.

Leftover for this workaround is to find out why the number of certificate retrieved is
zero in those cases, despite the fact that there's a valid 'host_certificate.pem' from
tls_socket() calls.  Thus still run_server_defaults() stops after 'TLS clock skew'
as certificate_info() is not being called in run_server_defaults(). For now in
those cases 'Problem: Host certificate found but we can't continue with "server defaults"'
is being printed.

In general for the future it would be great if we could e.g. retrieve the header over
TLS sockets.
2019-02-11 20:24:41 +01:00
Dirk Wetter 691ca28bb9
Merge pull request #1192 from dcooper16/fix_second_clienthello
Fix TLSv1.3 second ClientHello
2019-02-09 13:44:26 +01:00
David Cooper a26aae381c
Fix TLSv1.3 second ClientHello
This PR fixes two problems with modify_clienthello().

First, the function was incorrectly using the variable $key_share instead of $new_key_share. Since $key_share is defined when modify_clienthello() is called from resend_if_hello_retry_request(), but not when it is called from client_simulation_sockets(), this bug does not seem to result in incorrect behavior, but it should still be fixed.

Second, when this function is used to create a second ClientHello in response to a HelloRetryRequest, it removes the key_share extension from the original ClientHello and then appends the new key_share extension at the end. According to https://mailarchive.ietf.org/arch/msg/tls/8ZKCyamcYFaV90h6nf4MUnSPkEE, however, extensions must appear in the second ClientHello in the same order in which they appeared in the first ClientHello.

I am not aware of any servers that will actually complain if the extensions in the second ClientHello do not appear in the same order as in the first ClientHello, bug this fix helps to ensure that the ClientHello messages testssl.sh sends are in compliance with the standard.
2019-02-08 15:40:28 -05:00
Dirk Wetter b5dbe73a7e Minor output tuning for #1188
Prefer quoted name, see https://datatracker.ietf.org/liaison/1616/.
Also added visibility info for now.
2019-02-08 14:11:52 +01:00
Dirk Wetter 28dd530c45 Merge branch '2.9dev' of github.com:/drwetter/testssl.sh into 2.9dev 2019-02-07 10:42:16 +01:00
Dirk Wetter 98505590b3
Merge pull request #1188 from dcooper16/eTLS
WIP: Detecting "eTLS" 1.3 (Fix #1185)
2019-02-06 16:59:03 +01:00
David Cooper 30da1cdd72
WIP: Detecting "eTLS" 1.3 (Fix #1185)
This PR is an attempt at addressing #1185. According to https://www.etsi.org/deliver/etsi_ts/103500_103599/10352303/01.01.01_60/ts_10352303v010101p.pdf, if eTLS is in use, then the certificate should contain a subjectAltName extension with one or more "names" containing "visibility information." The "visibility information" is encoded as an otherName with a type-id of 0.4.0.3523.3.1 and a value of

   VisibilityInformation ::= SEQUENCE {
        fingerprint         OCTET STRING (SIZE(10)),
        accessDescription   UTF8String }

The etsi_etls_visibility_info() function determines whether the certificate includes an "visibility information," and, if it does, extracts the fingerprints and access descriptions.

This PR is a work-in-progress for two reasons. First, it has not been tested against any real certificates that contain "visibility information." Testing against real certificates would be helpful to verify that the parsing of the certificate is correct.

Second, the presentation of the visibility information (both in the printed text and in what is sent to fileout()) may need improvement. Having seen no examples, it is not clear what the contents of accessDescription can be expected to look like. The document says that the contents will be "human-readable text," but it is not clear whether the description will be relatively short or very long.
2019-01-31 11:51:16 -05:00
Dirk 7f8a0f2c8b Readd final 00,ff to high_ciphers to previous commit 2019-01-28 21:23:51 +01:00
Dirk ed8f26b2f6 Improve standard cipher lists section
To finalize #1157 following was done

* move 3DES one line above
* put 128 bit CBC ciphers (ARIA, Camellie and AES) SEED and IDEA into 128Bit
* the remaining 256 bit ciphers NOT supporting AEAD Mac into high

Also 128 bit ciphers are getting a small complaint (LOW, yellow) if
available instead of red (for SEED and IDEA before)

To ease the (future) output rated_output() is included, but it's not being used
yet.

Also often I have docker hosts for testing. If I use them while the external
network is down, I still experience DNS timeouts. I added for dig
timeout values which proved to be reasonable in my tests with and
without network.

Also if an IPv4 or an IPv6 address was supplied testssl.sh doesn't
do (futile) DNS lookups anymore.
2019-01-28 18:31:57 +01:00
Dirk Wetter e02b313394 Test code for a HTTP GET request over sockets
It doesn't run per default and it doens't work yet. Just to share the idea.
2019-01-26 14:18:04 +01:00
Dirk Wetter 7de93e26dc Introducing HAS_X448 and HAS_X25519
... to enable checks whether a curve has been detected by sockets
won't be detected and/or makes problems with remaining openssl
s_client + other calls

Related to #1087
2019-01-26 12:50:16 +01:00
Dirk Wetter 966f414a2b Multiple IP scan fix (no --file) + single cipher scan works with multiple IPs
This PR addresses the problem (#1037) that if a hostname resolves to multiple IP
addresses only the first one is being scanned as this IP isn't reachable and the
scan terminated here and didn't continue with the 2nd, 3rd etc.

Same applied to scans with --mx.

This based on the global MULTIPLE_CHECKS which is set to true whenever
such a scan is started.

One minor point: Also if the last IP isn't reachable the output will say
"proceeding with next:". I guess that should be clear looking at the (different outputs).

This PR also fixes a scan problem with a single cipher/pattern (-x/--single-cipher):
previously where only one IP was scanned.

Furthermore some redundant quotes were removed.
2019-01-24 17:57:29 +01:00
Dirk Wetter ef6deb037e ~Empty files when scan problems encountered
This commit addresses #934. It adds a line in the log/csv output
and a json object named "scanProblem" when either the function
fatal() is being called and it logs the reason of the fatal error.
2019-01-21 12:19:56 +01:00
Dirk 7ffd46447a Add new openssl helper binaries for Linux and FreeBSD
.. based on https://github.com:/drwetter/openssl-1.0.2.bad

plus the patch fedora-dirk-ipv6.patch applied.

In addition to Peter Mosman's branch the brauch and as a result the binaries
contain following additional STARTTLS features: LMTP, NNTP, IRC .

See also #741, #559, #1093, #179
2019-01-18 19:53:19 +01:00
Dirk 5c74af3694 Change in cipher categories / run_cipherlists()
As noted in #1157 the logic of different cipher suites put to categories
needed an improvement.

This commit addresses it by moving first RC2 and RC4 cipher suites to
the low category.

More to follow.
2019-01-17 13:05:17 +01:00
Dirk 6926bb330c Remove some redundant double quotes
.. in sub_cipherlists() and neat_list()
2019-01-10 14:40:02 +01:00
Dirk 6d2b42b38d Fix regression (TLS 1.3) in server preference test
.. see #1179
2019-01-10 14:17:42 +01:00
Dirk 53bd3bf736 Server preference for TLS 1.3
This commit fixes #1163 which lead to the misleading output when
a TLS 1.3 enabled server had no preferences for the TLS 1.3 ciphers
but for anything below (like currently for testssl.NET).

The TLS 1.3 handshake in sockets plus the following openssl handshake
was moved to the top in run_server_preference() so that it can be better
determined whether TLS 1.3 is available. If this section's outcome is
TLS 1.3 is negotiated a single TLS 1.3 handshake with 5 ciphers only is
done forward and reverse. The resulting ciphers are later on compared
whether there's a cipher order for TLS 1.3.

Basically this section should be redone, so that all openssl handshakes
are replaced by sockets. As this would consume more time as it appears
reasonable at this point of time, this was not done yet. A starting point
for this would be tls13_list_fwd + reverse. After release of 3.0 90%
of the code will be replaced anyway.

DHE-RSA-SEED-SHA and SEED-SHA was added to the reverse and forward lists
as some old openssl versions + apache use it.

Also:

Googles ALPN_PROTO grpc-exp was added (to be reconsidered at some certain point)

Some redundant quotes in double square brackets were removed.

All "do_*" variables are now in quotes when tested w if or [[
2019-01-09 15:33:15 +01:00
Dirk e29b1f40e6 Improve HTML-Formatting, minor additions
The HTML manual is now post processed through tidy
which removes the problem of ">" not HTML encoded.

--color 0 is now explicitly mentioned to avoid escaped codes in the
output.

Minor changes wrt certificate stores
2019-01-08 13:56:55 +01:00
Dirk 0b98b7c735 Small clarification to --help and <fname> 2018-12-19 11:45:02 +01:00
Dirk 75a9c021e5 Add -iL as a n alternative to --file 2018-12-17 21:09:12 +01:00
Dirk d10f66a4c5 Minor additions
1) add --add-ca option and remove eother references to ADDITIONAL_CA_FILES
2) add a paragraph 'TUNING OPTIONS'
3) add -iL as a n alternative to --file
2018-12-17 21:04:47 +01:00
Dirk e739b54664 Updated to version 3.0rcX
Reflect we're at 3.0rcX, 2.9.5 is past, 2.8 not supported

Removed features from 2.9.5. Added all(?-->David?) features implemented
in 2.9dev.

Mention docker image

Clarify when testssl.sh should be mentioned (license).
2018-12-14 14:47:41 +01:00
Dirk f708e1420e Updated Trust Stores, Java added
This is an update of the root certificate stores. Date from each store
is from yesterday.

Description update.

Also the Java certificate store was added. Previously Java was omitted
as it appeared not to be complete. I tested successfully this store.
2018-12-14 10:00:23 +01:00
Dirk 1416ff620b Major update, review
Review: grammar, spelling.  Errorneous and obsolete description.
        Some items reordered.

Updated: to reflect the current capabilities.

Moreover: (Almost) complete the tuning variables section.
2018-12-13 18:07:20 +01:00
Dirk 4f920a389a Don't output warnings in JSON in parent when mass testing
This addresses #1169: When using JSON as output format when mass testing
AND we have a non-fatal condition when e.g. openssl lacks support for
something it led to an invalid JSON as the warning was put into file w/o
a trailing comma.

The commit removes the warning to be put into the output. We still have the
message on screen + in HTML which is not as optimal as it could be.

Also I did some cleanups related to redundant double quotes I stumbled over while
fixing this.
2018-12-11 10:03:58 +01:00
Dirk ab7ca281c0 Minor polishing 2018-12-07 14:35:49 +01:00
Dirk 6fe5adbbc3 Improved connection failure conditions
As a kind of a pre-warning this commit allows the n-1 connection problem to
give feedback on the screen (that wasn't working before).

Also the message on the screen is now more clear and the manpage
gives better advice.

Related to #1172
2018-12-05 16:09:36 +01:00
Dirk Wetter 11cf06d6e9
Merge pull request #1171 from dcooper16/fix1165
Fix #1165
2018-12-04 21:42:20 +01:00
Dirk Wetter 78722ef982
Merge pull request #1170 from dcooper16/fix990
Fix #990
2018-12-04 21:36:49 +01:00
David Cooper d3c29f24e4
Fix #1165
This PR fixes #1165 by changing resend_if_hello_retry_request() to modify the initial ClientHello rather than having it call prepare_tls_clienthello() to try to generate a new ClientHello that is almost the same as the first. The modification is done using a revised version of create_client_simulation_tls_clienthello(), which is now renamed as modify_clienthello().

Since prepare_tls_clienthello() is no longer used to create a second ClientHello message, argument 7 to that function is no longer needed.
2018-12-04 14:07:39 -05:00
David Cooper 93da0919a9
Fix #990
There is at least one extension that will fail on a TLSv1.3 ClientHello if the psk_key_exchange_modes extension is not present (see #990). The PR adds the extension to TLSv1.3 ClientHello messages. OpenSSL, Firefox, and Chrome all include this extension in their ClientHello messages, so including it is unlikely to cause problems for any servers.
2018-12-04 12:51:46 -05:00
Dirk e9c5435c0a Banners for NNTP added
... found during research for STARTTLS NNTP last week
2018-12-04 11:30:01 +01:00
Dirk c78c293bf8 Fix vim, err... typo ;-) 2018-12-01 21:18:17 +01:00
Dirk Wetter ab55c26424
Merge pull request #1168 from dcooper16/fix_typos
Fix typos
2018-11-30 18:38:31 +01:00
David Cooper 42687658ea
Fix typos
The primary purpose of this PR is to fix the typo on line 14157, but it also corrects a number of typos that appear in comments.
2018-11-30 10:54:30 -05:00
Dirk 22a11b1e75 Bump version # 2018-11-30 10:02:39 +01:00
Dirk 7d40041652 Re-add RFC 7919 primes into run_logjam()
.. after some discussion. As TLS 1.3 is not tested here
any RFC 7919 primes using this protocol will not show
up (they in in run_pfs() though). To avoid misunderstandings
" DH key detected with <= TLS 1.2" is now being printed.
2018-11-30 08:59:55 +01:00
Dirk Wetter 4e2bd0ef2f
Merge pull request #1166 from dcooper16/dont_test_unsupported_tls11
Don't run TLSv1.1 tests if server doesn't support TLSv1.1
2018-11-29 10:12:08 +01:00
Dirk Wetter 7a6ec6b8a2
Merge pull request #1164 from dcooper16/fix1159
Fix #1159
2018-11-29 10:02:26 +01:00
Dirk Wetter 51df09bbcf
Merge pull request #1167 from dcooper16/fix1159_part2
Fix #1159 part 2
2018-11-29 10:01:47 +01:00
David Cooper f723ec97a7
Fix #1159 part 2
This PR provides an additional fix for the issue raised by #1159. It defines a third option for the degree of processing that should be performed by tls_sockets(): "all+". When "all+" is provided, the processing is exactly the same as for "all" with the exception of the creation of the supported_groups extension. For a TLSv1.3 ClientHello, curves that are not supported by $OPENSSL are omitted from the supported_groups extension rather than offering these curves as the least preferred option.

The "all+" option is used in run_server_defaults() where, unlike with almost every other call to tls_sockets(), a successful connection is of no use unless the response can be decrypted. This is also the case for run_alpn(), and so the call to tls_sockets() was also changed to "all+" there. But, the change has no effect at the moment, since run_alpn() sends a TLSv1.2 ClientHello.
2018-11-28 14:58:17 -05:00
David Cooper 9a47b379d6
Don't run TLSv1.1 tests if server doesn't support TLSv1.1
As a result of #276, `run_server_defaults()` makes several attempts to find certificates that a server offers if the ClientHello is for TLSv1.2 and no SNI is offered. However, these tests are unnecessary if it is already known that the server does not support TLSv1.1.

This PR modifies `run_server_defaults()` so that the the TLSv1.1-only tests are skipped if the server is known to not support TLSv1.1.
2018-11-28 12:52:15 -05:00