run_prototest_openssl() currently checks only stdout for the string "no cipher list", which is an indication that the server supports SSLv2, but no ciphers for that protocol. However, the output that includes "no cipher list" is sent to stderr.
This PR fixes a minor problem with run_protocols() in "--ssl-native" mode if $OPENSSL does not support TLS 1.3. Currently, the warning message that $OPENSSL does not support a protocol is printed when run_prototest_openssl() is called. This causes a problem for the output if $OPENSSL does not support TLS 1.3, since the run_prototest_openssl() is called before the results for TLS 1.2 are printed. The result is something like this:
SSLv2 not offered (OK)
SSLv3 not offered (OK)
TLS 1 offered (deprecated)
TLS 1.1 offered (deprecated)
Local problem: /home/cooper/Desktop/testssl.sh/bin/openssl.Linux.x86_64 doesn't support "s_client -tls1_3"
TLS 1.2 offered (OK)
TLS 1.3 NPN/SPDY not offered
ALPN/HTTP2 http/1.1 (offered)
When processing a command line for parallel mass testing, create_mass_testing_cmdline() did not account for the newer shortened versions of the output file options: -oj, -oJ, -oC, and -oH. This caused the command line for the child processes to be incorrect.
If a server offers TLS 1.3 only and the cipher order is server side this commit changes the severity level to INFO.
Also it changes nope to no in two places
This commit addresses the comments in #1205. If a server only supports TLS 1.3, then it is not considered an issue if the server does not enforce a cipher order. However, if the server does not support a cipher order for TLS 1.2 and below, then that is an issue, even if the server does support a cipher order for TLS 1.3.
This PR is an attempt to fix#1163 by running separate tests for a server cipher order preference to TLSv1.3 and for SSLv3 - TLSv1.2.
If the server supports TLSv1.3, then a test is performed to determine whether the server enforces a cipher order to TLSv1.3. A separate test is performed for SSLv3 - TLSv1.2 unless it is known that the server does not support any of these protocols.
If the server enforces a cipher order for SSLv3 - TLSv1.2, but not for TLSv1.3, then cipher_pref_check() is not called for TLSv1.3, since cipher_pref_check() is intended to show the cipher order that the server enforces. As TLSv1.3 will be the negotiated protocol if it is supported, the negotiated cipher for TLSv1.3 will already be presented.
This PR still has one major flaw, which may create a problem when testing a TLSv1.3-only server. If run_protocols() is run before run_server_preference(), then everything will be okay, as run_server_preference() will be able to determine that SSLv3 - TLSv1.2 are not supported. However, if run_server_preference() is run by itself, run_server_preference() will not know that SSLv3 - TLSv1.2 are not supported and so it will try to determine whether the server enforces a cipher preference order for these protocols. The attempt to connect to the server will fail, but at the moment run_server_preference() doesn't know whether the failure is because the server does not support SSLv3 - TLSv1.2 or because the server supports at least one of these protocols, but does not support any ciphers in $list_fwd. At the moment, run_server_preference() incorrectly flags an error.
One option would be to perform additional tests against the server in this case to determine the reason for the connection failure. Another option would be to have some code that is always run earlier, such as determine_optimal_proto(), test whether a server that supports TLSv1.3 supports any earlier protocols (SSLv3 - TLSv1.2).
In run_drown(), $jsonID is set to "DROWN" and most calls to fileout() are of the form
fileout "$jsonID" ...
However, one call is written as
fileout "DROWN" ...
This PR changes this one call to be consistent with the others. This does not change the functionality of the program.
At the moment, $do_starttls is initialized to true in initialize_globals() and then it is set to true again in parse_cmd_line() if the --starttls command line option is used. Presumably the intention was to set $do_starttls to false in initialize_globals().
This PR fixes a minor bug in parse_tls_serverhello(). In some cases the server's entire response is not retrieved. In these cases, it is possible that the response from the server ends with a portion of a handshake message.
The loop at the beginning of parse_tls_serverhello() extracts the various handshake and alert messages from the server's response. If it gets to the end of the response, and what is at the end is not a complete message, it should just ignore that fragment and break out of the loop. At the moment, however, parse_tls_serverhello() just continues in the loop rather than breaking out. This has not been a problem up to now, since $msg_len is usually set to a positive value from a previous iteration of the loop, which causes the loop to end.
In the case of the server identified in #1353, however, $msg_len is 0 and so the continue rather than break results in an endless loop.
Add another pattern because the SEDs tested so far do not seem to be fine with header containing x0d x0a (CRLF) -- which is the usual case. So we also trigger on any sign on a single line which is not alphanumeric (plus _)
See #1351
This PR fixes a bug in modify_clienthello() that occurs when client simulation is being performed, the ClientHello contain an SNI extension, and $SNI is empty. In the case, modify_clienthello() should just skip over the SNI extension and not include one in the modified ClientHello. However, the code currently only skips over the 2-byte extension type. The result being that the remainder of the extension is included in the modified ClientHello. This PR fixes the problem by ensuring the $offset is advanced whether or not $SNI is empty.
PR #1336 included logic to pre-test the server side with sockets
and/or with openssl. However when the user supplied --ssl-native
sockets were never tested before. As a result ALL_FAILED_SOCKETS
was still true, so that the final eif statement complaint erroneously
that sockets didn't work but openssl does.
Also Travis complaint.
This PR fixes it by checking SSL_NATIVE to the final part of the
if statement.
One could also test sockets before and then set ALL_FAILED_SOCKETS
appropriately but that would only make sense if the socket methods
like run_robot() or run_heartbleed() would check ALL_FAILED_SOCKETS
first.
At the moment I went for this as it is easier and the case that sockets
aren't working but openssl does seems not very likely.
There are a few places where testssl.sh sends a TLS 1.2 (or TLS 1.3) ClientHello and expects the server to respond with a ServerHello as long as it supports TLS 1.2 (or TLS 1.3) or earlier.
run_protocols() performs a fairly thorough check for a server's ability to handle version negotiation, but the problem may also be caught by determine_optimal_sockets_params(), if the server rejects a TLS 1.2 ClientHello even though it supports some earlier protocol version.
In the future, we could try to make use of $OPTIMAL_SOCKETS_PROTO in order to make testssl.sh work a bit better with servers (if any still exist) that don't handle version negotiation correctly. At the moment, though, this PR just prints a warning to the user that the server is buggy, and that this may lead to problems in the scan. It doesn't call fileout() to add anything to the JSON/CSV output, since run_protocols() should already be doing that.
This PR modifies run_protocols() to use the information collected by determine_optimal_sockets_params(). If it has already been determined that a protocol is supported, then no test is run. run_protocols() will still run a test for a protocol even if it has been determined that the server does not support that protocol. The reason for running the test is to verify that the server handles version negotiation correctly. This could be a TLSv1 server that rejects a TLSv1.2 or TLSv1.3 ClientHello, or it could happen in the opposite direction. At one point there was a server that would respond to an SSLv3 ClientHello with a TLSv1.2 ServerHello.
This PR required a couple of changes to determine_optimal_sockets_params() so that additional information could be passed to run_protocols(). If the server supports TLS 1.3, then run_protocols() needs to know which version (RFC 8446, draft 28, draft 27, etc.) rather than just that TLS 1.3 is supported. If the server supports TLS 1.2, but not TLS 1.3, then run_protocols() needs to know about at least one TLS 1.2 cipher that the server supports so that it can form a TLS 1.3 ClientHello that has no more than 128 ciphers and that should result in the server returning a TLS 1.2 ServerHello.
In a PR that I'm developing to to use the results of determine_optimal_sockets_params() in run_protocols() I add specific versions of TLS 1.3 to PROTOS_OFFERED (e.g., tls1_3_rfc8446, tsl1_3_draft28). If that PR is accepted, then the current check for TLS 1.3-only will no longer work. So, this commit changes the way that the check for TLS 1.3-only is performed in order to avoid problems if the other PR is merged.
This PR takes advantage of the testing done by determine_optimal_sockets_params() in order to simplify determine_sizelimitbug().
By the time that determine_sizelimitbug() is called, determine_optimal_sockets_params() has already determined whether TLSv1.2 ClientHello with 128 ciphers (including 00FF) sent by tls_sockets() works, and it has set TLS12_CIPHER to a list of exactly 128 ciphers (including 00FF) that works with the server. So, determine_sizelimitbug() doesn't have to check whether the server supports TLSv1.2 and no longer needs to send tests using 127 or 128 ciphers. determine_sizelimitbug() can just perform one test with 129 ciphers, if the server supports TLSv1.2, and use the results to set $SERVER_SIZE_LIMIT_BUG.
This PR reverts determine_optimal_proto() to use OpenSSL again rather than tls_sockets().
The primary reason for this is that the primary purpose of determine_optimal_proto() is to set OPTIMAL_PROTO, which is only used with $OPENSSL s_client. So, the best way to determine what works best on the $OPENSSL s_client command line is to use $OPENSSL s_client.
In most cases, determine_optimal_proto_sockets_helper() would set OPTIMAL_PROTO to an acceptable value, but it might not always do so. For example, suppose that a server
* supports different cipher suites with different protocols,
* supports TLSv1.2, but only with cipher suites not supported by $OPENSSL, but
* supports TLSv1.1 with at least one cipher suite supported by $OPENSSL.
In the above case, determine_optimal_proto_sockets_helper() would set OPTIMAL_PROTO to "-tls1_2", but testing using $OPENSSL would result in OPTIMAL_PROTO being set to "-tls1_1".
Using $OPENSSL for determine_optimal_proto() also allows for edge cases to be detected earlier:
* If the server only supports TLSv1.3, and $OPENSSL does not support TLSv1.3, then the code in this PR will detect that (rather than waiting until run_protocols() is executed).
* The code in this PR can also detect if the server only supports SSLv3 (and possibly also SSLv2), but $OPENSSL does not support SSLv3.
* This code can also detect the (rare) case in which connections using $OPENSSL succeed, but connections using tls_sockets() fail.
[Note also that in the current code, if $all_failed is true, then a message may be printed that $OPENSSL is not IPv6 aware, even if testing was performed using tls_sockets() rather than $OPENSSL.]
This PR fixes an issue with sub_session_resumption() when using OpenSSL 1.1.1.
As noted in #1335, some servers will return a session ticket for TLSv1.2, but not for TLSv1.3.
OpenSSL 1.1.1 does not support the "-no_ssl2" option, and so when using OpenSSL 1.1.1 sub_session_resumption() adds $OPTIMAL_PROTO to the $OPENSSL s_client command line. When determine_optimal_proto_sockets_helper() is called, $OPTIMAL_PROTO will generally be set to "-tls1_2" (or "-tls1_1" or "-tls1") unless the server is a TLSv1.3-only server. As a result sub_session_resumption() will specify that same protocol on the command line if OpenSSL 1.1.1 is being used.
If "--ssl-native" is used, however, then determine_optimal_proto() will set $OPTIMAL_PROTO to "-tls1_3" if the server supports TLSv1.3 (and doesn't use STARTTLS). Similarly, if the version of determine_optimal_proto() in #1336 is used, then $OPTIMAL_PROTO will usually be empty. In either case, sub_session_resumption() will send a TLSv1.3 ClientHello, even if the server only supports session tickets for TLSv1.2 and below.
This PR appears to fix the problem. This PR makes no changes when using a version of OpenSSL that supports "-no_ssl2". When using a version of OpenSSL that does not support "-no_ssl2", however, rather than using $OPTIMAL_PROTO, this PR has sub_session_resumption() use whatever protocol version the server connected with when $sessticket_lifetime_hint was set.
There is a problem if a TLSv1.3-only server is tested using the OpenSSL 1.0.2-chacha binary and $OSSL_SHORTCUT is true.
$HAS_NO_SSL2 is set to true when find_openssl_binary() is called with OpenSSL 1.0.2-chacha. /usr/bin/openssl does not have the -no_ssl2 option, but the second call to find_openssl_binary(), after setting $OPENSSL to /usr/bin/openssl, does not set $HAS_NO_SSL2 to false. So, later calls to $OPENSSL s_client include the -no_ssl2 option, resulting in connection failures.
This PR fixes the problem by modifying find_openssl_binary() to ensure that every OpenSSL-dependent variable is set by this function.
This PR fixes two issues with finding session tickets when using OpenSSL 1.1.1.
First, if OpenSSL connects to the server using TLSv1.3 and it receives more than one Post-Handshake New Session Ticket, then the "TLS session ticket lifetime hint" will appear more than once in $TMPFILE. This will cause the line to appear more than once in $sessticket_lifetime_hint, which causes problems when trying to extract the $lifetime and $unit from $sessticket_lifetime_hint.
This PR fixes the first problem by changing the awk expression in the lines that set sessticket_lifetime_hint so that only the first line with "session ticket lifetime" is extracted.
The second issue is that some servers (e.g., google.com) return a session ticket for TLSv1.2, but not for TLSv1.3. For such servers, testssl.sh will miss the session ticket if $OPTIMAL_PROTO is empty or "-tls1_3" and the --ssl-native flag is not set.
This PR addresses the second issue with the changes in lines 9047 - 9053 -- the code that is intended to provide a last chance to find a session ticket.
If $OPENSSL supports TLSv.1.3 and the server returns session tickets for TLSv1.3 connections, then the session ticket would have already been found by get_server_certificate(), since get_server_certificate() uses $OPENSSL for TLSv1.3 if $OPENSSL supports TLSv1.3. So, in such circumstances, the code in liens 9047 - 9053 should not try again with TLSv1.3. So, if $OPENSSL supports TLSv1.3 and $OPTIMAL_PROTO is empty or is set to "-tls1_3" (either of which would result in a TLSv1.3 ClientHello), the "$OPENSSL s_client" call is changed to specify -no_tls1_3 rather than $OPTIMAL_PROTO.
The code on line 9047 is also changed to only make this final try is $TLS13_ONLY is false. If $TLS13_ONLY is true, then either:
* $OPENSSL does not support TLSv1.3 and the connection attempt would fail anyway; or
* $OPENSSL supports TLSv1.3, in which case any session ticket would have been found by get_server_certificate(), since get_server_certificate() uses $OPENSSL for TLSv1.3 if $OPENSSL supports TLSv1.3.
In either case, there is no reason to try again to find a session ticket.
This commit adds some more checks to determine_optimal_sockets_params(). These additional checks will almost never need to be run, and so will not slow down the typical run of testssl.sh, but adding them will provide information that can be useful for other parts of testssl.sh.
These additional checks will only be run if the server does not support TLSv1.3 and a TLSv1.2 ClientHello is not successful. This means that either:
* The server is not an TLS/SSL enabled server.
* The server only supports SSLv2.
* The server supports some protocol in SSLv3 - TLSv1.1, but does not handle version negotiation correctly.
Adding these additional checks helps in at least the following ways.
If determine_optimal_proto() (assuming it is reverted to using OpenSSL) is unable to connect to the server using OpenSSL, it will be possible determine whether the problem is the the server does not support TLS/SSL or that a different version of OpenSSL is needed to test the server.
If the code in #1205 for run_server_preference() is unable to connect when checking for a cipher order, the reason for the failure will be known, making it possible to determine the correct response to the failure.
This PR adds a new helper function that is run just prior to determine_optimal_proto() and that determines the what information tls_sockets() should include in a ClientHello.
For a TLSv1.3 ClientHello, determine_optimal_sockets_params() determines whether tls_sockets() should use 0x33 or 0x28 are the extension number for the key_share extension. 0x33 should be used with servers that support RFC 8446 or drafts 23-28. 0x28 should be used with servers that support drafts 18-22.
For a TLSv1.2 ClientHello, determine_optimal_sockets_params() determines what cipher list tls_sockets() should send. For most servers, the list of ciphers in $TLS12_CIPHER works best. But, there are some servers that do not support any ciphers in $TLS12_CIPHER, but do support one or more ciphers in $TLS12_CIPHER_2ND_TRY.
This PR fixes a few issues with run_protocol():
* In the case that the call to `tls_sockets "03" "$TLS12_CIPHER"` had a return value of 2, the code determining what results to print was looking at `$DETECTED_TLS_VERSION`. However, the value of this variable was set by the later call to `tls_sockets "04" "$tls13_ciphers_to_test"`. This caused incorrect results in the case of a server that supports TLSv1.3 and TLS1.1 (or earlier), but not TLSv1.2. This PR saves the value of `$DETECTED_TLS_VERSION` in `$tls12_detected_version` and then uses this variable later rather than `$DETECTED_TLS_VERSION`.
* When running in debug mode with a server that does not support TLSv1.3, testssl.sh was printing
TLS 1.3 -- downgradednot offered and downgraded to a weaker protocol"
This PR fixes the output by not printing the "--downgraded"
* As noted in #1329, run_protocols() was treating a downgrade from TLSv1.2 as less bad if the server supports TLSv1.3. This PR changes this code back to treat any downgrade from TLSv1.2 as equally bad.
* In order to be consistent with the TLSv1.3 test, this PR changes the TLS1.2 test output to say "not offered and downgraded to a weaker protocol" if a TLSv1.2 ClientHello results in a downgraded connection.
