See pending PR #905 / issue #333.
There's still lots of work needed and probably the function
needs to be completely rewritten and to be in sync with
other parts of the program.
This PR fixes one odd formatting of header flags like X-Frame-Options,
where the output header maybe contained a LF "\r". X-XSS-Protection was
also not correctly formatted due to the fact that only a part of it until
the blank was displayed.
Also the file output may contain now 1x less blank, e.g.
"X-Content-Type-Options: nosniff" instead of
"X-Content-Type-Options: nosniff"
This PR adds an additional COLOR level (3). If color is set to 3 then all ciphers are printed according to pr_cipher_quality() rather than just the "Negotiated cipher" in run_server_preference().
This PR reduces the amount of work parse_tls_serverhello() does when processing alert messages when not in debug mode. It delays writing anything to $TMPFILE unless $DEGUG -ge 1 until it has reason to believe that the response was successful. If $DEBUG is 0 and alert messages are sent, then no file operations are performed processing the alert messages.
In almost every case, there is no attempt to look at the contents of $TEMPDIR/$NODEIP.parse_tls_serverhello.txt unless the connection was successful. So, in most cases, it is okay to not call tmpfile_handle() in parse_tls_serverhello() unless the connection was successful. There is, however. one place in run_grease() where the code reads the contents of $TEMPDIR/$NODEIP.parse_tls_serverhello.txt even if the connection was not successful. In order to address this, the DEBUG level is temporarily set to 1 when performing this test if its value is 0. Also in order to address this, changes were made in parse_tls_serverhello() to ensure that "tmpfile_handle $FUNCNAME.txt" is always called before returning if $DEBUG -ge 1.
If certificate-based client authentication is required by the server, then most HTTP-related checks are skipped, even if the "--assume-http" flag is used. If $CLIENT_AUTH is true, then $ASSUME_HTTP is ignored.
In some cases the checks are appropriately skipped, since the tests cannot be performed. In other places, the value of "$CLIENT_AUTH" is used as a hint as to whether HTTP is being used. For example, in run_tickbleed:
if [[ "$SERVICE" != HTTP ]] && ! "$CLIENT_AUTH"; then
outln "-- (applicable only for HTTPS)"
fileout "ticketbleed" "INFO" "Ticketbleed: not applicable, not HTTP" "$cve" "$cwe"
return 0
fi
There are some places, however, where tests are just skipped, even if both $CLIENT_AUTH and $ASSUME_HTTP are true, even though the test could be performed. For example, run_client_simulation() only simulates generic clients in this case.
This PR attempts to address this:
* In run_client_simulation() it runs all of the tests if $ASSUME_HTTP is true.
* In certificate_transparency() it only says that the lack of CT information is "N/A" it can verify that HTTP is not being used (if $SERVICE is not HTTP and $CLIENT_AUTH is false). Otherwise it just says "no" without flagging it as an issue.
* In certificate_info() it displays additional warnings (about use of SHA-1 or subjectAltName matching) only if it can verify that HTTP is being used ($SERVICE is HTTP or $ASSUME_HTTP is true).
* In run_crime(), if compression is used, it only says " but not using HTTP" if it can verify that HTTP is not being used (if $SERVICE is not HTTP and $CLIENT_AUTH is false).
If a supported_versions extension was included in the ClientHello, then check that the version returned by the server was included in the ClientHello's supported_versions extension.
OpenSSL will respond to a TLSv1.3 ClientHello that only specifies 0304 in its supported_versions extension with a ServerHello that specifies whatever draft of TLSv1.3 it currently supports (e.g., 7F16). The result is that run_protocols() incorrectly reports that OpenSSL supports TLSv1.3 "final" in addition to whatever draft version it supports.
This PR fixes that problem by treating it as a failed connection when the ClientHello offers only 0304 and the ServerHello specifies something else (e.g., 7F16).
Performing this check is actually a requirement for clients in Section 4.2.1 of draft-ietf-tls-tls13-22. So, including this check will also help make client simulations more accurate when clients that support TLSv1.3 are added to client-simulation.txt.
The default STARTTLS_SLEEP timeout was increased to 10 seconds in
d1e7498. This caused MySQL connections to timeout. Quick fix is to
parameterize the timeout and pass in 1 again.
Better future fix is to read MySQL as binary packets, parsing the fixed
sized header, to then read the variable sized payload. Doing this will
also greatly speed up testing.
This fixes issue #914.
It partly addresses #915, supposedly the openssl binary used supports
TLS 1.3 (and the correct draft/final).
It also reduces handshakes by not trying protocols which aren't supported
on the client side.
This PR adds support for TLSv1.3 draft 22. This PR has testssl.sh operate in "middlebox compatibility mode" as described in Appendix D.4 of draft-ietf-tls-tls13-22 to maximize the chances of being able to perform a successful test even if there is a misbehaving middlebox between testssl.sh and the server being tested. Support for drafts 18 through 21 is still maintained.
This PR has been tested against a few different implementations of draft 22 that were made available shortly before draft 22 was posted.
In TLSv1.3, when responding to a HelloRetryRequest, the second ClientHello should be sent through the same socket as the first ClientHello.
This PR adds an option to socksend_tls_clienthello() to not open a socket and then uses that option in resend_if_hello_retry_request() when sending the second ClientHello.
Revert the exception from previous commit 20b38d5aa6
that TLS alerts on SSLv2 client hello won't be a proper reply -- due to a reality check,
see #908.
In order to have better debugging info the TLS alert message is printed
in clear. Messages code and text assignements was moveed to a separate
function.
Some servers like the one from the satire magazine "focus.de"
choose to return gzip encoded body during run_http_haders().
This has led sometimes to misintepretation that an IPv4 address
is present in the header.
This commit fixes that by telling the server not to want a gzipped
response and if still returned grep properly the return.
When testing servers which fall back after succeeding the STARTTLS handshake
to the underlying protocol (smtp, ftp etc.) there was often misleading output
=not available instead of notifying the user that there's a problem.
Now it is being tested in parse*serverhello() functions whether the first
by resembles a 5XX code and STARTTLS is supposed to be tested and then
passes an error code back to the caller , i.e. tls_sockets() and then
run_protocols(). Also other error code after the handshake are passed
better.
This is only an addition to the top level function run_protocols().
We might want to look into other top level functions too
In scenarios where --ssl-native was chosen AND the
openssl binary wasn't supporting the protocol
there were two warnings. This has been addressed.
Also tls_sockets() can return different values -- for now: 6 -- as
tested by the caller in run_protocols. In order to make
it more robust a fixme statement was added so that the
user becomes iat least aware of it.
After changing the logic from $PROTOS_OFFERED
contaning each protocol detected to $PROTOS_OFFERED
contaning each protocol + colon yes or no there
wasn't a change for the sanity check that no protocols
are offered. This fixes it.
This PR adds TLSv1.3 support for the negotiated protocol and cipher in run_server_preference(). This mostly addresses #893, however, run_server_preference() will not work with a TLSv1.3-only server as it will fail when trying to determine whether the server has a cipher order.
Note that with this PR run_server_preference() will not always provide consistent results when testing a server that does not support TLSv1.3 using a version of OpenSSL prior to 1.1.1. If it is determined before running run_server_preference() (using run_protocols()) that the server does not support TLSv1.3, then run_server_preference() will using OpenSSL to determine the negotiated protocol and cipher. However, if it has not yet been determined that the server does not support TLSv1.3, then run_server_preference() will use tls_sockets(), which tries to simulate OpenSSL 1.1.1. Since the list of cipher lists sent will differ the negotiated cipher will sometimes differ. In addition, when a cipher suite that uses an ephemeral ECDH key is selected, the negotiated curve is sometimes different.
This PR fixes#901. When $OPENSSL_TIMEOUT is set and mass testing is being performed, this PR changes find_openssl_binary() so that only child instances modify $OPENSSL to add the call to timeout.
This PR also changes the warning in case $OPENSSL_TIMEOUT is set and the timeout binary cannot be found so that the warning message is printed even if $WARNINGS is "off" or "batch".
There's a new socksend() function which gives up to 8% performance
benefit (LAN) as is saves 3 sed and 1 tr command in every socksend call.
It has not been put in production yet as it shows a problem in ticketbleed
check and it is late and I call it a day ;-) and resolve that later (see #902).
For performance tests it's quite handy to also have in the flat json format
a footer with time consumed for each run. fileout_json_footer() was
patched accordingly.
Dirk
David Cooper
Steven Danneman