... from "Further IP addresses" and before calling ``get_aaaa_record``
in ``determine_ip_addresses()``. Logic appeared needlessly to difficult
and was as far as the "Further IP addresses" line was concerned incomplete.
This PR changes the logic the no-DNS switch works. The switch
now expects a value. "min" does minimum lookups, "none" does
no lookups at all (details see testssl.sh(1) ). "none" is
equivalent to the paranoid (boolean) value "true" before.
When performing client simulations in "--ssl-native" mode, provide the client's list of supported curves to "$OPENSSL s_client" in order to make the results even more accurate.
This PR improves client simulation in "--ssl-native" mode:
* It changes ${protos[i]} to list the protocols that should be disabled rather than those that should be enabled, except in the case that the client only supports one protocol.
* It sets the values for ${tlsvers[i]}, which is used in run_client_simulation(), but was not defined.
* It adds a new variable, ${ciphersuites[i]}, that lists the TLSv1.3 cipher suites supported by a client.
Client simulation still produces false results in "--ssl-native" mode, but the results are better than before.
This PR fixes three issues related to the testing for RFC 7919 DH groups in run_pfs():
* If the RFC 7919 DH groups are supported for both TLSv1.3 cipher suites and non-TLSv1.3 cipher suites, then the list of supported groups is printed twice.
* The finding that is used for CSV/JSON files includes the word "offered" after the list of groups, which is inconsistent with other findings.
* Since the $ffdhe_offered is only used to determine whether to test for use of RFC 7919 DH groups with non-TLSv1.3 ciphers, this flag should only be set if a non-TLSv1.3 ciphers that uses ephemeral DH is found.
If OpenSSL reports an error, sclient_connect_successful() may incorrectly interpret it as a connectivity problem, leading testssl.sh to stop testing before it has completed.
When not using "--ssl-native" mode, this happens if $OPENSSL does not support SSLv3, as both get_server_certificate() and run_beast() will attempt to connect using SSLv3 even if $OPENSSL does not support it.
When using "--ssl-native" mode, this happens in multiple places if $OPENSSL does not support the protocol being used or if $OPENSSL does not support any of the ciphers that are specified in the command line.
This PR fixes the above problems by adding checks for protocol support or for support for at least one cipher before calling $OPENSSL.
run_client_simulation() also has a problem in "--ssl-native" mode of calling $OPENSSL with parameters that cause $OPENSSL to report an error, but this is already addressed by temporarily setting MAX_OSSL_FAIL to 100 during client simulation tests and then, after client simulation testing is complete, returning $MAX_OSSL_FAIL and $NR_OSSL_FAIL to the values they had before client simulation testing began.
``run_logjam()`` contained in certain cases additional quotes
and a typo where only the word comment ended up in JSON/CSV.
Instead of ``$(awk '/Master-Key: / { print $2 }' "$2")`` the
admittedly performance sensitive function ``sclient_connect_successful()``
contains now a bash internal match according to #997 . First
tests didn't show much benefit (only default run with mostly
sockets was tested).
Unit tests showed no problems so far, but coverage is low.
See previous commit
This commit finally fixes#1005 so that either a --ssl-native scan
terminates on the next (defined) occasion if there are network connectivity
problems. It introduces another set of variables (MAX_OSSL_FAIL vs. NR_OSSL_FAIL).
As "openssl s_client connect" is sometimes still being used without --ssl-native
it also shortens the wait for regular scans if an outage is encountered.
To make things easier bot sets (incl. *_SOCKET_FAIL) of variables are independent.
For the seldom case that somebody uses --ssl-native with client checks an exception
had to be made as otherwise only MAX_OSSL_FAIL client check would be performed.
This hasn't been understood yet...
As sometimes HTTP header requests (over OpenSSL) fail repeatedly in a way that an empty
reply is returned, the same strategy of detecting problems is applied here,
using MAX_HEADER_FAIL and NR_HEADER_FAIL.
All three detection mechanisims share a new function connectivity_problem().
This commit finally fixes#1005 so that either a --ssl-native scan
terminates on the next (defined) occasion if there are network connectivity
problems. It introduces another set of variables (MAX_OSSL_FAIL vs. NR_OSSL_FAIL).
As "openssl s_client connect" is sometimes still being used without --ssl-native
it also shortens the wait for regular scans if an outage is encountered.
To make things easier bot sets (incl. *_SOCKET_FAIL) of variables are independent.
For the seldom case that somebody uses --ssl-native with client checks an exception
had to be made as otherwise only MAX_OSSL_FAIL client check would be performed.
This hasn't been understood yet...
As sometimes HTTP header requests (over OpenSSL) fail repeatedly in a way that an empty
reply is returned, the same strategy of detecting problems is applied here,
using MAX_HEADER_FAIL and NR_HEADER_FAIL.
All three detection mechanisims share the new function connectivity_problem().
In addition unit tests showed that some vulnerability checks lost their
CVEs+CWEs whcich have been readded. For ROBOT a CVE was added (F5)
parse_tls_serverhello() checks $TLS_CLIENT_HELLO for a supported_versions extension, and if it contains one, checks that the negotiated version is listed in that extension. However, while $TLS_CLIENT_HELLO is always set in socksend_tls_clienthello() it is not set by client_simulation_sockets() (or any of the functions that client_simulation_sockets() calls). As a result, when the server's response to a client simulation is parsed, parse_tls_serverhello() may compare the negotiated version against the supported_versions extension from a ClientHello message from a previous test.
This PR fixes the problem by having client_simulation_sockets() set $TLS_CLIENT_HELLO.
In addition to 080840f655 and to
address #1005 eventually this is the first part of dealing
with connectivity problems. As opposed to the commit mentioned
above this is the counterpart for openssl which is useful for
just normal usage and for using --ssl-native.
It adds another global MAX_OSSL_FAIL (preset to 2) representing
a threshold how many unsucessful openssl connections are needed to
quit the whole scan. It should again reduce scan time in those
cases.
This is the framework part in sclient_connect_successful() which
is mainly commented out. The hooks will follow soon.
In certain cases (see e.g. #939) the server side "kind of" falls back
from a TLS channel to the underlying plaintext STARTTLS with a 5xx
or 4xx.
This PR adds the detection of 4xx for all protocols (if STARTTLS has been
used) and labels the detection as "likely not av" (and keeps the warning). This
leads to two JSON/CSV objects in those cases.
The detection isn't quite perfect as the issue #939 shows the first
TLS message fragment resembles a TLS alert.
As noticed it also removes the recommendation to increase the
debug level if the level is already sufficient.
FIX#1010 where a missing HSTS header lead to HIGH severity
whereas screen output was on info level.
In order to emphasize the importance of HSTS a missing header
flag will now be slighly penalized. To be in line with HSTS=0
the latter needed to be readjusted to medium as well.
As suggested by krissi for better readability a missing HSTS
header will now be 'not offered' instead of '--' on screen.
This PR changes find_openssl_binary() so that $OSSL_VER_APPENDIX contains everything from $OSSL_VER that appears after $OSSL_VER_MAJOR.$OSSL_VER_MINOR.
This PR fixes the issue raised in #1013. It primarily does this in two ways:
* In calls to `$OPENSSL s_client` that specify ciphers, the TLSv1.3 ciphers are provided separately using the `-ciphersuites` option. Then, the `s_client_options()` function manipulates the command-line options as necessary based on the version of OpenSSL being used.
* Calls to `$OPENSSL ciphers` were replaced with calls to `actually_supported_ciphers()`, which calls `$OPENSSL ciphers`. `actually_supported_ciphers()` modifies the parameters for the call to `$OPENSSL ciphers` as necessary, based on the version of OpenSSL being used.
When using testssl.sh with the current development branch of OpenSSL 1.1.1, determine_trust() incorrectly reports that "/openssl <= 1.0.2 might be too unreliable to determine trust." The problem is that find_openssl_binary() is not correctly extracting the version information. The version is "1.1.1-pre3-dev," but find_openssl_binary() extracts:
OSSL_VER_MAJOR: 1
OSSL_VER_MINOR: 1.13
OSSL_VER_APPENDIX: -pre-dev
This PR fixes the problem and also eliminates the use of external functions in extracting version information for $OSSL_VER.
Note that this code makes a change from the current code. Currently, $OSSL_VER_APPENDIX is intended to contain anything from $OSSL_VER that comes after $OSSL_VER_MAJOR.$OSSL_VER_MINOR. For example, if $OSSL_VER is 1.1.0-dev, then $OSSL_VER_APPENDIX is "-dev". In this PR, the "-dev" is dropped and so does not appear in $OSSL_VER_MAJOR, $OSSL_VER_MINOR, or $OSSL_VER_APPENDIX. The reason for this is that testssl.sh is only using $OSSL_VER_APPENDIX in cases in which $OSSL_VER_MAJOR.$OSSL_VER_MINOR is 0.9.8 to determine whether 0.9.8a - 0.9.8l or 0.9.8m - 0.9.8z. So, it seems that testssl.sh isn't interested in things such as "-dev" or "-pre3-dev".
If desired, this PR could be changed to that $OSSL_VER_APPENDIX contains everything in $OSSL_VER that appears after $OSSL_VER_MAJOR.$OSSL_VER_MINOR.
This PR fixes#1015 by adding underscore ('_') to the list of characters that may appear in a domain name label. a178f3e183 already addressed this for the check of whether the Common Name field in a certificate is a DNS name. This PR fixes it for checks of whether a domain name is a wildcard name and whether the server's host name is a wildcard match against a name in a certificate.
In cases where HTTP/2 was the only ALPN protocol, the
JSON output was missing the finding as the additional
protocol was empty.
This fix tests whether there's an additional protcol
and only calls fileout() if there's one.
When testssl.sh was using -ssl-native and the local openssl version
didn't support any DH ciphers the deliberately lazy approach was to
not complete the second part of the test: check for common primes.
This is a FIX for issue #1016.
This PR adds a missing check for local TLSv1.3 support in cipher_pref_check(). If the "--ssl-native" mode is being used and $OPENSSL does not support TLSv1.3, then a warning should be displayed indicating that TLSv1.3 cannot be checked rather than displaying the same results as if the server did not support TLSv1.3.
Drafts 25 and 26 of TLS 1.3 were issued in early March. Neither made any change that is relevant to testssl.sh.
This PR adds checks for these two drafts.
The implementation of read_sigalg_from_file() was changed on January 29 in 88cd5528e7. The new implementation does not work correctly in cases in which read_sigalg_from_file() is called with $TMPFILE as as parameter.
The current implementation of the function is:
```
read_sigalg_from_file() {
local hostcert_txt="${1//pem/txt}"
[[ -r "$hostcert_txt" ]] || $OPENSSL x509 -noout -text -in "$1" 2>/dev/null >$hostcert_txt
awk -F':' '/Signature Algorithm/ { print $2; exit; }' $hostcert_txt
}
```
When called using $TMPFILE (/tmp/testssl.XXXXXX/tempfile.txt), hostcert_txt is set to $TMPFILE, and since this file exists and is readable, the next line does nothing and the final line tries to read the signature algorithm from $TMPFILE rather than from a parsed version of the certificate.
This PR reverts read_sigalg_from_file() to its previous implementation, at least as a temporary solution.
Support for X448 was recently added to the development branch of OpenSSL 1.1.1. This PR adds an X448 key pair to etc/tls_data.txt (that was generated using OpenSSL 1.1.1) and adds X448 to the supported_groups extension for TLS 1.3 ClientHello messages.
Previously testssl.sh ran into problems when both Public-Key-Pins
and Public-Key-Pins-Report-Only were present in the header.
Now if both are present only Public-Key-Pins is evalutated and a
hint is given. If Public-Key-Pins-Report-Only is only present
a hint is given and those are evaluated.
If more than 1 from either Public-Key-Pins-Report-Only or
Public-Key-Pins is detected only the first one will be processed
and a warning will be displayed.
This commit improves assessments of server which decides after
a few connects either to drop all tcp packets or send a tcp
reset, see #1005 . It adds another global MAX_SOCKET_FAIL
(preset to 2) representing a threshold how many bash socket
connections are needed to quite the whole scan. It dramatically
reduces scan time and makes sure that the user is clear that
the scan had a problem and the results at maximum partly useful
(CSV, HTML, JSON). It also writes a valid JSON file.
See also #769.
It also fixes non-initialized variables in ``fileout_json_finding()``
and ``fileout()``
Dirk
David Cooper
Karsten Weiss