This PR addresses a bug where a user encountered the question "The results
might look ok but they could be nonsense. Really proceed".
That happened under Darwin and probably some LibreSSL versions when
checking some hosts. sclient_auth() returned 1 indicating no SSL/TLS
handshake could be established.
This PR modifies sclient_auth() so that in those cases 0 is returned by
skipping the check for the session ID. As NO_SSL_SESSIONID needs to
be set when there's no session ID. This is done separately.
This fixes#2052 for 3.0
As discussed in #2079, this commit adds "</dev/null" to calls "$OPENSSL s_client" that are intended to test whether a certain option is supported. This is done to prevent hanging in the case that the option is supported, a TLS server happens to be listening on the port to which s_client tries to connect, and the connection is successful.
See #2032 / #2067, kudos @Wahnes. This is a backport for 3.0
Currently, when "HTTP clock skew" is calculated, this is taken to be the
difference between the timestamp of the moment the HTTPS request was sent and
the date given in the HTTP Date header. This does not yield valid results in
case a HTTP cache is used, either on the client side or on the server side.
According to the HTTP specs, the Date field will contain the timestamp the
response was created, which may not be the timestamp the response was
delivered.
Consider the following example that queries the Varnish project's web server.
Note that Varnish is a popular HTTP caching server, so HTTP caching will of
course be used when serving HTTP responses from this project's web server.
testssl.sh https://varnish-cache.org/ This will typically output a HTTP clock
skew of some thousand seconds.
The patch takes into account the HTTP Age header that caching servers add to
the HTTP response to signal the response's freshness. As client-side caches
normally do not cache HTTPS requests (except maybe for "enterprise" HTTP proxy
caches that do MITM HTTPS proxying), this is mostly targeted to HTTPS websites
that employ server side HTTP caching.
Addtional polishing:
* address my comments in #2032
* add JSON field HTTP_headerAge if they exists
* output HTTP_AGE if it was detected
* do stripping of line feeds closer to where variables were set
This commit fixes an issue with the headers in the structured JSON output. If run_cipher_match() is performed the header that is used is "pretest" rather than "singleCipher".
This commit adds several variables to reset_hostdepended_vars() that are not currently being reset for each host being tested, but that should be reset.
Sending the entire log to /dev/null is a rather expensive way of checking whether
we are inside a git working tree. Use `git rev-parse --is-inside-work-tree` instead.
(cherry picked from commit 1e3f932932)
This commit fixes#2049 by converting newline characters to spaces in JSON and CSV findings.
fileout() calls newline_to_spaces() on any $finding that is to be written to a JSON or CSV file. However, this only affects actual newline characters in the string, not escaped newline characters (i.e., "\n"). Escaped newline characters pass through this function unchanged, but then get converted to newline characters when they are written to the JSON and/or CSV files. This commit fixes the problem by also converting escaped newline characters ("\n") to spaces.
This fixes#1834 and #1435 for branch 3.0. (3.1dev: #2047)
The --connect-timeout option had the problem that under certain circumstances
like parallel mass scanning didn't work. The culprit was that a subshell command
was used to connect to the target but the file descriptor wasn't exported.
The PR changes the logic so that the connect pre-check is still done in a subshell
If this fails it proceeds with error handling if NR_SOCKET_FAIL is above threshold.
Otherwise it just connects again.
This commit fixes a bug in sub_cipherlists() when using sockets to test whether a server supports a set of ciphers with SSLv2. Code that is supposed to extract the list of SSLv2 ciphers to test for is incorrectly reading from the list of non-SSLv2 ciphers.
Some servers are configured to prioritize ChaCha ciphers if those ciphers are preferred by the client, even if the server is generally configured to use the server's cipher preferences rather than the client's. As a result of this, if a ChaCha cipher appears in the ClientHello before a non-ChaCha cipher, the server may select the ChaCha cipher even if the server is configured to prefer the non-ChaCha cipher.
In a few cases, e.g., cloudflare.com for TLS 1.2, this affects the ordering of the ciphers presented by cipher_pref_check(). This PR fixes the problem by having cipher_pref_check() (and check_tls12_pref()) always place any ChaCha ciphers at the end of the cipher list in the ClientHello. This ensures that cipher_pref_check() presents the ciphers in the server's preference order.
As noted in #2016 the detection of curves in find_openssl_binary()
was not yet perfect.
This commit removes another connect call. Also it corrects the port statement
for LibreSSL-like pre-checks as port 0 generally seems to me better suited.
Inline documentation was added.
It worked so far with OpenSSL 1.0.2, 1.1.1 and 3.0 and LibreSSL 3.4.
This is for 3.0. Similar commit (a11bd1585e)
was for 3.1dev, see PR #2020 .
Same as #2014, this is for 3.0 though.
OpenSSL shows certificate serial numbers >35 with a LF (0A). Testssl.sh
just output that which makes JSON invalid and displays the LF in the terminal
too.
This PR fixes that (#2010) by adding text filters so that the
serial number is not a multiline string.
Also this PR introduces a new function: a size check of the cert serial.
Below 8 bytes the CAB Forum's lower limit is hit which says the entropy
from a CSPRNG should be at least 64 bits. It is assumed that below 8 bytes
length this requirement isn't possible to meet (needs to be clarified with
Shannon, 8 bytes seems to low to me).
The high threshold is according to RFC 5280, Section-4.1.2.2 .
See also #2013.
The output has changed, so that on the terminal the serial has one line,
SHA1 and SHA256 each one line. The new json key is "cert_serialNumberLen".
This commit provides a global variable to the RFC 6761 use of "invalid."
which WSL clients don't seem to handle very well, see #1738, #1812.
"invalid." is used as a target to find out in a couple of pre-checks what
is supported by the openssl version.
This PR reduces the number of ``openssl s_client -connect`` by a huge factor.
For the remaining invocations the OS used is being determined and if WSL is
assumed (the check is probably not 100% accurate) it uses ``127.0.0.1:0`` instead.
In (unfortunately only a few) pre-tests the response was immediate.
Also it is possible to use another target if needed by
NXCONNECT=<mytargethere>:<myport> ./testssl.sh <URL>
This is for 3.0. For 3.1dev see #1988 .
Newer configuration files from openssl may include statements
which aren't compatible with our supplied old openssl version.
This commit adds an autodetection of such a file and uses a
openssl.cnf provided by this project then.
The openssl.cnf is from a Debian Jessie package.
See also #1987
A longer while back the section ~ "Testing standard ciphers" was
renamed to "Testing cipher categories". However the internal help
didn't reflect that.
This fixes that, including an addtion to the documentation.
Note: the help still lists "-s --std, --standard" as a cmd line
switch.
Similar to #1972.
* the ignore ~/.digrc option from dig is now parsed from the builtin help
* there was a potential DNS call which is now avoided
* the +noidnout check is now parsed from the builtin help
* the OPENSSL_CONF="" in check_resolver_bins() was moved a few lines higher to avoid other errors in the terminal
* redo check for noidnout by also using the help function of dig
See also #1949
