... from "Further IP addresses" and before calling ``get_aaaa_record``
in ``determine_ip_addresses()``. Logic appeared needlessly to difficult
and was as far as the "Further IP addresses" line was concerned incomplete.
This PR changes the logic the no-DNS switch works. The switch
now expects a value. "min" does minimum lookups, "none" does
no lookups at all (details see testssl.sh(1) ). "none" is
equivalent to the paranoid (boolean) value "true" before.
``run_logjam()`` contained in certain cases additional quotes
and a typo where only the word comment ended up in JSON/CSV.
Instead of ``$(awk '/Master-Key: / { print $2 }' "$2")`` the
admittedly performance sensitive function ``sclient_connect_successful()``
contains now a bash internal match according to #997 . First
tests didn't show much benefit (only default run with mostly
sockets was tested).
Unit tests showed no problems so far, but coverage is low.
See previous commit
This commit finally fixes#1005 so that either a --ssl-native scan
terminates on the next (defined) occasion if there are network connectivity
problems. It introduces another set of variables (MAX_OSSL_FAIL vs. NR_OSSL_FAIL).
As "openssl s_client connect" is sometimes still being used without --ssl-native
it also shortens the wait for regular scans if an outage is encountered.
To make things easier bot sets (incl. *_SOCKET_FAIL) of variables are independent.
For the seldom case that somebody uses --ssl-native with client checks an exception
had to be made as otherwise only MAX_OSSL_FAIL client check would be performed.
This hasn't been understood yet...
As sometimes HTTP header requests (over OpenSSL) fail repeatedly in a way that an empty
reply is returned, the same strategy of detecting problems is applied here,
using MAX_HEADER_FAIL and NR_HEADER_FAIL.
All three detection mechanisims share a new function connectivity_problem().
This commit finally fixes#1005 so that either a --ssl-native scan
terminates on the next (defined) occasion if there are network connectivity
problems. It introduces another set of variables (MAX_OSSL_FAIL vs. NR_OSSL_FAIL).
As "openssl s_client connect" is sometimes still being used without --ssl-native
it also shortens the wait for regular scans if an outage is encountered.
To make things easier bot sets (incl. *_SOCKET_FAIL) of variables are independent.
For the seldom case that somebody uses --ssl-native with client checks an exception
had to be made as otherwise only MAX_OSSL_FAIL client check would be performed.
This hasn't been understood yet...
As sometimes HTTP header requests (over OpenSSL) fail repeatedly in a way that an empty
reply is returned, the same strategy of detecting problems is applied here,
using MAX_HEADER_FAIL and NR_HEADER_FAIL.
All three detection mechanisims share the new function connectivity_problem().
In addition unit tests showed that some vulnerability checks lost their
CVEs+CWEs whcich have been readded. For ROBOT a CVE was added (F5)
In addition to 080840f655 and to
address #1005 eventually this is the first part of dealing
with connectivity problems. As opposed to the commit mentioned
above this is the counterpart for openssl which is useful for
just normal usage and for using --ssl-native.
It adds another global MAX_OSSL_FAIL (preset to 2) representing
a threshold how many unsucessful openssl connections are needed to
quit the whole scan. It should again reduce scan time in those
cases.
This is the framework part in sclient_connect_successful() which
is mainly commented out. The hooks will follow soon.
In certain cases (see e.g. #939) the server side "kind of" falls back
from a TLS channel to the underlying plaintext STARTTLS with a 5xx
or 4xx.
This PR adds the detection of 4xx for all protocols (if STARTTLS has been
used) and labels the detection as "likely not av" (and keeps the warning). This
leads to two JSON/CSV objects in those cases.
The detection isn't quite perfect as the issue #939 shows the first
TLS message fragment resembles a TLS alert.
As noticed it also removes the recommendation to increase the
debug level if the level is already sufficient.
FIX#1010 where a missing HSTS header lead to HIGH severity
whereas screen output was on info level.
In order to emphasize the importance of HSTS a missing header
flag will now be slighly penalized. To be in line with HSTS=0
the latter needed to be readjusted to medium as well.
As suggested by krissi for better readability a missing HSTS
header will now be 'not offered' instead of '--' on screen.
In cases where HTTP/2 was the only ALPN protocol, the
JSON output was missing the finding as the additional
protocol was empty.
This fix tests whether there's an additional protcol
and only calls fileout() if there's one.
When testssl.sh was using -ssl-native and the local openssl version
didn't support any DH ciphers the deliberately lazy approach was to
not complete the second part of the test: check for common primes.
This is a FIX for issue #1016.
Previously testssl.sh ran into problems when both Public-Key-Pins
and Public-Key-Pins-Report-Only were present in the header.
Now if both are present only Public-Key-Pins is evalutated and a
hint is given. If Public-Key-Pins-Report-Only is only present
a hint is given and those are evaluated.
If more than 1 from either Public-Key-Pins-Report-Only or
Public-Key-Pins is detected only the first one will be processed
and a warning will be displayed.
This commit improves assessments of server which decides after
a few connects either to drop all tcp packets or send a tcp
reset, see #1005 . It adds another global MAX_SOCKET_FAIL
(preset to 2) representing a threshold how many bash socket
connections are needed to quite the whole scan. It dramatically
reduces scan time and makes sure that the user is clear that
the scan had a problem and the results at maximum partly useful
(CSV, HTML, JSON). It also writes a valid JSON file.
See also #769.
It also fixes non-initialized variables in ``fileout_json_finding()``
and ``fileout()``
If a severity level >= INFO was supplied,JSON* and CSV output were
not writing error conditions in fileout(). As those indicate a problem
of the assessment, this has been fixed so that if any severity is
supplied to testssl.sh it will always write WARN conditions
to file when detected.
Also if the scan was interrupted on the client side where previously
a scanTime of 0 was written to the log file now it will raise a warning
that the scan was interrupted.
See also #769. Error conditions after starting a scan on the server side
(TCP Reset, DROP after a while) still won't be treated accurately in the file output
with an error message but the resulting JSON should be still valid.
OpenBSD doesn't seem to be able to convert from $(date)
output to a more readable format as performed on notBefore
and notAfter from x509(1). Thus an exemption is made and
just display the format like x509(1) shows.
Also newer OpenBSD come with a newer LibreSSL version,
thus the pattern when to display a warning when the
"too unreliable to determine trust" warning is displayed.
As per @krissi's suggestion this is a smart addition to add privat CAs
to all certificate stores -- during runtime only. The switch --add-ca or
--add-CA expects an argument of one additional CA file or a comma separated
lists of them. The enviroment file ADDITIONAL_CA_FILES can be used
alternatively.
This fixes eventually #230.
For recognising error conditions during run any testssl.sh run, the program
returns 0 if all checks have been performed successfully and unambiguously. 1
is returned per error or unambiguous condition and per IP. E.g. ./testssl.sh
--ssl-native testssl.net with two IP addresses returns 2 as each client
simulation gets a "1" (ssl-native returns currently wrong results here).
This fixes#986.
A kind of vulnerability scoring for monitoring tools or CI will be done
later separately, see #985 and #327.
As before (see #971) and as indicated (#970) some minor
changed to the jsonID have been taking place. Some
redundant verbosity has been eliminated, some IDs
changed the name.
For HTTP2 you should now ALPN_HTTP2.
Some obvious CWEs were added in the JSON section.
ke for null cipher list, information leakage in
the http headers. There's probably space for improvements.
A few early time marks were added to debug time spend.
Error codes now done until run_server_defaults(). Previous
sections modified so that execution errors are added.
Modified in compare_server_name_to_cert() ret --> subret.
From the code perspective a clear distinction between
passing an error code and a functional return code
would be great. Still has to be determined whether it
maybe better to rename ret into something different.
Removed a stale code line in run_server_preference() for STARTTLS.
Following the recommendation from @dcooper16 this commit is addressing
a situation when the scan couldn't finish for external reasons and as
a consequence left a non-valid JSON file behind.
It also starts addressing #986 so that the protcol section only returns
a non-zero value if a check coundn't be performed or gave results which
weren't clear.
It also fixes a typo where in the TLS 1.3 check a status from the TLS 1.2
check was not correctly interpreted (TLS 1.2 not offered).
In order to be more consistent with the other output functions having
"svrty" in their name, *_done_best and *_done_good were changed to
*_svrty_best and *_svrty_good.
run_spdy/run_http2 were renamed to run_npn and run_alpn as this is
what is actually being tested. Also the terminal and file output
is now reflecting this.
Also #980 was fixed in a sense that (only) for ALPN the protocol
h2 will get a "good". There will be an additional CSV/JSON line
for this.
In ``must_staple()`` and ``certificate_info()`` were the jsonIDs
changed to OpenSSLi / IETF names so that testssl.sh is more compliant
to the rest of the world. There might be still space for improvements
are far as common naming scheme is concerned.
The host certificate is now being delivered in JSON and CSV. For
further usage " " needs to be converted back to linefeeds.
Certificate Expiration was renamed to Certificate Validity.
The order of outputting the certificate serial and SHA1 fingerprint
has been swapped.
Also ``certificate_info()`` makes more use of HOSTCERT_TXT.
For certificate start+end time it is now displaying the
time on UTC and without mentioning the timezone twice.
Also if neither CRL nor OCSP URI is provided it'll appear
on the screen below those two checks. JSON/CSV has then an
additional finding
HOSTCERT_TXT is now generated each time a HOSTCERT is
written. For now it is just being used in ``read_sigalg_from_file()``.
More to come.
Also in JSON output the start date and end date of the certificate
are now in separate objects. (the time format could need some polishing though).
``run_tls_fallback_scsv()`` was upgraded to the new scheme.
It also changes a couple of more functions using a variable ``$jsonID`` instead
of a repeating fixed value for `fileout()`.
Also some it adds some ``fileout`` calls which were missing.
In the sense of the previous commits this one adds
further changes to the JSON output parameters ID and findings.
Findings in JSON are now more crunchy and don't repeat parts
of ID. Also ID changed so that it is more reflecting what
has been tested.
As the cipherlist checks in the beginning have been less
and less to do with the OPsnSSL standard lists
a change to remove the word "standard" was long overdue.
That has been addressed now alos in the code and in the
JSON/CSV output.
$HOSTCERT has now an .pem file extension. $HOSTCERT_TXT
will contain the text putput of the x509 openssl operation
on the certificate which enables testssl.sh to remove
some of the redundant "openssl x509 -in $HOSTCERT -text"
calls.
fileout <somestrings> has not been consistently replaced
by fileout $jsonID yet.
This commit just adds more changes to those two parameters.
It is not completely done yet (see downgrade protection via SCSV).
Also json_prefix was changed to jsonID. The complete change of the fist
fileout field is pending.
The server default run had several JSON objects which weren't, looking at just
the ID, either clear or contained a redundant explanation in "finding". Purely
certificate related JSON objects are now having the id "cert_<object>" like
cert_CN or cert_SAN.
This commit changes all this, also it avoids another colon in finding (see #830).
Also the implicit strategy "output for the screen s followed by only one output with
fileout" has been relaxed -- which results on more, better parsable JSON objects.
Some example of the changes:
Old:
----
{
"id" : "Server Certificate #1 fingerprint",
"severity" : "INFO",
"finding" : "Fingerprints / Serial: SHA1 2940BC13ECF7DAF30B9084CC734C3B971D73B3BB / 01BFD1DC15006E0ABBA7C670FF5E1101, SHA256 30BA61012FFE7CEAAF9A148A0CB0C5C852A9C04F4B1C27DB6
EFA9919C7F49CCF"
}
[..]
{
"id" : "Server Certificate #2 ocsp_stapling",
"severity" : "OK",
"finding" : "OCSP stapling : offered"
}
New:
----
{
"id" : "cert_key_size <cert#1>",
"severity" : "INFO",
"finding" : "Server keys 2048 bits"
},{
"id" : "cert_fingerprint_SHA1 <cert#1>",
"severity" : "INFO",
"finding" : "2940BC13ECF7DAF30B9084CC734C3B971D73B3BB"
},{
"id" : "cert_fingerprint_SHA256 <cert#1>",
"severity" : "INFO",
"finding" : "30BA61012FFE7CEAAF9A148A0CB0C5C852A9C04F4B1C27DB6EFA9919C7F49CCF"
},{
"id" : "cert_serial <cert#1>",
"severity" : "INFO",
"finding" : "01BFD1DC15006E0ABBA7C670FF5E1101"
}
[..]
{
"id" : "OCSP_stapling <cert#2>",
"severity" : "OK",
"finding" : "offered"
}
This PR also fixes the JSON output where for "OCSP must staple" the id was just
'id" : "OCSP must staple: ocsp_must_staple",' for multiple server
certificates without the certificate number.
As far as the code is concerned: $json_prefix should be a variable which is
used for the id object. If there was more then one certificates for a single
host detected, $json_postfix carries the certificate number.
Unit tests need to be fixed -- if possible.
Dirk