Reduce the offensive tests to 4: the others are "just" / mostly cipher
based checks which should not cause an IDS to block. (This maybe
subject to reconsider at a later time.)
Added a switch --ids-friendly
Updated VULN_COUNT accordingly
Added this (including PHONE_OUT to env debugging output)
Added help()
Manual section added
The certificate from self-signed.badssl.com was about to expire
which raises a MEDIUM type issue in testssl.
This commit does a workaround for this, so that those certificates
will be ok in Travis CI.
(Same problem exists in 2.9.5)
This commit is a FIX for #1069, thus when running in
wide mode it corrects an additional line feed which
happened sometimes.
As @dcooper16 pointed out it also cleans up the needless
if-statements in run_rc4(), run_lucky13() and run_beast().
It also inserts for wide mode lines a blank so the alignment
is not at the left border anymore (check for leftovers
needed).
My previous commit added a host header but didn't properly
format the host header (trailing slashes / path). This commit
corrects that so that the 305 times HTTP 400 in #1056
should now be gone (TBC), including Google CA responders.
One issue which needs to be addressed (same as in CRL
revocation checks): Not trusted certificates (zhanqi.tv,
taken from my Alexa scans) fail for obvious reasons.
Error from OCSP responder is now being displayed (and logged to JSON, ...)
Whole replay is kept in $tmpfile for debugging purposes
JSON output added for OCSP responderi query failures
Furtermore wget was replaced by "type -p" and grep by fgrep.
* Extra function for ldap_get()
* Hint when curl is not installed and LDAP URI is encountered
* Rename jsonID cert_cRLDistributionPoints to cert_crlDistributionPoints
* Fix trailing _ in jsonID
Open/to be clarified:
* Proxy for curl / proxy needs to come from testssl.sh
* Proxy support for HTTP bash socket GET
* cert_CRLrevoked comes before cert_cRLDistributionPoints
* Unit tests
Still open: OCSP
In order to use it one has to use --phone-out (PHONE_OUT
is the respective ENV) like
``./testssl.sh --phone-out --json-pretty -S wikipedia.org``
This makes use of curl (if available) or wget (if available) and
falls back to bash socket GET. The latter uses HTTP/1.0 as
chunked transfers by the server (used for bigger files normally)
can't be reasonably separated from their HTTP header. (HTTP/1.0
doesn't support chunked transfers).
curl and wget use the enviroment variables automatically. Probably
we want to use those proxies only if told by a switch to testssl.sh.
"-crl_download" would have been an option. Support would have
been needed to check beforehand. Alos information on proper
usage seems limited, so for now a solution which works is
preferred.
Open/to be clarified:
* Documentation
* Proxy for curl / proxy needs to come from testssl.sh
* Proxy support for HTTP bash socket GET
* JSON ID is cert_CRLrevoked_ (trailing underscore)
* cert_CRLrevoked_ comes before cert_cRLDistributionPoints
(* reconsider naming of cert_cRLDistributionPoints)
* Unit tests
Still open: OCSP
As it would be a possible privacy violation a new flag PHONE_OUTSIDE
is introduced (later accompanied by a switch). It determines whether
the client is allowed to retrieve the CRL specified (HTTP only supported).
Tested ok against wikipedia.de and revoked.badssl.com.
To do:
* look into -crl_download
* fileout
* Unit tests
OCSP verification
In certain situations while testting for CCS injection it could have happened
that an error code was sent which was not interpreted properly by testssl.sh.
(https://tools.ietf.org/html/rfc5246#section-7.2)
This has now been fixed and thus addresses #906. Also it has been made sure
that other error codes are reported appropiately.
The case where this test failed before was a non-patched Ubuntu 12.04
with openssl/postfix on port 25.
For the upcoming release this commit initiated the beta phase: important features
will be allowed. On the agenda is otherwise to fix bugs.
I ran shellcheck (see #434), and fixed some complaints and adjusted some coding
style mismatches.
There were some cases where security headers were served two times by the
server. The result (screen+html) wasn't properly formatted in those cases.
match_httpheader_key() was improved so that it keeps track when
a CR or an indentation needs to be done.
Some egrep statements were replaced by grep -E as this has been used
already and it is the thing testssl.sh should settle for. (precursor
to #1022).
run_more_flags was renamed to sun_security_headers and names of
variables is better.
HAS_SPDY is now HAS_NPN (similar to renaming the function a while
back)
mktemp should only be used when not avoidable (performance, code). For
temporarily local variables names can often be borrowed from globals
which were already generated by mktemp (SOCK_REPLY_FILE).
This is a fix for #722. It updates the client simulation data from
the SSLlabs API. As usual data was pulled, resorted and clients
to display were hand-selected.
Wishlist: Missing is Oreo, OpenSSL 1.1.1, Safari on OX 11, Firefox
52.x (ESR)
With the recent PR #1033 from @dcooper it can also show TLS 1.3
handshakes.
According to programming standards e.g. C-style defines) testssl.sh has now
internal error variables (ERR_*) which are defined to deal with exit codes in
error conditions. Details see ``testssl.sh(1)``, section exit. Thus exit codes
because of an error are now standardized and if needed can be easily changed to
other values.
This is part of a cleanup mentioned in #985 and #752. Codes for monitoring
tools (#327) which imply some kind of rating are still to be done.
The beginning section was reformatted and some items were reordered to keep
variables and functions together which serve similar purposes.
``readonly`` was replaced by ``declare -r`` (closer to C's define and it
makes more sense to settle on one variable if both are being used
for the same purpose)
... from "Further IP addresses" and before calling ``get_aaaa_record``
in ``determine_ip_addresses()``. Logic appeared needlessly to difficult
and was as far as the "Further IP addresses" line was concerned incomplete.
This PR changes the logic the no-DNS switch works. The switch
now expects a value. "min" does minimum lookups, "none" does
no lookups at all (details see testssl.sh(1) ). "none" is
equivalent to the paranoid (boolean) value "true" before.
``run_logjam()`` contained in certain cases additional quotes
and a typo where only the word comment ended up in JSON/CSV.
Instead of ``$(awk '/Master-Key: / { print $2 }' "$2")`` the
admittedly performance sensitive function ``sclient_connect_successful()``
contains now a bash internal match according to #997 . First
tests didn't show much benefit (only default run with mostly
sockets was tested).
Unit tests showed no problems so far, but coverage is low.
See previous commit
This commit finally fixes#1005 so that either a --ssl-native scan
terminates on the next (defined) occasion if there are network connectivity
problems. It introduces another set of variables (MAX_OSSL_FAIL vs. NR_OSSL_FAIL).
As "openssl s_client connect" is sometimes still being used without --ssl-native
it also shortens the wait for regular scans if an outage is encountered.
To make things easier bot sets (incl. *_SOCKET_FAIL) of variables are independent.
For the seldom case that somebody uses --ssl-native with client checks an exception
had to be made as otherwise only MAX_OSSL_FAIL client check would be performed.
This hasn't been understood yet...
As sometimes HTTP header requests (over OpenSSL) fail repeatedly in a way that an empty
reply is returned, the same strategy of detecting problems is applied here,
using MAX_HEADER_FAIL and NR_HEADER_FAIL.
All three detection mechanisims share a new function connectivity_problem().
This commit finally fixes#1005 so that either a --ssl-native scan
terminates on the next (defined) occasion if there are network connectivity
problems. It introduces another set of variables (MAX_OSSL_FAIL vs. NR_OSSL_FAIL).
As "openssl s_client connect" is sometimes still being used without --ssl-native
it also shortens the wait for regular scans if an outage is encountered.
To make things easier bot sets (incl. *_SOCKET_FAIL) of variables are independent.
For the seldom case that somebody uses --ssl-native with client checks an exception
had to be made as otherwise only MAX_OSSL_FAIL client check would be performed.
This hasn't been understood yet...
As sometimes HTTP header requests (over OpenSSL) fail repeatedly in a way that an empty
reply is returned, the same strategy of detecting problems is applied here,
using MAX_HEADER_FAIL and NR_HEADER_FAIL.
All three detection mechanisims share the new function connectivity_problem().
In addition unit tests showed that some vulnerability checks lost their
CVEs+CWEs whcich have been readded. For ROBOT a CVE was added (F5)
In addition to 080840f655 and to
address #1005 eventually this is the first part of dealing
with connectivity problems. As opposed to the commit mentioned
above this is the counterpart for openssl which is useful for
just normal usage and for using --ssl-native.
It adds another global MAX_OSSL_FAIL (preset to 2) representing
a threshold how many unsucessful openssl connections are needed to
quit the whole scan. It should again reduce scan time in those
cases.
This is the framework part in sclient_connect_successful() which
is mainly commented out. The hooks will follow soon.
In certain cases (see e.g. #939) the server side "kind of" falls back
from a TLS channel to the underlying plaintext STARTTLS with a 5xx
or 4xx.
This PR adds the detection of 4xx for all protocols (if STARTTLS has been
used) and labels the detection as "likely not av" (and keeps the warning). This
leads to two JSON/CSV objects in those cases.
The detection isn't quite perfect as the issue #939 shows the first
TLS message fragment resembles a TLS alert.
As noticed it also removes the recommendation to increase the
debug level if the level is already sufficient.
FIX#1010 where a missing HSTS header lead to HIGH severity
whereas screen output was on info level.
In order to emphasize the importance of HSTS a missing header
flag will now be slighly penalized. To be in line with HSTS=0
the latter needed to be readjusted to medium as well.
As suggested by krissi for better readability a missing HSTS
header will now be 'not offered' instead of '--' on screen.
In cases where HTTP/2 was the only ALPN protocol, the
JSON output was missing the finding as the additional
protocol was empty.
This fix tests whether there's an additional protcol
and only calls fileout() if there's one.
When testssl.sh was using -ssl-native and the local openssl version
didn't support any DH ciphers the deliberately lazy approach was to
not complete the second part of the test: check for common primes.
This is a FIX for issue #1016.
Previously testssl.sh ran into problems when both Public-Key-Pins
and Public-Key-Pins-Report-Only were present in the header.
Now if both are present only Public-Key-Pins is evalutated and a
hint is given. If Public-Key-Pins-Report-Only is only present
a hint is given and those are evaluated.
If more than 1 from either Public-Key-Pins-Report-Only or
Public-Key-Pins is detected only the first one will be processed
and a warning will be displayed.
This commit improves assessments of server which decides after
a few connects either to drop all tcp packets or send a tcp
reset, see #1005 . It adds another global MAX_SOCKET_FAIL
(preset to 2) representing a threshold how many bash socket
connections are needed to quite the whole scan. It dramatically
reduces scan time and makes sure that the user is clear that
the scan had a problem and the results at maximum partly useful
(CSV, HTML, JSON). It also writes a valid JSON file.
See also #769.
It also fixes non-initialized variables in ``fileout_json_finding()``
and ``fileout()``
If a severity level >= INFO was supplied,JSON* and CSV output were
not writing error conditions in fileout(). As those indicate a problem
of the assessment, this has been fixed so that if any severity is
supplied to testssl.sh it will always write WARN conditions
to file when detected.
Also if the scan was interrupted on the client side where previously
a scanTime of 0 was written to the log file now it will raise a warning
that the scan was interrupted.
See also #769. Error conditions after starting a scan on the server side
(TCP Reset, DROP after a while) still won't be treated accurately in the file output
with an error message but the resulting JSON should be still valid.
OpenBSD doesn't seem to be able to convert from $(date)
output to a more readable format as performed on notBefore
and notAfter from x509(1). Thus an exemption is made and
just display the format like x509(1) shows.
Also newer OpenBSD come with a newer LibreSSL version,
thus the pattern when to display a warning when the
"too unreliable to determine trust" warning is displayed.
Dirk