* the ignore ~/.digrc option from dig is now parsed from the builtin help
* there was a potential DNS call which is now avoided
* for +noidnout check however there's a call to invalid. added
* the OPENSSL_CONF="" in check_resolver_bins() was moved a few lines
higher to avoid other errors in the terminal
Tested on (EOL) Ubuntu 14.04 which only has dig in an older version
See also #1950
This commit fixes#1961 in the 3.1dev branch by leaving NODEIP set to the server's IP address rather than changing it to the DNS name in the case of STARTTLS XMPP.
In order to address the problem of $OPENSSL s_client not working with STARTTLS XMPP if an IP address is provided to -connect, the -xmpphost option is used to provide the DNS name.
The fact that debugme1() redirects to stderr and the calls to this functions
redo that is deliberately as in the future we might want to use debugme1
without redirection.
... to address #1956 and other places. Similar to #1957,
only for the 3.1dev rolling release branch.
Also it changes debugme1() back? to output debug
statements only when $DEBUG >= 1. Per default here
also stderr is used.
get_server_certificate() includes a few calls to tls_sockets() in which the response will be TLS 1.3 and in which the response will be useless if it cannot be decrypted (since the goal is to obtain the server's certificate). So, these calls to tls_sockets() should specify "all+" rather than "all".
This commit changes run_server_defaults() so that the test for certificate compression is not run in --ssl-native mode. This fixes an issue that was caught by 21_baseline_starttls.t.
This commit adds a check for whether the server supports certificate compression (RFC 8879). If it does, then the list of supprted compression methods is output in the server's preference order.
If the order of the cmdline is '-U --ids-friendly' then we need to make sure we catch --ids-friendly. Normally we do not,
see #1717. The following statement makes sure. In the do-while + case-esac loop the check for --ids-friendly will be
executed again, but it does not hurt
Newer dig versions have an option to ignore $HOME/.digrc, older don't.
This commit adds a patch checking for the availability of such an option and
uses it by default. See #1894 .
If this option doesn't exist then still dig is used and can still lead to
wrong output. Unfortunately Debian-based distros are not very
good at this. Debian 10, Ubuntu 18.04 still use dig 9.11, whereas
Opensuse 15.2 has 9.16. Debian 11 and Ubuntu 20.04 use that too.
This commit adds a new function, print_n_spaces(), which prints a sequence of (up to 80) space characters.
This new function is used to replace a few places in testssl.sh in which a sequence of space characters is printed by calling 'out " "' in a loop. The new function is much faster than the current code, so it will make testssl.sh run slightly faster.
As mentioned in #1931 the port detection for nmap greppable files
leaves space for improvements.
Ths PR adds a pattern detection of ssl and https in the forth or fifth
parameter of an open port, so those ports will be added to a scan when
a nmap greppable output file is as input to testssl.sh .
Also it does minor code adjustments to utils/gmap2testssl.sh .
This addresses a bug filed in #1935 in 3.1dev when the supplied file
has a .txt extension. In this scenario the input file was nulled
as from the input file in nmap format an internal input file was
generated which has a .txt extension, in the same directory.
The idea was to persist the file for the user.
Now, this internal input file is ephemeral and only written to $TEMPDIR.
In parse_cmd_line() error messages are usually printed to stderr, but in three places the messages are printed to stdout. This commit modifies those three lines so that they also print to stderr.
The commit also replaces a call to a non-existant function, tmln_magenta_term, with a call to tmln_magenta.
There was by mistake a 179 days threshold and also the error message
was wrong when HSTS was exactly set to 179 days.
This commit sets it to 180 days and corrects the error messages on
both screen and JSON.
This commit addresses a few issued related to the use of testssl.sh with OpenSSL 3.0.0-alpha14.
First, when pkey is used to print a DH key that uses an unknown group, OpenSSL 3.0.0-alpha14 labels the prime and generator using "P:" and "G:" rather than "prime:" and "generator:". (In PR #1586 it was noted that OpenSSL 3.0.0-alpha1 used "prime P:" and "generator G:". The x509 command in OpenSSL 3.0.0-alpha14 still uses "prime P:" and "generator G:" when printing a DH public key in a certificate, but the pkey command just uses "P:" and "G:").
Second, when the pkey command is used to print a DH key that uses certain common primes (e.g., groups from RFC 3526 or RFC 7919), OpenSSL 3.0.0-alpha14 simply prints "GROUP: " followed by a short name for the group rather than printing the value of the prime and generator.
Finally, the "-text" option no longer works if the input is a public key. Fortunately, the "-text_pub" option provides the expected results with all versions of OpenSSL and LibreSSL.
This commit makes some minor improvements to code that converts ASCII-HEX to binary.
First, testssl.sh currently has two functions that do the same thing: asciihex_to_binary() and hex2ascii(). This commit all calls to either of these functions with calls to hex2binary(), which is based on the current asciihex_to_binary().
This commit also changes direct use of printf to calls to hex2ascii() in generate-ccm-counter-blocks(), ccm-compute-tag(), and generate_gcm_counter().
Finally, this commit cleans up the implemention of hex2binary() a bit and introduces the option to use xxd, if that program is available. Using xxd rather than multiple calls to printf has a couple of advantages. xxd is a bit faster. However, the primary advantage is that when debugging (i.e., using bash -x), each call to hex2binary() only includes a few steps, regardless of how long the string is, whereas using printf the call to hex2binary() could take hundreds of steps.
Modify sclient_auth() to use checks similar to sclient_connect_successful() to determine whether the connection attempt was successful. Replace uses of awk and grep with Bash internals string comparisons.
This commit is a first step towards addressing #1709. It attempts to determime whether certificate-based client authentication is (1) not requested, (2) optional, or (3) required. If it is either optional or required, then it extracts the list of CA names (DNs) that the server sends in its CertificateRequest message.
The code for extracting the CA list from the CertificateRequest message seems to be working correctly. However, this commit is incomplete for a couple of reasons. First, it does not produce any new output, it just collects the information. Second, sclient_auth() needs some work.
The current sclient_auth() simply returns 0 if $OPENSSL returned 0. This may be okay if only trying to determine whether certificate-based client authentication is required. However, if it is optional, then the output will include "CertificateRequest", but $OPENSSL will return 0, since the connection was successful even though the client did not provide a certificates.
If $OPENSSL does not return 0, then sclient_auth() checks whether Master-Key is present. This works for TLS 1.2 and earlier, but not for TLS 1.3. So, sclient_auth() needs to be updated to work correctly with TLS 1.3.
The modified version of sclient_auth() will set CLIENT_AUTH and CLIENT_AUTH_CA_LIST for any version of TLS, but the remaining part of the code needs work. As I am not clear on the reason for this code, I need some help with it. Why does the code only look for "CertificateRequest" if "Master-Key" is present? Why is there a check for Session-ID in a function that is supposed to just be checking for client authentication. Why is CLIENT_AUTH set to false if SESSION-ID is absent (this is a no-op since CLIENT_AUTH would already have been false)?
See #1148 and #1805.
As noted in #1148, testssl.sh is not current designed to handle a mass testing file in which CSV, HTML, LOG, and/or JSON file names are provided in the mass testing file. If a child process receives a command line with one of the files, it assumes the same command-line option was provided to the parent so that the output of every test is being written to this one file. If this assumption is wrong, then either the file will not be created at all or it will be malformed since it will be missing header and/or footer information.
This PR partially addresses the problem by introducing new command-line arguments that are for internal use only. These command line arguments allow a child process to distinguish between a CSV, HTML, LOG, or JSON file that it is supposed to create itself versus one that is to be shared by all of the child processes.
There is one major limitation to this PR. The code for handle command-line arguments in the mass testing file is very simple and cannot handle whitespace characters, whether they are enclosed in quotes or are escaped. So, any file names included in the mass testing file cannot have whitespace characters.
... in order to be consistent with run_server_preference().
The wide formatting of other tests need some inspection and
off the top off my head are not as perfectly formatted so that
they should not run per default in wide mode.
This fixes#1779. There was a problem introduced in
3cd1273439 which counted
the size of the file name rather than the size of the
socket reply.
The helper function count_chars() is now not used anymore.
It maybe useful in the future though.
Sometimes it is needed to overwrite existing output files.
This has been requested in the past (#927). For safety reasons
it was not implemented.
However I realized that it could be useful. It requires some
responsible usage though.
Code added, help() and manpages added -- warnings added too.
While we are not sure yet how we deal with "other" colors and different
backgrounds users can have, I'll remove the light cyan here until we
settle on a standard. (other=not yellow,reds,brown,greens)
Despite the fact google doesn't support RC4 ciphers, testssl.sh called
sslv2_sockets(). Google answered with a >= TLS alert. Building a sum then
failed then in sslv2_sockets().
This fixes sslv2_sockets() and introduces count_chars() as a helper function
(tested also under old FreeBSD to make sure it works under MacOSX).
This fixes#1754 by avoiding further strings operations if the socket
reply is empty as bash 5.1 seems to have a problem with that. The fix
is done in sslv2_sockets() .
Also sslv2 is not being used in run_freak() if known not to be supported.
XMPP can be used with SNI in two contexts:
- Standard RFC 6120 STARTTLS-based connections; in that case, SNI
is most likely to be ignored, as XMPP uses another way to signal
the target domain name (via the @to attribute on the stream
header, which is already set correctly by testssl.sh). However,
setting SNI to a different value than the @to attribute may
lead to confusion.
- XEP-0368 (XMPP-over-TLS) connections which omit the STARTTLS
phase and go right for TLS (and inside that, XMPP). In that case,
SNI is obviously required to be correct. XEP-0368 specifies that
the SNI name MUST be the domain name of the service (not
necessarily the host name of the endpoint, thanks to SRV
records).
Hence, this patch forces the SNI name to be the --xmpphost value,
if --xmpphost is given. Note that it blatantly ignores whether
XMPP is used otherwise.
Travis failure was due to debug output in function which return a string.
The debug statement was removed, (stderr would have been choice \#2).
Issuer is heading now the intermediate certificate section, not
sure whethe this is redundant info.
* reorder sequence of checks in certificate info so that the chain relevant points are closer
together
* determine_cert_fingerprint_serial() doesn't need fil input anymore, thus removed that part
* cert_validityPeriod in JSON/CSV may lead to misunderstandings, thus renamed to cert_extlifeSpan
* reorganized loop for the intermediate certificate checks, so that also i is used and not the variable
which defines the number of certificates, i.e. certificates_provided. In addition made the counting
more hiuma friendly, which starts now at 1 instead of 0
* add cn and issuer_CN to the output both on screen and file
* the severity rating for intermediates are just a shot (20/40 days) and
deserve a second thought
* replace the expiry check by one test statement and make grep futile
* replace at some places "$openssl x509 -in $filename" by "$openssl x509 <<< $var"
* the thing with 25*60*60 was fie readability. When it's used >20 times it maybe is not
(and maybe costs to much time) --> replaced by $secsaday
* adjusted the loop for bad ocsp check for readability
* UI feed back for expiration date of intermediates: 20 days: HIGH, 40 days: MEDIUM
* also in JSON/CSV
* list the end date of validity
* works for >1 intermediates too
* section moved to the end of certificate_info()
* renamed <cert#${certificate_number}> --> <hostCert#${certificate_number}> to avoid coinfusion with intermediate certs
* removed blanks in return values of determine_dates_certificate
determine_dates_certificate() is now determining the important dates
of a certificate passed via argument. It works of course for host and
any other certificates.
Returning multiple parameters is being done via CSV and passed to a
read statement which seemed the best choice for bash.
ToDo:
* $expok is not set properly for intermediate certificates
* check if expired at least in the UI (JSON+CSV: echo the dates so far)
* for multiple host certificates the naming scheme (jsonID + intermediate
certnumnber kind of sucks:
"id" : "intermediate_cert_fingerprintSHA256 <cert#1> 1",
"id" : "intermediate_cert_notAfter <cert#2> 1",
The whole thing is kind of hackish as the code has been historically grown.
At some certian point we may want to reconsider how we determine properties of
certificates in certificate_info()
See #1683, #1653, #1004, #1264
* separate code for bad ocsp a bit
* output intermediate cert in json/csv
* replace sed statements from cert_fingerprint* and -serial by bash funcs
This commit adds
* a check for the elliptical curves
* and a check for TLS extensions
which will again reduces false positives.
Background:
* https://en.wikipedia.org/wiki/Comparison_of_TLS_implementations#Supported_elliptic_curves
* https://en.wikipedia.org/wiki/Comparison_of_TLS_implementations#Extensions
Also:
* Docu phrased more precise (we're not checking ciphers and
HTTP Server banner only
* As a last resort we also take 'Microsoft-HTTPAPI/2.0' as a server header on the HTTPS branch
and query the HTTP branch for Microsoft-IIS/8.x.
* $EXPERIMENTAL overrides some banner and service related checks. So that e.g. SMTP servers can also
be checked. Last but bot least ist's a vulnerability of the TLS stack.
For better debugging we'll keep the TLS extensions and offered curves in a file.
Also it adds a debug1() function which may be needed on other occasions.
Also the output is better coded as we put "check patches locally to confirm"
into a variable.
There's still room for improvement:
* More extensions (see https://raw.githubusercontent.com/cisco/joy/master/doc/using-joy-fingerprinting-00.pdf)
* We could need a separate determine_curves() function, see #1730 as otherwise
we can't use the curves in a non-default run.
Server side closed the connection but openssl retrieved
a zero exit code. In addition now we look for "closed"
and if that was returned from the server we label it
as not vulnerable.
This fixes#1725
This commit implements a detection of Winshock from 2014 (aka MS14-066, CVE-2014-6321).
It does that by analyzing
* the ciphers supported -- MS' rollup patch introduced new GCM ciphers
* AND grabbing the server banner which should match IIS 8.0 oder IIS 8.5
Admittedly this is not a strong detection. But it worked in the cases I tested
(no RDP yet). The other known method remotely testing for it against IIS is
using a patched openssl binary (see https://github.com/drwetter/testssl.sh/issues/331#issuecomment-211534954)
-- the diff "jules" (hi) provided a while back. That seems to stem from securitysift
albeit his decription was not complete and he didn't provide a PoC (I've
seen also polarssl + a little bit of python here: https://vimeo.com/112089813
The catch is securitysift's method, is not as trivial to implement and it dosses the
sass.exe process, see: http://www.securitysift.com/exploiting-ms14-066-cve-2014-6321-aka-winshock/.
* Todo: man page
This commit also removes -BB from the help. We haven't settled yet finally
where we go with short options for the cmd line for vulnerabilities. One
is for sure though: Using one letter uppercase doesn't scale. As winshock
can be executed with --WS and --winshock --BB brings that in line. For now
also -BB works (as -WS) but it isn't advertised anymore.
SERVICE global was previously set to $protocol which was
meant to set this for STARTTLS services. However it
was executes outside the corresponding if-statement.
This commit moves the statement where it belongs.
For not vulnerable hosts the low level starttls_* functions
returned an error when the STARTTLS injection was tested which
confused Travis/CI ( "Oops: STARTTLS handshake failed (code: 2)" )
* Ensured the random char generation worked under every OS supported
* Got POP3 and IMAP working
* always define SERVICE so that we can us it also for SMTP starttls injection
* fixed error in starttls_smtp_dialog where arg1 was taken as payload instead of arg2
* squashed error msg when killed socat or openssl process to avoid mess on screen
when processes already terminated
(* removed some redundant quotes at RHS if [[]] expressions)
todo:
* more tests for positives
* are tests for negatives sufficent? ("prove" is happy except one issue which
is probably not related but still need to understand)
For the record: t/25_baseline_starttls.t line 50 and 67:
"Oops: STARTTLS handshake failed (code: 2)"
This commit fixes#1699 by setting FIRST_FINDING to true in fileout_banner() if $do_json_pretty is true.
When $do_json_pretty is true, fileout_banner() calls fileout_pretty_json_banner(), which starts a new sectio in the JSON file. Setting FIRST_FINDING to true ensures that a comma is not placed before the first entry in this new section. This is the same as is done in other places when a new section is stated: fileout_section_header() and fileout_insert_warning().
