In parse_cmd_line() error messages are usually printed to stderr, but in three places the messages are printed to stdout. This commit modifies those three lines so that they also print to stderr.
The commit also replaces a call to a non-existant function, tmln_magenta_term, with a call to tmln_magenta.
There was by mistake a 179 days threshold and also the error message
was wrong when HSTS was exactly set to 179 days.
This commit sets it to 180 days and corrects the error messages on
both screen and JSON.
This commit addresses a few issued related to the use of testssl.sh with OpenSSL 3.0.0-alpha14.
First, when pkey is used to print a DH key that uses an unknown group, OpenSSL 3.0.0-alpha14 labels the prime and generator using "P:" and "G:" rather than "prime:" and "generator:". (In PR #1586 it was noted that OpenSSL 3.0.0-alpha1 used "prime P:" and "generator G:". The x509 command in OpenSSL 3.0.0-alpha14 still uses "prime P:" and "generator G:" when printing a DH public key in a certificate, but the pkey command just uses "P:" and "G:").
Second, when the pkey command is used to print a DH key that uses certain common primes (e.g., groups from RFC 3526 or RFC 7919), OpenSSL 3.0.0-alpha14 simply prints "GROUP: " followed by a short name for the group rather than printing the value of the prime and generator.
Finally, the "-text" option no longer works if the input is a public key. Fortunately, the "-text_pub" option provides the expected results with all versions of OpenSSL and LibreSSL.
This commit makes some minor improvements to code that converts ASCII-HEX to binary.
First, testssl.sh currently has two functions that do the same thing: asciihex_to_binary() and hex2ascii(). This commit all calls to either of these functions with calls to hex2binary(), which is based on the current asciihex_to_binary().
This commit also changes direct use of printf to calls to hex2ascii() in generate-ccm-counter-blocks(), ccm-compute-tag(), and generate_gcm_counter().
Finally, this commit cleans up the implemention of hex2binary() a bit and introduces the option to use xxd, if that program is available. Using xxd rather than multiple calls to printf has a couple of advantages. xxd is a bit faster. However, the primary advantage is that when debugging (i.e., using bash -x), each call to hex2binary() only includes a few steps, regardless of how long the string is, whereas using printf the call to hex2binary() could take hundreds of steps.
Modify sclient_auth() to use checks similar to sclient_connect_successful() to determine whether the connection attempt was successful. Replace uses of awk and grep with Bash internals string comparisons.
This commit is a first step towards addressing #1709. It attempts to determime whether certificate-based client authentication is (1) not requested, (2) optional, or (3) required. If it is either optional or required, then it extracts the list of CA names (DNs) that the server sends in its CertificateRequest message.
The code for extracting the CA list from the CertificateRequest message seems to be working correctly. However, this commit is incomplete for a couple of reasons. First, it does not produce any new output, it just collects the information. Second, sclient_auth() needs some work.
The current sclient_auth() simply returns 0 if $OPENSSL returned 0. This may be okay if only trying to determine whether certificate-based client authentication is required. However, if it is optional, then the output will include "CertificateRequest", but $OPENSSL will return 0, since the connection was successful even though the client did not provide a certificates.
If $OPENSSL does not return 0, then sclient_auth() checks whether Master-Key is present. This works for TLS 1.2 and earlier, but not for TLS 1.3. So, sclient_auth() needs to be updated to work correctly with TLS 1.3.
The modified version of sclient_auth() will set CLIENT_AUTH and CLIENT_AUTH_CA_LIST for any version of TLS, but the remaining part of the code needs work. As I am not clear on the reason for this code, I need some help with it. Why does the code only look for "CertificateRequest" if "Master-Key" is present? Why is there a check for Session-ID in a function that is supposed to just be checking for client authentication. Why is CLIENT_AUTH set to false if SESSION-ID is absent (this is a no-op since CLIENT_AUTH would already have been false)?
See #1148 and #1805.
As noted in #1148, testssl.sh is not current designed to handle a mass testing file in which CSV, HTML, LOG, and/or JSON file names are provided in the mass testing file. If a child process receives a command line with one of the files, it assumes the same command-line option was provided to the parent so that the output of every test is being written to this one file. If this assumption is wrong, then either the file will not be created at all or it will be malformed since it will be missing header and/or footer information.
This PR partially addresses the problem by introducing new command-line arguments that are for internal use only. These command line arguments allow a child process to distinguish between a CSV, HTML, LOG, or JSON file that it is supposed to create itself versus one that is to be shared by all of the child processes.
There is one major limitation to this PR. The code for handle command-line arguments in the mass testing file is very simple and cannot handle whitespace characters, whether they are enclosed in quotes or are escaped. So, any file names included in the mass testing file cannot have whitespace characters.
... in order to be consistent with run_server_preference().
The wide formatting of other tests need some inspection and
off the top off my head are not as perfectly formatted so that
they should not run per default in wide mode.
This fixes#1779. There was a problem introduced in
3cd1273439 which counted
the size of the file name rather than the size of the
socket reply.
The helper function count_chars() is now not used anymore.
It maybe useful in the future though.
Sometimes it is needed to overwrite existing output files.
This has been requested in the past (#927). For safety reasons
it was not implemented.
However I realized that it could be useful. It requires some
responsible usage though.
Code added, help() and manpages added -- warnings added too.
While we are not sure yet how we deal with "other" colors and different
backgrounds users can have, I'll remove the light cyan here until we
settle on a standard. (other=not yellow,reds,brown,greens)
Despite the fact google doesn't support RC4 ciphers, testssl.sh called
sslv2_sockets(). Google answered with a >= TLS alert. Building a sum then
failed then in sslv2_sockets().
This fixes sslv2_sockets() and introduces count_chars() as a helper function
(tested also under old FreeBSD to make sure it works under MacOSX).
This fixes#1754 by avoiding further strings operations if the socket
reply is empty as bash 5.1 seems to have a problem with that. The fix
is done in sslv2_sockets() .
Also sslv2 is not being used in run_freak() if known not to be supported.
XMPP can be used with SNI in two contexts:
- Standard RFC 6120 STARTTLS-based connections; in that case, SNI
is most likely to be ignored, as XMPP uses another way to signal
the target domain name (via the @to attribute on the stream
header, which is already set correctly by testssl.sh). However,
setting SNI to a different value than the @to attribute may
lead to confusion.
- XEP-0368 (XMPP-over-TLS) connections which omit the STARTTLS
phase and go right for TLS (and inside that, XMPP). In that case,
SNI is obviously required to be correct. XEP-0368 specifies that
the SNI name MUST be the domain name of the service (not
necessarily the host name of the endpoint, thanks to SRV
records).
Hence, this patch forces the SNI name to be the --xmpphost value,
if --xmpphost is given. Note that it blatantly ignores whether
XMPP is used otherwise.
Travis failure was due to debug output in function which return a string.
The debug statement was removed, (stderr would have been choice \#2).
Issuer is heading now the intermediate certificate section, not
sure whethe this is redundant info.
* reorder sequence of checks in certificate info so that the chain relevant points are closer
together
* determine_cert_fingerprint_serial() doesn't need fil input anymore, thus removed that part
* cert_validityPeriod in JSON/CSV may lead to misunderstandings, thus renamed to cert_extlifeSpan
* reorganized loop for the intermediate certificate checks, so that also i is used and not the variable
which defines the number of certificates, i.e. certificates_provided. In addition made the counting
more hiuma friendly, which starts now at 1 instead of 0
* add cn and issuer_CN to the output both on screen and file
* the severity rating for intermediates are just a shot (20/40 days) and
deserve a second thought
* replace the expiry check by one test statement and make grep futile
* replace at some places "$openssl x509 -in $filename" by "$openssl x509 <<< $var"
* the thing with 25*60*60 was fie readability. When it's used >20 times it maybe is not
(and maybe costs to much time) --> replaced by $secsaday
* adjusted the loop for bad ocsp check for readability
* UI feed back for expiration date of intermediates: 20 days: HIGH, 40 days: MEDIUM
* also in JSON/CSV
* list the end date of validity
* works for >1 intermediates too
* section moved to the end of certificate_info()
* renamed <cert#${certificate_number}> --> <hostCert#${certificate_number}> to avoid coinfusion with intermediate certs
* removed blanks in return values of determine_dates_certificate
determine_dates_certificate() is now determining the important dates
of a certificate passed via argument. It works of course for host and
any other certificates.
Returning multiple parameters is being done via CSV and passed to a
read statement which seemed the best choice for bash.
ToDo:
* $expok is not set properly for intermediate certificates
* check if expired at least in the UI (JSON+CSV: echo the dates so far)
* for multiple host certificates the naming scheme (jsonID + intermediate
certnumnber kind of sucks:
"id" : "intermediate_cert_fingerprintSHA256 <cert#1> 1",
"id" : "intermediate_cert_notAfter <cert#2> 1",
The whole thing is kind of hackish as the code has been historically grown.
At some certian point we may want to reconsider how we determine properties of
certificates in certificate_info()
See #1683, #1653, #1004, #1264
* separate code for bad ocsp a bit
* output intermediate cert in json/csv
* replace sed statements from cert_fingerprint* and -serial by bash funcs
This commit adds
* a check for the elliptical curves
* and a check for TLS extensions
which will again reduces false positives.
Background:
* https://en.wikipedia.org/wiki/Comparison_of_TLS_implementations#Supported_elliptic_curves
* https://en.wikipedia.org/wiki/Comparison_of_TLS_implementations#Extensions
Also:
* Docu phrased more precise (we're not checking ciphers and
HTTP Server banner only
* As a last resort we also take 'Microsoft-HTTPAPI/2.0' as a server header on the HTTPS branch
and query the HTTP branch for Microsoft-IIS/8.x.
* $EXPERIMENTAL overrides some banner and service related checks. So that e.g. SMTP servers can also
be checked. Last but bot least ist's a vulnerability of the TLS stack.
For better debugging we'll keep the TLS extensions and offered curves in a file.
Also it adds a debug1() function which may be needed on other occasions.
Also the output is better coded as we put "check patches locally to confirm"
into a variable.
There's still room for improvement:
* More extensions (see https://raw.githubusercontent.com/cisco/joy/master/doc/using-joy-fingerprinting-00.pdf)
* We could need a separate determine_curves() function, see #1730 as otherwise
we can't use the curves in a non-default run.
Server side closed the connection but openssl retrieved
a zero exit code. In addition now we look for "closed"
and if that was returned from the server we label it
as not vulnerable.
This fixes#1725
This commit implements a detection of Winshock from 2014 (aka MS14-066, CVE-2014-6321).
It does that by analyzing
* the ciphers supported -- MS' rollup patch introduced new GCM ciphers
* AND grabbing the server banner which should match IIS 8.0 oder IIS 8.5
Admittedly this is not a strong detection. But it worked in the cases I tested
(no RDP yet). The other known method remotely testing for it against IIS is
using a patched openssl binary (see https://github.com/drwetter/testssl.sh/issues/331#issuecomment-211534954)
-- the diff "jules" (hi) provided a while back. That seems to stem from securitysift
albeit his decription was not complete and he didn't provide a PoC (I've
seen also polarssl + a little bit of python here: https://vimeo.com/112089813
The catch is securitysift's method, is not as trivial to implement and it dosses the
sass.exe process, see: http://www.securitysift.com/exploiting-ms14-066-cve-2014-6321-aka-winshock/.
* Todo: man page
This commit also removes -BB from the help. We haven't settled yet finally
where we go with short options for the cmd line for vulnerabilities. One
is for sure though: Using one letter uppercase doesn't scale. As winshock
can be executed with --WS and --winshock --BB brings that in line. For now
also -BB works (as -WS) but it isn't advertised anymore.
SERVICE global was previously set to $protocol which was
meant to set this for STARTTLS services. However it
was executes outside the corresponding if-statement.
This commit moves the statement where it belongs.
For not vulnerable hosts the low level starttls_* functions
returned an error when the STARTTLS injection was tested which
confused Travis/CI ( "Oops: STARTTLS handshake failed (code: 2)" )
* Ensured the random char generation worked under every OS supported
* Got POP3 and IMAP working
* always define SERVICE so that we can us it also for SMTP starttls injection
* fixed error in starttls_smtp_dialog where arg1 was taken as payload instead of arg2
* squashed error msg when killed socat or openssl process to avoid mess on screen
when processes already terminated
(* removed some redundant quotes at RHS if [[]] expressions)
todo:
* more tests for positives
* are tests for negatives sufficent? ("prove" is happy except one issue which
is probably not related but still need to understand)
For the record: t/25_baseline_starttls.t line 50 and 67:
"Oops: STARTTLS handshake failed (code: 2)"
This commit fixes#1699 by setting FIRST_FINDING to true in fileout_banner() if $do_json_pretty is true.
When $do_json_pretty is true, fileout_banner() calls fileout_pretty_json_banner(), which starts a new sectio in the JSON file. Setting FIRST_FINDING to true ensures that a comma is not placed before the first entry in this new section. This is the same as is done in other places when a new section is stated: fileout_section_header() and fileout_insert_warning().
* todo: parse the return strings for detection
* test ;-)
* check whether the random char thing works under every OS supported
* definition of five_random var
run_starttls_injection():
* kill background openssl process when not needed anymore
* kill background socat process when not needed anymore
* close socket
add line in help() for STARTTLS injection
Furthermore:
* for close_socket() calls always add the fd (atm not needed)
* in help() rather advertise --BB instead of -BB
It's more a PoC style and needs some work
* use $OPENSSL or $OPENSSL2
* remove exit 0
* put run_starttls_injection below run_rc4
* test with more vulnerable servers
debugme1() was defined
This is a small cleanup of find_openssl_binary(). It tries also
to find a newer openssl version which we could need for any
new features. As stated in the comment at some point we should
decide whether we stick with our old version or rather supply
a new one. (xmpp-server is also not builtin for our 1.0.2) or
maybe find a good way (code) how to use both.
Also it looks for socat and if found it populates the according
global var.
It does a minor resort of global vars in the beginning.
This PR fixes#1671.
Primarily there's now an additional case statement in the main while loop
which just calls fatal() when it detects --help -b --banner -v or --version.
The documentation was also updated to reflect that.
(Some grammar and other errors which I stumbled over were corrected too)
This fixes#1648.
Java store doesn't seem to be as complete. No downgrading of trust rating
to T but we still need to raise a red flag for some Java clients
This PR will replace #1566. It addresses that if the server side doesn't show STARTTLS
testssl.sh should exit and label it accordingly (see #1536).
For this to achieve starttls_just_send() was were changed so that a return value from of 3
signals the STARTTLS pattern wasn't found is passed back to the parent fd_socket() whcih
will then act accordingly.
Also:
* starttls_full_read() + starttls_just_send() were improved for readability and debugging.
* The caller of starttls_full_read() + starttls_just_send() had redundant indentations which were moved to the callee
* minor bugs were squashed (e.g. ``fd_socket()``'s return values =!0 always were referring to STARTTLS also when no STARTTLS was requested)
This was tested (negative + test and positive) for FTP and SMTP which worked as expected. For POP, IMAP and NNTP it should work
accordingly but I had trouble finding a server whcih DID NOT support STARTTLS.
All other protocols basically should also cause testssl.sh to bail out but haven't been tested either. However
here starttls_io() won't return 3. It returns 1 in a case of problems. It uses NR_STARTTLS_FAIL. If it's encountered 2+
times that STARTTLS fails it early exists using fatal(). So we maybe want to consider changing starttls_io() in the future
to also use return 3 in the case STARTTLS is not offered.
This commit modifies a few functions to use fewer external function calls. In most cases this involves replacing external function calls with Bash internal functions, but in one case it involves replacing multiple external function calls with one call to awk.
This commit makes a few changes to the way that some functions work.
is_ipv4addr() and is_ipv6addr() will now strictly only accept a string that is an IPv4 (or IPv6) address and nothing else.
A couple of changes were also made to match_ipv4_httpheader(). First, lines that match $excluded_header (formerly $whitelisted_header) are not processed in the while loop. This prevents the excluded header from being output in the case that $HEADERFILE includes a non-excluded header with an IPv4 address and an excluded header with a string that looks like an IPv4 address.
The list of excluded headers was also modified to exclude any line that begins "Server: " rather than just lines that begin "Server: PRTG". According to https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server, the "Server" header describes the software used by the server, so it seems reasonable to expect that this header line will never contain an IPv4 address. Also, looking at some old test results I found cases in which Oracle software version numbers in the Server header were mistakenly matched as IPv4 addresses.
The commit updates the test for the "Visibility Information" certificate extension used in the ETSI Enterprise Transport Security protocol.
The main change is to support OpenSSL 3.0.0, which prints more information about subject alternative names that are encoded as otherName. For otherName types for the OpenSSL has no information, it prints "otherName: <OID>::<unsupported>" rather than just "otherName: <unsupported>". So, testssl.sh needs to account for the possibility that the OID for the visibility information certificate extension will be printed.
This commit also updates the reference for this extension and changes the name of the function from etsi_etls_visibility_info() to etls_ets_visibility_info() since the name of the protocol was changed from Enterprise TLS (eTLS) to Enterprise Transport Security (ETS).
This commit does not change the output to the terminal or to JSON/CSV, even though those outputs use the previous name of eTLS rather than ETS.
The ALPN GREASE test was written based on draft-ietf-tls-grease-01, which reserved all ALPN identifiers beginning with the prefix "ignore/". This commit changes the test to align with RFC 8701, which instead reserves {0x0A,0x0A}, {0x1A,0x1A}, ..., {0xFA,0xFA}.
Improved:
* readability of my old code
* readability of debugging statements
* honor $SNEAKY for SMTP greeting
* hook (arg2 to starttls_smtp_dialog() ), if we plan to add / replace SMTP greeting at some point
