As noted in #1481, testssl.sh has a problem with printing percent ('%') characters.
At one point, the function out() was implemented as `/usr/bin/printf -- "${1//%/%%}"`. When this was the case, any '%' needed to be replaced with '%%' since '$1' was being used as the format string. This was changed, however, by 8a2fe5915a. Since the format string is now "%b" rather than '$1', the replacement is not needed anymore. Instead, the replacement now causes any '%' to be printed to be duplicated.
This problem does not happen very often, but does sometimes occur when a '%' character appears in a URI, such as in an HTTP redirect, a certificate revocation list, or an OCSP URI.
.. as it may not be available everywhere, see #1521 (NixOS).
This commit replaces all instances from pwd or /bin/pwd by $PWD.
It is a bash internal and the fastest. Also it added some quotes
to PWD a it may contain white spaces in the future (currently
there's a check for it that it won't)
.. as it may not be everywhere available, see #1521 (NixOS).
This commit replaces all instances from pwd or /bin/pwd by `pwd -P`
(-P -> no symbolic link)
This PR fixes two issues related to the generation of HTML files.
First, text that is to appear in the HTML file is first passed through html_reserved() to replace reserved characters with their corresponding entity names (e.g., '>' becomes '>'). html_reserved() seems to work correctly on Ubuntu Linux, but it does not work as expected on MacOS. On MacOS, rather than converting '>' to '>', it gets converted to '\>', and the backslash is rendered by browsers.
This PR appears to fix the problem. However, given that the original version of html_reserved() was not portable, this revised version should be tested on multiple platforms.
I also noticed that in almost every case in which a string is passed to html_out(), it is first run through html_reserved(), but for some reason that is not the case in out() and outln(). I can't see any reason why html_reserved() is not called first in these two cases, so this PR adds in the calls.
This commit fixes two minor issues related to HSTS_MIN:
* If there is a misconfiguration the recommended max-age should be based on $HSTS_MIN rather than being hardcoded to 15552000 seconds = 180 days.
* If max-age is exactly $HSTS_MIN, testssl.sh shouldn't say that max-age is too short while also say that >= $HSTS_MIN seconds is recommended.
This commit addresses two bugs: #1506 and #1508.
First, the variable rDNS can contain multiple lines due to multiple PTR DNS
records, though this is not recommended. In those cases the multiple PTR DNS
were concatenated on the screen, without any blank.
Secondly - depending on the name server entries and on the output of the DNS
binaries used it can contain non-printable characters or characters which are
printable but later on interpreted on the output device (\032 was mentioned
in #1506) which on the screen was interpreted as octal 32 (decimal 26 = ▒,
try echo "\032"), so basically a terminal escape sequence was smuggled
from the DNS server to the screen of the users. In JSON pretty output we
had also this escape sequence which was fine for jsonlint but caused jq
to hiccup.
Fix: we use a loop to check for each FQDN returned. There we remove chars which
under those circumstances can show up. The blacklist is taken from RFC 1912
("Allowable characters in a label for a host name are only ASCII, letters, digits,
and the `-' character").
Since, in cases in which the server enforces a cipher order, both run_cipher_per_proto() and run_server_preference() list every cipher supported by the server for each protocol, there was a discussion at one point about eliminating run_cipher_per_proto() and extending run_server_preference().
This PR takes a step in that direction by providing the option to present the "Cipher order" in wide mode.
This PR fixes one Shellcheck issue:
In testssl_3.1dev_20200208.sh line 2395:
HEADERVALUE="$(fgrep -Fai "$key:" $HEADERFILE | head -1)"
^-- SC2197: fgrep is non-standard and deprecated. Use grep -F instead.
The man page for grep states that fgrep is the same a grep -F and that grep is deprecated. So, fgrep -F is just redundant.
Currently, the function neat_list() uses the variable "export", but does not define it. The result is that "export" variable in the calling function is used.
This PR fixes that by defining "export" as a local variable in neat_list() and by setting its value via a new parameter to the function.
This PR also removes a "FIXME" from run_rc4() since the problem has already been fixed.
This PR fixes a minor bug in get_pub_key_size(). If the key size is being determined manually and length encoding requires 4 bytes, then the current code computes the length incorrectly. This is a very insignificant bug, since does not apply to RSA or ECC keys, and the key would have to be at least 16 megabytes long for it to require 4 bytes to encode.
This PR also cleans up get_pub_key_size() a bit by replacing `i=$i+...` with `i+=...` and by enclosing math in `$(( ... ))`.
Hostnames can contain a trailing dot (and sometimes they should).
If they are supplied to testssl.sh however they will be also interpreted
as a URL PATH when the servive is HTTP.
This commit fixes that.
This switch had no effect. There was probably a regression
problem as it worked before.
Besides fixing that the large case statement in parse_cmd_line()
was simplified, in a sense that banner and help functions were
moved to a separate case statement.
This PR adds support for post-handshake messages when using sockets with TLS 1.3 connections. If a TLS 1.3 connection is established and the connection is to remain open after tls_sockets() finishes, then after the client's Finished message is sent the master secret and the application traffic keys are computed. This PR also adds two new functions to send and receive application data over a TLS 1.3 connection.
This PR also includes two proofs-of-concept for the use of the new functions. receive_app_data() is called immediately after the client's Finished message is sent. Some server's will send new session tickets immediately after the handshake is complete. If they do, then the code will decrypt and parse the session ticket messages.
This PR also modifies service_detection() to try using sockets if the server only supports TLS 1.3 and $OPENSSL does not support TLS 1.3. After the handshake is complete, this code sends an HTTP GET request and reads the response. The code is fairly slow and it doesn't always work. However, since it is only used in cases in which $OPENSSL cannot work, it can't hurt to try using sockets.
Due to a bug, the shellcheck program will complain if a variable is defined as an array but is later used as an ordinary string, even if the two uses are locally defined variables in different contexts. The error message is:
SC2178: Variable was used as an array but is now assigned a string.
While the warnings are not highlighting any actual problems in testssl.sh, this PR gets rid of the warnings by renaming a few variables.
The implementation of AES-GCM in #1473 is much slower than the original version, even when the authentication tag is not being computed. This PR modifies the code in gcm() in order to significantly speed up the encryption/decryption time (when authentication tags are not being computed).
This PR adds processing of the Finished messages in TLS 1.3 handshakes. It also addresses some shellcheck issues.
If in debug mode, the HMAC of the transcript hash of the handshake context ($msg_transcript) is computed and compared against the Finished message sent by the server.
If the full server response is parsed and the connection with the server is not to be closed when tls_sockets() completes, then the TLS 1.3 handshake is completed by creating the client Finished message and sending it to the server.
This PR reorganizes the code for deriving TLS 1.3 symmetric keys in order to facilitate implementing the full key schedule. For example, rather than having a single function to derive the handshake traffic keys, this PR creates one function to derive the handshake secret and a separate function to derive the handshake traffic keys. The second function has been generalized so that it can derive either client or server traffic keys. Separating into two functions also makes the handshake_secret available for later use to derive the master secret and then the application traffic secrets and the application traffic keys.
This PR also changes where there message transcript is created, a message transcript will also be needed to derive the application traffic secrets. This PR includes the code to add the messages to the initial message transcript that will be needed for the input to the application traffic secret derivation function.
RFC 8446 specifies cipher suites that use three symmetric encryption algorithms, all of which are Authenticated Encryption with Associated Data (AEAD) algorithms. In each of these algorithms when data is encryption an authentication tag is created, which allows the recipient to verify that the data has not been modified. The authentication may also cover some additional data that was not encrypted.
The current implementations of these algorithms in testssl.sh decrypt the ciphertext, but do not check that the authentication tag is correct (which involves the recipient computing the correct tag for the received data and then comparing it to the provided tag). While testssl.sh can get away with not checking authentication tags when receiving data, the ability to compute authentication tags is needed in order to send encrypted data as TLS servers would reject any encrypted data that did not have a correct authentication tag. Being able to send encrypted data is necessary to be able to complete the TLS 1.3 handshake.
This PR replaces the current implementations of the symmetric encryption algorithms with full implementations of each of the algorithms. These full implementations include the ability to encrypt data for sending, and can also verify the authentication tag when decrypting data. Since the Bash implementations of these algorithms is very slow, the decryption code is designed to only compute and check authentication tags in debug mode.
While the implementation of the code to compute authentication tags for AES-CCM was based on NIST Special Publication 800-38C, I was not able to implement the code for AES-GCM or Poly1305 from their specifications (NIST Special Publication 800-38D and RFC 8439, respectively). So, I would very much like to thank the implementers of https://github.com/mko-x/SharedAES-GCM and https://github.com/floodyberry/poly1305-donna. The implementations of AES-GCM and Poly1305 in the PR were developed by translating the C code in https://github.com/mko-x/SharedAES-GCM and https://github.com/floodyberry/poly1305-donna into Bash. I don't understand what that code is doing, but it seems to work. :-)
I have only tested this code on a computer with a 64-bit operating system. While I have not tested it, I believe that the decryption code will work with 32-bit integers if not in debug mode (i.e., if not trying to compute the authentication tags). I also believe that the AES-CCM code for computing authentication tags will work with 32-bit integers. However, AES-GCM and Poly1305 code for computing authentication tags will definitely only work on systems that have 64-bit integers. So, on systems that do not have 64-bit integers, encryption will not work for AES-GCM or ChaCha20-Poly1305, and decryption will not work for these algorithms if in debug mode.
For a fresh start it seemed a good idea to cleanup
the order of functions and some variables so that
those with the same functionality are somewhat grouped.
Some of the functions have now a header and a foooter
to make it easier to spot and use then. Also for added future
functions the hope is that they will be put where they better
fit
This PR simplifies the code for determining which draft version of TLS 1.3 a server is offering by making use of a simple regular expression and $BASH_REMATCH rather than looping through every possible draft version.
PR #1463 changed run_ssl_poodle() to only run the test if it is known that the server supports SSLv3. However, support for SSLv3 may be unknown at the time run_ssl_poodle() is run (e.g., if the server supports TLS 1 and SSLv3, and run_ssl_poodle() is the first test performed). So, run_ssl_poodle() should perform testing unless it is known that SSLv3 is not supported.
There's a check for >825 days certificate lifetime. That
check emits a debug statement when the lifetime is within
this limit. It does that also when the certificate expired.
This commit adds now the word "total"
DEBUG: all is fine with total certificate life time
to make sure the life time left not is what should be understood.
Several vulnerability checks add a time penalty when the server
side only support TLS 1.3 as The TLS 1.3 RFC 8446 and implementations
known so far don't support the flaws being checked for.
This PR adds "shortcut" checks for all TLS 1.3, assuming that the
TLS 1.3 implementation is correct which seems at this time a valid
assumpution. That either saves a TCP connect or at least some logic to
be executed. Also in some cases a TLS 1.3 only server emitted unnecessary
warnings, see #1444.
If $DEBUG -eq 1 then it outputs information that a shortcut was
used. It doesn't do that in other cases because the screen output
seems too obtrusive.
It also adds a shortcut for beast when SSL 3 or TLS 1.0 is is known
not to be supported.
This commit radds 747fb039ed which
was accidenially reverted in 45f28d8166.
It fixes#1462.
See also #1459.
