This commit fixes ticketbleed so that using socksend_clienthello().
can being used.
The function for retrieving the TLS session ticket is now using SNI
and it was renamed to session_ticket_from_openssl() so that this
can be used elsewhere. Also for the sake of better programming
it is using bash only.
In order to ease stripping whitespaces the bash option "extglob"
was IN GENERAL set. This should only add the possibility to do
extended pattern matching when using round brackets:
?(pattern-list)
Matches zero or one occurrence of the given patterns.
*(pattern-list)
Matches zero or more occurrences of the given patterns.
+(pattern-list)
Matches one or more occurrences of the given patterns.
@(pattern-list)
Matches one of the given patterns.
!(pattern-list)
Matches anything except one of the given patterns.
... see bash(1). The man page though warns "separate matches against
shorter strings, or using arrays of strings instead of a single long
string, may be faster.". So when using ~100x we should do s.th. else.
It also works under bashv3.
The check_bytestream() function which was previously introduced now
also list the offending string.
This moves the run_ticketbleed function to the socketsend_clienthello.
It is not working yet, see also #1535 why. This is just for the PoC,
I'll explain:
It has now a function named check_bytestream() which will be called
in debug mode 1 and checks whether the byte stream to be send via
bash sockets is properly formatted. It can detect bugs which otherwise
would be hard to discover.
DO NOT USE IT for anything else than the check
---snip:
code:
check_bytestream() {
local line=""
local -i i=0
# We do a search and replace so that \xaa\x29 becomes
# _xaa
# _x29
#
# "echo -e" helps us to get a multiline string
while read -r line; do
if [[ $i -eq 0 ]]; then
# first line is empty because this is a LF
:
elif [[ ${#line} -ne 4 ]] && [[ $i != 0 ]]; then
echo "length of byte $i called from $2 is not ok"
elif [[ ${line:0:1} != _ ]]; then
echo "char $i called from $2 doesn't start with a \"\\\""
elif [[ ${line:1:1} != x ]]; then
echo "char $i called from $2 doesn't have an x in second position"
elif [[ ${line:2:2} != [0-9a-fA-F][0-9a-fA-F] ]]; then
echo "byte $i called from $2 is not hex"
fi
i+=1
done < <( echo -e ${1//\\/\\n_})
}
socksend_clienthello() {
local data=""
code2network "$1"
data="$NW_STR"
if [[ "$DEBUG" -ge 1 ]]; then
check_bytestream "$data" "${FUNCNAME[1]}"
[[ "$DEBUG" -ge 4 ]] && echo && echo "\"$data\""
[..]
Result (./testssl.sh -q --debug=1 -U dev.testssl.sh):
Testing vulnerabilities
Heartbleed (CVE-2014-0160) not vulnerable (OK), no heartbeat extension
CCS (CVE-2014-0224) not vulnerable (OK)
Ticketbleed (CVE-2016-9244), experiment. length of byte 311 called from run_ticketbleed is not ok
length of byte 312 called from run_ticketbleed is not ok
length of byte 313 called from run_ticketbleed is not ok
length of byte 314 called from run_ticketbleed is not ok
length of byte 315 called from run_ticketbleed is not ok
length of byte 316 called from run_ticketbleed is not ok
length of byte 317 called from run_ticketbleed is not ok
[..]
---snap
Besides that:
* dec02hex was corrected (only being used for run_ticketbleed)
* dec04hex is still buggy and part of the problem
* some quotes removed from rhs of [[]]
This commit chamges a few functions / calls so that
the hexbyte syntax with leading x was changed to
one without. The calls then need to change from
socksend --> socksend_clienthello .
The goal is basically to remove socksend() at
some point. Also socksend_clienthello()'s use
of NW_STR should be reconsidered.
This PR removes also some blanks, at the right
hand side of some double square brackets and
at some empty lines
This function had before a mixture of sed and tr commands
which was now replaced by bash internal functions. It makes
the code better, performance gain in the LAN is neglectable (1s).
This brings code2network somewhat in line with socksend(). This
function does basically the same (and thus is probably prone
to extinction ;-) ). Albeit there the good thing is it does
conversion and sending in one shot.
As noted in #1481, testssl.sh has a problem with printing percent ('%') characters.
At one point, the function out() was implemented as `/usr/bin/printf -- "${1//%/%%}"`. When this was the case, any '%' needed to be replaced with '%%' since '$1' was being used as the format string. This was changed, however, by 8a2fe5915a. Since the format string is now "%b" rather than '$1', the replacement is not needed anymore. Instead, the replacement now causes any '%' to be printed to be duplicated.
This problem does not happen very often, but does sometimes occur when a '%' character appears in a URI, such as in an HTTP redirect, a certificate revocation list, or an OCSP URI.
.. as it may not be available everywhere, see #1521 (NixOS).
This commit replaces all instances from pwd or /bin/pwd by $PWD.
It is a bash internal and the fastest. Also it added some quotes
to PWD a it may contain white spaces in the future (currently
there's a check for it that it won't)
.. as it may not be everywhere available, see #1521 (NixOS).
This commit replaces all instances from pwd or /bin/pwd by `pwd -P`
(-P -> no symbolic link)
This PR fixes two issues related to the generation of HTML files.
First, text that is to appear in the HTML file is first passed through html_reserved() to replace reserved characters with their corresponding entity names (e.g., '>' becomes '>'). html_reserved() seems to work correctly on Ubuntu Linux, but it does not work as expected on MacOS. On MacOS, rather than converting '>' to '>', it gets converted to '\>', and the backslash is rendered by browsers.
This PR appears to fix the problem. However, given that the original version of html_reserved() was not portable, this revised version should be tested on multiple platforms.
I also noticed that in almost every case in which a string is passed to html_out(), it is first run through html_reserved(), but for some reason that is not the case in out() and outln(). I can't see any reason why html_reserved() is not called first in these two cases, so this PR adds in the calls.
This commit fixes two minor issues related to HSTS_MIN:
* If there is a misconfiguration the recommended max-age should be based on $HSTS_MIN rather than being hardcoded to 15552000 seconds = 180 days.
* If max-age is exactly $HSTS_MIN, testssl.sh shouldn't say that max-age is too short while also say that >= $HSTS_MIN seconds is recommended.
This commit addresses two bugs: #1506 and #1508.
First, the variable rDNS can contain multiple lines due to multiple PTR DNS
records, though this is not recommended. In those cases the multiple PTR DNS
were concatenated on the screen, without any blank.
Secondly - depending on the name server entries and on the output of the DNS
binaries used it can contain non-printable characters or characters which are
printable but later on interpreted on the output device (\032 was mentioned
in #1506) which on the screen was interpreted as octal 32 (decimal 26 = ▒,
try echo "\032"), so basically a terminal escape sequence was smuggled
from the DNS server to the screen of the users. In JSON pretty output we
had also this escape sequence which was fine for jsonlint but caused jq
to hiccup.
Fix: we use a loop to check for each FQDN returned. There we remove chars which
under those circumstances can show up. The blacklist is taken from RFC 1912
("Allowable characters in a label for a host name are only ASCII, letters, digits,
and the `-' character").
Since, in cases in which the server enforces a cipher order, both run_cipher_per_proto() and run_server_preference() list every cipher supported by the server for each protocol, there was a discussion at one point about eliminating run_cipher_per_proto() and extending run_server_preference().
This PR takes a step in that direction by providing the option to present the "Cipher order" in wide mode.
This PR fixes one Shellcheck issue:
In testssl_3.1dev_20200208.sh line 2395:
HEADERVALUE="$(fgrep -Fai "$key:" $HEADERFILE | head -1)"
^-- SC2197: fgrep is non-standard and deprecated. Use grep -F instead.
The man page for grep states that fgrep is the same a grep -F and that grep is deprecated. So, fgrep -F is just redundant.
Currently, the function neat_list() uses the variable "export", but does not define it. The result is that "export" variable in the calling function is used.
This PR fixes that by defining "export" as a local variable in neat_list() and by setting its value via a new parameter to the function.
This PR also removes a "FIXME" from run_rc4() since the problem has already been fixed.
This PR fixes a minor bug in get_pub_key_size(). If the key size is being determined manually and length encoding requires 4 bytes, then the current code computes the length incorrectly. This is a very insignificant bug, since does not apply to RSA or ECC keys, and the key would have to be at least 16 megabytes long for it to require 4 bytes to encode.
This PR also cleans up get_pub_key_size() a bit by replacing `i=$i+...` with `i+=...` and by enclosing math in `$(( ... ))`.
Hostnames can contain a trailing dot (and sometimes they should).
If they are supplied to testssl.sh however they will be also interpreted
as a URL PATH when the servive is HTTP.
This commit fixes that.
This switch had no effect. There was probably a regression
problem as it worked before.
Besides fixing that the large case statement in parse_cmd_line()
was simplified, in a sense that banner and help functions were
moved to a separate case statement.
This PR adds support for post-handshake messages when using sockets with TLS 1.3 connections. If a TLS 1.3 connection is established and the connection is to remain open after tls_sockets() finishes, then after the client's Finished message is sent the master secret and the application traffic keys are computed. This PR also adds two new functions to send and receive application data over a TLS 1.3 connection.
This PR also includes two proofs-of-concept for the use of the new functions. receive_app_data() is called immediately after the client's Finished message is sent. Some server's will send new session tickets immediately after the handshake is complete. If they do, then the code will decrypt and parse the session ticket messages.
This PR also modifies service_detection() to try using sockets if the server only supports TLS 1.3 and $OPENSSL does not support TLS 1.3. After the handshake is complete, this code sends an HTTP GET request and reads the response. The code is fairly slow and it doesn't always work. However, since it is only used in cases in which $OPENSSL cannot work, it can't hurt to try using sockets.
