The actual code grep for "MongoDB" keyword in the head of the HTTP
session.
In case of "compressed" HTML, a big page is on one line.
On a IT page, we could encounter the "MongoDB" keyword and
miss-identify the application protocol.
Fixed by matching on a longuer string taken from a live MogoDB
server.
OpenSSL will buffer only the first command till the establishment of the
session.
In case of slow session establishment, we could:
* loose some renego trys missing proper mitigation implementation
* loose some renego trys missing a real vulnerable host if 2/3 of the
tries are lost during session establishment (very slow startup).
Wait for the session to be fully establised before starting the renego
loop.
Reduce wait between reneg test to 0.25s. Still robust and accelerates
the test as now we do up to 10 renego tests.
With the global loop timeout, the backoff identification seem unneeded.
But if we switch to 0.25s, we no longuer trigger the global timeout so
it is still valuable.
Adjust write out messages as bash do not support floating point number
arithmetic.
Some site hang/block the connection after some renego reties
Example: https://feedback.amadeus.com
Hand written timeout logic because:
- we want to get the result of the command in case of normal exit
- we want to have working log fd redirection
- we want to known the timeout condition
If a user chose a broken umask testssl.sh will start but emits
subsequent errors.
This patch adds two sanity checks whether it is allowed to create
and read files in the temp directory.
Fixes#2449
As suggested in #1844, this commit changes testssl.sh so that the parent process quits immediately if there is an error in the command line for one of the child processes.
Currently, a signal is sent to the parent process to quit if the child process encounters an error and calls help(), but sometimes parse_cmd_line() just prints an error message and calls fatal() rather than help(), in which case the parent process does not stop. This commit addresses the issue by creating a new function, fatal_cmd_line(), which is almost the same as fatal(), but additionally sends a signal to the parent indicating that the parent should stop. This commit also changes calls to fatal() to calls to fatal_cmd_line() if json_header(), csv_header(), html_header(), or prepare_logging() encounter a problem. The same is done if prettyprint_local() with the command-line option provided for it.
There may be other places in which it would be appropriate to call fatal_cmd_line() rather than fatal() (e.g., in parse_hn_port() or check_proxy()), but those changes are not made in this commit.
What was problematic was the error message when the certificate
stores were missing. This fixes it by redirecting the error
message to /dev/null so that if the sub function detects the missing
file it returns with an error by the program and not by executing
"basename"
RFC 8996, Deprecating TLS 1.0 and TLS 1.1, states that TLS clients MUST NOT send a TLS 1.0 or TLS 1.1 ClientHello and MUST respond to a TLS 1.0 or TLS 1.1 ServerHello with a "protocol_version" alert.
At the moment, all versions of OpenSSL support TLS 1.0, TLS 1.1, and TLS 1.2. However, TLS 1.0 and TLS 1.1 are disabled in LibreSSL 3.8.1 and it is possible to compile OpenSSL without support for these protocols (using the configure options no-tls1, no-tls1_1, and no-tls1_2). This commit adds support for versions of $OPENSSL that do not support TLS 1.0 or TLS 1.1.
As suggested in #2381 this parses strictly the value for mag-age
in the HSTS header line. While it is implemented only in run_hsts()
it could be extracted to a separate functioni in the future and used
elsewhere too.
The improvement is more strict and catches e.g. '==' signs and issues
a warning. See https://www.rfc-editor.org/rfc/rfc6797#section-6.1.1 .
Also it is picky regarding quotes now which are only allowed enclosing
the value.
As this option shows inconsistencies / wrong results and a fix would require
too much work at this moment this option is being hidden from the help. It
wasn't in the ~/doc .
See #849 , #2382, #1732 etc.
If we run with supplied binary AND /usr/bin/openssl supports TLS 1.3
we now have a variable OPENSSL2_HAS_TLS_1_3 which is set to true.
It can be used for subsequent changes where we need TLS 1.3 for or
where it would be better to use TLS 1.3.
If downloaded CRL file is already in PEM format, openssl command will fail as it is always trying to convert from a DER-encoded CRL.
This commit is for adding a test of the CRL format prior to running the openssl crl conversion.
Note: as the openssl verify command then assumes that a .pem tmpfile has been generated by the conversion, there would be an issue when the conversion was not needed (i.e. CRL already PEM-encoded) as that .pem would be missing; therefore I also added a copy of the .crl file to a .crl.pem file before the optional conversion.
See #2328, original PR #2295 from @w4ntun .
Formally testssl.sh returned an error when it wasn't not possible to determine IP
addresses through DNS resolution, even if --proxy and --ip=proxy flags are set.
The main function always tried to determine IP addresses via DNS and exits with
a fatal error if it cannot do it. Although the client cannot get the IP, the
proxy could, so the SSL/TLS analysis is still possible.
This PR allows the analysis for an HTTP service via a proxy server and the DNS
traffic can be sent directly or through the proxy using the flag --ip=proxy.
ATTENTION: This may be a breaking change for those who don't have a local resolver.
They now have to add --ip=proxy.
In addition:
* help() was amended to add --ip=proxy (was only in the ~i/doc dir before)
* amending ~/doc dir to document it's better to add --nodns=min when there's
no local resolver
This addresses the bug #2330 by implementing a function which removes
control characters from the file output format html,csv,json at the
output.
In every instance called there's a check before whether the string
contains control chars, hoping it'll save a few milli seconds.
A tr function is used, omitting LF.
It doesn't filter the terminal output and the log file output.
see #2325.
"whenever HTTP/1.1 is used then the Accept header uses "text/*" as a MIME type.
This causes some minor issues with some of the checks we are doing"
As noted in #2304, the way that the '&' character is treated in the string part of a pattern substitution changed in Bash 5.2. As a result, the change that was made in #1481 to accommodate older versions of Bash (e.g., on MacOS) now causes testssl.sh to produce incorrect HTML output when run on Bash 5.2.
This commit encodes the '&' characters in the substitution strings in a way that produces correct results on multiple versions of Bash (3.2 on MacOS, 5.2 on Ubuntu 23.10, 5.0 on Ubuntu 20.04).
There are two different scenarios. x0C is the buffsize reply from openldap-like servers
whereas AD servers probably have x84 and return also the OID. The following is kind of
hackish as ldap_ExtendedResponse_parse() in apps/s_client.c of openssl is kind of hard
to understand. It was deducted from a number of hosts.
Bottom line: We'll look at the 9th byte or at the 17th when retrieving the result code
AD:
30 84 00 00 00 7d 02 01 01 78 84 00 00 00 74 0a 01 34 04 00 04 55 30 30 30 30 30 30 30 30 3a 20 [ failed AD .. LdapErr + OID..]
30 84 00 00 00 28 02 01 01 78 84 00 00 00 1F 0A 01 00 04 00 04 00 8A 16 [.. OID ..]
^^ bufflen ^^ resultcode
30 0C 02 01 01 78 07 0A 01 00 04 00 04 00
^^ bufflen ^^ result code
When a server supports client authentication, extract_calist() extracts the list of supported certification authorities sent by the server. extract_calist() uses different code to extract the list from a TLS 1.3 response than from a TLS 1.2 or earlier response, since the CertificateRequest message was changed for TLS 1.3.
For TLS 1.2 and earlier, extract_calist() assumes that the CertificateRequest message is a sequence of certificate types, signature algorithms, and certification authorities. However, the signature algorithms field was added in TLS 1.2 and does not appear in TLS 1.1 and earlier. So, the current code does not work unless the server supports TLS 1.2 or TLS 1.3.
This commit fixes the problem by checking whether the response is a TLS 1.2 response, and skipping over the extraction of the signature algorithms field if the response is neither TLS 1.2 nor TLS 1.3.
Some servers get confused if the signature_algorithms extension is too large. This commit addresses the problem by:
* For TLS 1.2, generally limiting the signature algoritms to those consistent with the key type being tested.
* For TLS 1.3, breaking the list of signature schemes in two, and testing each half of the list separately.
This commit modifies run_fs() to show the signature algorithms the server supports in the ServerKeyExchange message for TLS 1.2 and in the CertificateVerify message for TLS 1.3.
Signature algorithms are not shown for TLS 1.1 and earlier, since for those protocol versions the signature algorithm to use is specified by the protocol. While the signature algorithm used in TLS 1.1 and earlier is weak, testssl.sh already warns if these protocol versions are supported.
This commit fixes#1747 by converting PEM encoded certificates that are sent to fileout() to a single line. As suggested in #1747, '\n' is added after the '----- BEGIN ... -----' line and before the '------ END ... ------' line.
In order to ensure that '\n' appears in the string in the JSON and CSV files, '\\n' is sent to fileout() so that 'printf -- "%b"' converts '\\n' to '\n' rather than converting '\n' to a newline character.
In order to prevent fileout() from converting '\\n' to '\ ', this commit move the fix for #2049 (see PR #2050) from fileout() to fatal().
As a first cleanup action I removed in run_server_preference()
the line with Negotiated Protocol and Negotiated Cipher as
the don't have any real information, see #2235 , comment below:
https://github.com/drwetter/testssl.sh/pull/2235
This commit fixes#2271 by adding the `-no_ssl2` option to the call to get_host_cert() in run_drown(). There is at least one server that causes OpenSSL to hang if this call to get_host_cert() results in an SSLv2 ClientHello being sent. Since this call to get_host_cert() only needs to find the server's certificate in cases in which the server does not support SSLv2, there is no need to send an SSLv2 ClientHello.
This commit simplifies the adding of padding data in a few places. Rather than adding one or two bytes at a time in a "for" loop, all of the padding is added in one step by extracting it from a long padding string. (The one exception is in run_robot(), where a "for" loop is used to add additional padding in case in which the RSA modulus is longer than the pre-defined padding string.)
Extracting the padding from a long string is faster than using a "for" loop and it makes the debugging file a little cleaner.
The idea is the same as PR #1940.
