Commit Graph

1711 Commits

Author SHA1 Message Date
Dirk
0ec5586dfb Changing JSON objects in server defaults ATTENTION: breaking change!!
The server default run had several JSON objects which weren't, looking at just
    the ID, either clear or contained a redundant explanation in "finding". Purely
    certificate related JSON objects are now having the id "cert_<object>" like
    cert_CN or cert_SAN.

    This commit changes all this, also it avoids another colon in finding (see #830).

    Also the implicit strategy "output for the screen s followed by only one output with
    fileout" has been relaxed -- which results on more, better parsable JSON objects.

    Some example of the changes:

    Old:
    ----
    {
        "id"           : "Server Certificate #1 fingerprint",
        "severity"     : "INFO",
        "finding"      : "Fingerprints / Serial: SHA1 2940BC13ECF7DAF30B9084CC734C3B971D73B3BB / 01BFD1DC15006E0ABBA7C670FF5E1101, SHA256 30BA61012FFE7CEAAF9A148A0CB0C5C852A9C04F4B1C27DB6
    EFA9919C7F49CCF"
    }
    [..]
    {
        "id"           : "Server Certificate #2 ocsp_stapling",
        "severity"     : "OK",
        "finding"      : "OCSP stapling : offered"
    }

    New:
    ----
    {
        "id"           : "cert_key_size <cert#1>",
        "severity"     : "INFO",
        "finding"      : "Server keys 2048 bits"
    },{
        "id"           : "cert_fingerprint_SHA1 <cert#1>",
        "severity"     : "INFO",
        "finding"      : "2940BC13ECF7DAF30B9084CC734C3B971D73B3BB"
    },{
        "id"           : "cert_fingerprint_SHA256 <cert#1>",
        "severity"     : "INFO",
        "finding"      : "30BA61012FFE7CEAAF9A148A0CB0C5C852A9C04F4B1C27DB6EFA9919C7F49CCF"
    },{
        "id"           : "cert_serial <cert#1>",
        "severity"     : "INFO",
        "finding"      : "01BFD1DC15006E0ABBA7C670FF5E1101"
    }
    [..]
    {
            "id"           : "OCSP_stapling <cert#2>",
            "severity"     : "OK",
            "finding"      : "offered"
    }

    This PR also fixes the JSON output where for "OCSP must staple" the id was just
    'id"           : "OCSP must staple: ocsp_must_staple",' for multiple server
    certificates without the certificate number.

    As far as the code is concerned: $json_prefix should be a variable which is
    used for the id object.  If there was more then one certificates for a single
    host  detected, $json_postfix carries the certificate number.

    Unit tests need to be fixed -- if possible.
2018-01-22 23:56:03 +01:00
Jack Stevens
4cca15027d
Fix typos in HSTS tests: days -> seconds 2018-01-22 22:46:48 +00:00
Dirk
f5c2199369 Polishing #965
Add fileout() to #965. This commit also contains a change which needs
to be commited before: separation of ``json_prefix`` from ``json_postfix``.

Open issue: sed in openssl x509 statments look GNUish ([ \t]). Needs clarification.
2018-01-22 19:50:50 +01:00
David Cooper
f0c9875696 Check keyUsage and extended key usage extensions
This commit prints the contents of the keyUsage and extended key usage extensions in certificates and checks the public keys in the certificates are not being used in a manner that is inconsistent with these extensions.
2018-01-19 11:36:22 -05:00
David Cooper
db0efc48b5 run_tls_fallback_scsv() improvements
This PR is intended to improve the functionality of run_tls_fallback_scsv().

The original goal of this PR was to address servers that support TLSv1.3 when using OpenSSL 1.1.1. That does not seem to be an issue, as using `$OPENSSL s_client` with the `-no_tls1_2` flag results in a TLSv1.1 ClientHello, even if `$OPENSSL` supports TLSv1.3. However, if the server supports TLSv1.3, then a message that says "No fallback possible, TLS 1.2 is the only protocol (OK)" isn't entirely correct.

The main issue this PR fixes is some false positives in servers that do not support TLSv1.2. On a few servers that I tested, the current code incorrectly reports "Downgrade attack prevention NOT supported." Some of the servers only support TLSv1, so it should report that fallback is not possible. Another server supports TLSv1.1 and TLSv1, and it supports fallback protection. In both cases, the current code produces a false positive, since it assumes that TLSv1.2 is supported.
2018-01-19 11:19:13 -05:00
Dirk
7a10e5514f add a comment for STARTTLS + TLS 1.3
related to David's comment in https://github.com/drwetter/testssl.sh/pull/958#issuecomment-358990186
2018-01-19 16:35:28 +01:00
David Cooper
58d495e249
Fix #962
This commit fixes issue #962 by reordering the items in fileout_json_section() to match the order in which the tests are called in lets_roll().
2018-01-19 09:54:21 -05:00
Dirk Wetter
bce69e8583
Merge pull request #961 from tissieres/2.9dev
Certificate-based auth host scanning produces invalid JSON output
2018-01-19 14:10:29 +01:00
Dirk Wetter
966ef3286e
Merge pull request #958 from dcooper16/tls13_draft23
Add support for TLSv1.3 draft 23
2018-01-19 12:55:43 +01:00
Cédric Tissières
e0d6bb3aba Certificate-based auth host scanning produces invalid JSON output 2018-01-18 14:30:22 +01:00
David Cooper
42bea1d0cf
TLSv1.3 related bugfix
In three different places there is a line that is supposed to check whether the list of ciphers to be tested contains any TLSv1.3 ciphers. This check currently fails if there is only one TLSv1.3 cipher in the list and it is the first cipher in the list. This commit fixes the problem.
2018-01-17 11:06:21 -05:00
David Cooper
7387f87597 Add support for TLSv1.3 draft 23
This commit adds support for draft 23, which contains 2 changes that are relevant for testssl.sh. It adds a few new values for the signature_algorithms extension and it changes to extension number for the key_share extension from 40 to 51.

With the change in the extension number, it is no longer possible to send a single ClientHello that works for all supported drafts of TLSv1.3. (I tried sending a ClientHello with two key share extensions, 40 and 51, but that didn't work.) So, this commit adds a test to determine_optimal_proto() to determine whether TLSv1.3 is supported and if so whether draft 23 is supported or only some earlier draft (18-22). In subsequent tests, the ClientHello uses the appropriate number for the key share (40 or 51) and specifies the appropriate draft version(s) in the supported_versions extension (either 23 or 18-22). In the case of run_protocols() the test for each draft version uses the appropriate key share extension number so that servers that support both draft 23 and an earlier draft can be detected.
2018-01-17 10:56:48 -05:00
Dirk
b0f4253ff2 add X-Permitted-Cross-Domain-Policies 2018-01-14 12:22:06 +01:00
Dirk
77071bf50d a few IBM data power headers added 2018-01-11 17:28:48 +01:00
Dirk
a3b4609d24 make 0964cbc5d3 easier in execution (session resumption) 2017-12-31 21:10:41 +01:00
Dirk
0964cbc5d3 OpenSSL 1.1.1 seems to have changed the behaviour regarding Session ID
support. If the server doesn't supply an session identifier the file
is just empty.

This commit fixes that by adding a separate case for OpenSSL 1.1.1
and an empty file. It is deliberately only changing this as this
was tested to work.

It is prelimary and a save-the-work-patch as it might be better
to catch this earlier.
2017-12-31 11:49:25 +01:00
Dirk
75e9add0f1 make ID for session_resumption_id common (allo lowercase) 2017-12-31 09:12:16 +01:00
Dirk
c4c8ff4434 fix needed answer "YES" in protocol section 2017-12-29 23:14:14 +01:00
Dirk
85c470fcdc Merge branch 'jp-host+port' into 2.9dev 2017-12-27 10:14:21 +01:00
Dirk
0bc1f6f708 make MAX_PARALLEL and MAX_WAIT_TEST configurable + documentation 2017-12-27 09:50:34 +01:00
Dirk
9981d76879 Move target host + port to different JSON (pretty) object
For JSON pretty the host specific parameters target host + port
could be better placed in the scanResult object.

It is still under discussion as logically the parent object is deduced
the command line.
2017-12-25 17:42:27 +01:00
Dirk Wetter
eeaceb2845
Merge pull request #949 from dcooper16/run_crime_ossl111
run_crime() and OpenSSL 1.1.1
2017-12-25 16:56:03 +01:00
Dirk
7a1fb66957 Further (slight) update on cipher quality coloring
TLS_RSA_* which don't fall into the aleady mentioned
    categories (CBC cipher, export, RC4 etc.) are now
    a bit more more penalized. Those are the ones which have
    an RSA key exchange AND a modern encryption.

    pr_cipher_quality() needs to be redone after carefully
    reconsidered which cipher should have which rating.
2017-12-25 14:55:12 +01:00
Dirk
85fa5f4fa9 Merge branch '2.9dev' of github.com:drwetter/testssl.sh into 2.9dev 2017-12-25 14:17:06 +01:00
Dirk
8fbbb211b9 Fix error where a ECDHE-GCM cipher was labeled low
FIX #953
2017-12-25 14:15:17 +01:00
David Cooper
581b7cf579
run_crime() and OpenSSL 1.1.1
This PR is similar to #944. If using OpenSSL 1.1.1 to connect to a server that supports TLSv1.3, `run_crime()` will connect to the server using TLSv1.3, which does not support TLS-level compression. So, the server will be reported as "not vulnerable" even if would use compression for connections at TLSv1.2 and below.

I have not encountered any "live" servers that support both TLSv1.3 and TLS-level compression. I verified this problem by using OpenSSL 1.1.1 to create a server that supports both TLSv1.3 and TLS-level compression:
```
openssl111 s_server -cert cert.pem -key key.pem -accept 8443 -WWW -comp
```
I then tested the server using `testssl.sh --crime` with both openssl111 and OpenSSL 1.0.2-chacha.
2017-12-21 11:22:08 -05:00
Dirk
978478fd0c Fix "typo" 2017-12-21 15:06:08 +01:00
David Cooper
b9e67fcf29
run_renego() and OpenSSL 1.1.1
run_renego() appears to produce a false positive if OpenSSL 1.1.1 is used and the server being tested supports TLSv1.3 (i.e., the server supports the same draft version of TLSv1.3 as the version of OpenSSL 1.1.1 being used does). This PR fixes the problem by telling calls to $OPENSSL s_client in run_renego() to not use TLSv1.3.
2017-12-20 16:38:10 -05:00
Dirk
5c005ac139 Add '--full' / '-9' ...
.. to check during the default run for server implemenation bugs
and run cipher per procol check instead of cipher check.

Please not that this option could disappear later.
2017-12-20 19:21:33 +01:00
Dirk Wetter
6d53a1e9f4
Merge pull request #942 from dcooper16/status_request
Process TLSv1.3 status_request extension
2017-12-20 18:52:40 +01:00
David Cooper
14908bac98
Process supported_groups extension
In TLSv1.3 servers may send a supported_groups extension, which "SHOULD contain all groups the server supports, regardless of whether they are currently supported by the client."

This PR extracts the contents of the supported_groups extension, if `parse_tls_serverhello()` is to process "all" of the server's response. The contents of the extension are also displayed on the terminal if $DEBUG -ge 3.
2017-12-20 10:40:17 -05:00
David Cooper
65e435eb70
Process TLSv1.3 status_request extension
In TLSv1.2 and below, servers respond to a status_request extension (a request for a stapled OCSP response) by returning an empty status_request extension and then including a CertificateStatus message, which follows the Certificate message. In TLSv1.3 the CertificateStatus response is included as the value of the status_request extension, which now appears as an extension within the Certificate message.

This PR extracts the contents of the status_request extension sent by the server so that it can later be processed in the same way as if it had sent in a TLSv1.2 or below response.
2017-12-20 10:17:21 -05:00
David Cooper
e8be1f441b
Decrypt server's TLSv1.3 response
This PR adds code to decrypt the encrypted portion of the server's response for TLSv1.3 and to then process any certificates and encrypted extensions. This code supports all 5 TLSv1.3 cipher suites, and so any response can be decrypted as long as the session key can be derived (which requires OpenSSL to support the ephemeral key that was used - see #938).

For the symmetric decryption, the sym-decrypt() function uses OpenSSL when possible and internal Bash functions when needed.

For AES-GCM and AES-CCM ciphers sym-decrypt() normally uses internal Bash functions, which rely on using "$OPENSSL enc" in AES-ECB mode to generate the key stream and then Bash functionality to XOR the key stream with the ciphertext. With some version of OpenSSL the AES-GCM ciphers are decrypted using "$OPENSSL enc" in AES-GCM mode directly. On my system, however, both methods seem to work about equally fast.

For ChaCha20 ciphers, "$OPENSSL enc -chacha20" is used, if supported (OpenSSL 1.1.x only). and Bash internal functions (without any OpenSSL support) are used otherwise. In this case, if the Bash internal functions need to be used, decryption is very, very, very slow. Fortunately, in a typical run of testssl.sh there won't be many cases in which the connection will be TLSv1.3 with ChaCha20 and the entire response needs to be processed (requiring decryption). In most cases, even if the connection is TLSv1.3 with ChaCha20, will at most need the ephemeral key, which is available in plain text.
2017-12-15 16:40:47 -05:00
David Cooper
cfecc3c8c4 Derive handshake traffic key for TLSv1.3
This is the first in a series of PRs to add support for processing the encrypted portions of the server's response in a TLSv1.3 handshake.

This PR adds the code to derive the handshake traffic key needed to decrypt the response (the next PR will add the code to perform the symmetric-key decryption of the encrypted portions of the response).

Since this PR does not make use of the traffic key that it derives, it doesn't yet add any new functionality.

Note that testssl.sh will not always be able to derive the session keys. If the version of OpenSSL that is bundled with testssl.sh is used and the server chooses to use an X25519 ephemeral key, OpenSSL will be unable to perform the shared secret in derive-handshake-traffic-secret(). (OpenSSL 1.1.0 supports X25519.) Since X25519 use a different encoding than ECDH keys, the lack of X25519 support will be discovered in parse_tls_serverhello() when $OPENSSL pkey is unable to convert the key from DER to PEM. So, in debugging mode, parse_tls_serverhello() now displays a warning if it receives a key share that $OPENSSL pkey cannot handle.
2017-12-14 16:55:18 -05:00
Dirk
c23f47858d Overwrite as a default empty files
This commit tries to address #934. Empty files can now
be overwritten without exiting with an error.
2017-12-14 10:06:19 +01:00
David Cooper
d8839b375b Fix check for whether certificates were found
get_server_certificate() uses an awk script to extract the certificates from the output of OPENSSL s_client and it then uses the following line to determine how many certificates were found:

     nrsaved=$(count_words "$(echo level?.crt 2>/dev/null)")

If $nrsaved is 0, then get_server_certificate() returns 1 (indicating failure); otherwise it returns 0 (indicating success).

However, the check for the number of certificates returned doesn't work if no certificates were found, as nrsaved will be set to 1 if no certificates were found:

     > touch level0.crt
     > echo level?.crt
     level0.crt
     > touch level1.crt
     > echo level?.crt
     level0.crt level1.crt
     > rm level0.crt level1.crt
     > echo level?.crt
     level?.crt

This PR fixes the problem by first checking that level0.crt exists (-s is used instead of -e, since an empty file wouldn't have a certificate).
2017-12-13 11:23:41 -05:00
David Cooper
cea570836f
Check for pkey support
Similar to the recently added HAS_PKUTIL (f829878a43), this PR adds HAS_PKEY, which indicates whether OpenSSL has the pkey utility. HAS_PKEY is then checked before attempting to do something that requires the pkey utility.
2017-12-13 11:01:58 -05:00
Dirk
69956016b3 CVEs from https://robotattack.org/ added (in JSON) 2017-12-13 13:21:17 +01:00
Dirk Wetter
fe7feda228
Merge pull request #932 from dcooper16/reduce_file_ops
Reduce use of temporary files
2017-12-12 19:41:39 +01:00
Dirk
5978d37c74 Merge branch '2.9dev' of github.com:drwetter/testssl.sh into 2.9dev 2017-12-12 19:41:07 +01:00
Dirk
f829878a43 Introduce property global HAS_PKUTIL 2017-12-12 19:37:33 +01:00
David Cooper
274d8a8b3a
Reduce use of temporary files
This PR reduces testssl.sh's use of temporary files by making greater use of pipes. This results in a very slight performance improvement.
2017-12-12 11:51:15 -05:00
David Cooper
4a209cceb4
Correct comment
The comment at the beginning of the run_robot() function uses the wrong title for the paper. This PR fixes the title and adds a URL.
2017-12-12 10:25:44 -05:00
David Cooper
5de873f8bc Test for vulnerability to Bleichenbacher attack
This PR adds a test to check whether a server that supports ciphers suites that use RSA key transport (TLS_RSA) are vulnerable to Bleichenbacher attacks (see http://archiv.infsec.ethz.ch/education/fs08/secsem/bleichenbacher98.pdf).
2017-12-12 09:51:48 -05:00
Frank Breedijk
393e4fb920 JSON output fixed in --parallel too 2017-12-06 21:54:59 +01:00
Frank Breedijk
3eb402a3d5 JSOn output is broken by the JSON footer in combination with mass mode 2017-12-06 16:10:53 +01:00
Dirk
2a0a0475ab Fix for POP3, Robustness for STARTTLS
POP3 STARTTLS handshakes were often unsuccessful as
a regex wasn't properly escaped.

Furthermore if a STARTTLS handshake doesn't succeed, there's
a warning now.
2017-12-06 12:24:26 +01:00
Dirk
9e1ed1c9cf Add check for -oA in nma greppable output 2017-12-06 10:23:24 +01:00
Dirk
f92d3988af Better consistency (cipher coloring)
Especially for yellow=low the key exchange is considered
to be equal now. That needs to be subject to reconsideration.
2017-12-04 21:53:39 +01:00
Dirk Wetter
cf89488ab9
Merge pull request #905 from dcooper16/color_all_ciphers
Color all ciphers
2017-12-04 21:12:01 +01:00
Dirk
7f6ff5db3b Sloppy try to address in the worst cases of wrong coloring of ciphers
See pending PR #905 / issue #333.

There's still lots of work needed and probably the function
needs to be completely rewritten and to be in sync with
other parts of the program.
2017-12-04 21:06:47 +01:00
Dirk
0c779d1684 proper indentation of "client-based" in service detection
Also correct "based" to "-based" (Marriam-Webster + Oxford Dict.)
where appropriate.
2017-12-04 17:14:06 +01:00
Dirk
721ca1e45c Fix regression in HTTP header
This PR fixes one odd formatting of header flags like X-Frame-Options,
where the output header maybe contained a LF "\r". X-XSS-Protection was
also not correctly formatted due to the fact that only a part of it until
the blank was displayed.

Also the file output may contain now 1x less blank, e.g.

"X-Content-Type-Options: nosniff" instead of
"X-Content-Type-Options:  nosniff"
2017-12-04 15:47:45 +01:00
David Cooper
853b8333e3 Color all ciphers
This PR adds an additional COLOR level (3). If color is set to 3 then all ciphers are printed according to pr_cipher_quality() rather than just the "Negotiated cipher" in run_server_preference().
2017-12-04 09:18:27 -05:00
David Cooper
b6a022d84f Reducing processing of alert messages
This PR reduces the amount of work parse_tls_serverhello() does when processing alert messages when not in debug mode. It delays writing anything to $TMPFILE unless $DEGUG -ge 1 until it has reason to believe that the response was successful. If $DEBUG is 0 and alert messages are sent, then no file operations are performed processing the alert messages.

In almost every case, there is no attempt to look at the contents of $TEMPDIR/$NODEIP.parse_tls_serverhello.txt unless the connection was successful. So, in most cases, it is okay to not call tmpfile_handle() in parse_tls_serverhello() unless the connection was successful. There is, however. one place in run_grease() where the code reads the contents of $TEMPDIR/$NODEIP.parse_tls_serverhello.txt even if the connection was not successful. In order to address this, the DEBUG level is temporarily set to 1 when performing this test if its value is 0. Also in order to address this, changes were made in parse_tls_serverhello() to ensure that "tmpfile_handle $FUNCNAME.txt" is always called before returning if $DEBUG -ge 1.
2017-12-01 15:45:36 -05:00
Dirk
62f925a1bc Merge branch '2.9dev' of github.com:drwetter/testssl.sh into 2.9dev 2017-12-01 21:21:22 +01:00
Dirk Wetter
137f2fa3c3
Merge pull request #920 from dcooper16/client_auth_and_assume_http
HTTP-related checks and certificate based client authentication
2017-12-01 21:19:23 +01:00
Dirk Wetter
9c8fa5419d
Merge pull request #917 from dcooper16/check_version
Compare selected version against supported_versions
2017-12-01 21:03:52 +01:00
Dirk
dda07cd1d0 commenting #918 2017-12-01 21:02:03 +01:00
Dirk Wetter
4be759afea
Merge pull request #918 from sdann/mysql_read_timeout
Reduce read timeout for MySQL back to 1 second
2017-12-01 20:54:37 +01:00
Dirk
99d12434dd fix strip_*_space() 2017-12-01 20:09:16 +01:00
David Cooper
1ba4b395ff
HTTP-related checks and certificate based client authentication
If certificate-based client authentication is required by the server, then most HTTP-related checks are skipped, even if the "--assume-http" flag is used. If $CLIENT_AUTH is true, then $ASSUME_HTTP is ignored.

In some cases the checks are appropriately skipped, since the tests cannot be performed. In other places, the value of "$CLIENT_AUTH" is used as a hint as to whether HTTP is being used. For example, in run_tickbleed:

     if [[ "$SERVICE" != HTTP ]] && ! "$CLIENT_AUTH"; then
          outln "--   (applicable only for HTTPS)"
          fileout "ticketbleed" "INFO" "Ticketbleed: not applicable, not HTTP" "$cve" "$cwe"
          return 0
     fi

There are some places, however, where tests are just skipped, even if both $CLIENT_AUTH and $ASSUME_HTTP are true, even though the test could be performed. For example, run_client_simulation() only simulates generic clients in this case.

This PR attempts to address this:
* In run_client_simulation() it runs all of the tests if $ASSUME_HTTP is true.
* In certificate_transparency() it only says that the lack of CT information is "N/A" it can verify that HTTP is not being used (if $SERVICE is not HTTP and $CLIENT_AUTH is false). Otherwise it just says "no" without flagging it as an issue.
* In certificate_info() it displays additional warnings (about use of SHA-1 or subjectAltName matching) only if it can verify that HTTP is being used ($SERVICE is HTTP or $ASSUME_HTTP is true).
* In run_crime(), if compression is used, it only says " but not using HTTP" if it can verify that HTTP is not being used (if $SERVICE is not HTTP and $CLIENT_AUTH is false).
2017-12-01 10:58:06 -05:00
David Cooper
76c75ae8f9 Replace external calls with Bash functions 2017-12-01 10:31:30 -05:00
David Cooper
74c1a6bcb3 Compare selected version against supported_versions
If a supported_versions extension was included in the ClientHello, then check that the version returned by the server was included in the ClientHello's supported_versions extension.

OpenSSL will respond to a TLSv1.3 ClientHello that only specifies 0304 in its supported_versions extension with a ServerHello that specifies whatever draft of TLSv1.3 it currently supports (e.g., 7F16). The result is that run_protocols() incorrectly reports that OpenSSL supports TLSv1.3 "final" in addition to whatever draft version it supports.

This PR fixes that problem by treating it as a failed connection when the ClientHello offers only 0304 and the ServerHello specifies something else (e.g., 7F16).

Performing this check is actually a requirement for clients in Section 4.2.1 of draft-ietf-tls-tls13-22. So, including this check will also help make client simulations more accurate when clients that support TLSv1.3 are added to client-simulation.txt.
2017-12-01 10:31:29 -05:00
Dirk
851030ea8f Fixes in headers
- double word match: Content-Security-Policy matched also X-Content-Security-Policy
- X-UA-Compatible always appeared twice
2017-12-01 13:31:43 +01:00
Steven Danneman
2fb7e45799 Reduce read timeout for MySQL back to 1 second
The default STARTTLS_SLEEP timeout was increased to 10 seconds in
d1e7498. This caused MySQL connections to timeout. Quick fix is to
parameterize the timeout and pass in 1 again.

Better future fix is to read MySQL as binary packets, parsing the fixed
sized header, to then read the variable sized payload. Doing this will
also greatly speed up testing.

This fixes issue #914.
2017-11-30 15:10:47 -08:00
Dirk
1b7e6630d7 Extending OPTIMAL_PROTO tests with TLS 1.3
It partly addresses #915, supposedly the openssl binary used supports
TLS 1.3 (and the correct draft/final).

It also reduces handshakes by not trying protocols which aren't supported
on the client side.
2017-11-30 16:07:51 +01:00
David Cooper
fee2f68a64
Support TLSv1.3 draft 22
This PR adds support for TLSv1.3 draft 22. This PR has testssl.sh operate in "middlebox compatibility mode" as described in Appendix D.4 of draft-ietf-tls-tls13-22 to maximize the chances of being able to perform a successful test even if there is a misbehaving middlebox between testssl.sh and the server being tested. Support for drafts 18 through 21 is still maintained.

This PR has been tested against a few different implementations of draft 22 that were made available shortly before draft 22 was posted.
2017-11-29 14:47:22 -05:00
Dirk Wetter
affc9d6bb6
Merge pull request #909 from dcooper16/remove_extra_spaces
Remove extra spaces
2017-11-29 19:39:54 +01:00
David Cooper
77b5a58b2d Remove extra spaces
This PR just removes some extra spaces in the printing of TLS extensions in parse_tls_serverhello().
2017-11-29 09:15:01 -05:00
David Cooper
0a6e88041c Don't open new socket for second ClientHello
In TLSv1.3, when responding to a HelloRetryRequest, the second ClientHello should be sent through the same socket as the first ClientHello.

This PR adds an option to socksend_tls_clienthello() to not open a socket and then uses that option in resend_if_hello_retry_request() when sending the second ClientHello.
2017-11-29 09:14:16 -05:00
Dirk
f61b701f5a recognise TLS alert as a sign that SSLv2 is not supported
Revert the exception from previous commit 20b38d5aa6
that TLS alerts on SSLv2 client hello won't be a proper reply -- due to a reality check,
see #908.

In order to have better debugging info the TLS alert message is printed
in clear. Messages code and text assignements was moveed to a separate
function.
2017-11-29 11:43:35 +01:00
Dirk
ece1d868c4 change wording and color for not valid SSLv2 server hello 2017-11-28 10:03:49 +01:00
Dirk
20b38d5aa6 Make SSLv2 parsing more robust 2017-11-27 21:55:17 +01:00
Dirk Wetter
0633ca595d
Merge pull request #904 from dcooper16/negotiated_proto_and_cipher
Add TLSv1.3 support for negotiated protocol and cipher
2017-11-27 12:42:29 +01:00
Dirk
0896c901db fix problem with servers which return gzip encoded body
Some servers like the one from the satire magazine "focus.de"
choose to return gzip encoded body during run_http_haders().
This has led sometimes to misintepretation that an IPv4 address
is present in the header.

This commit fixes that by telling the server not to want a gzipped
response and if still returned grep properly the return.
2017-11-25 17:55:55 +01:00
Dirk
d1e7498458 Increase STARTTLS_SLEEP for lame servers
.. as otherwise the connection will be tore down too early.
It doens't cost anything for faster servers as it is the max
time to wait.
2017-11-24 22:55:13 +01:00
Dirk
f2b37c767d Remove output of ticketbleed for STARTTLS 2017-11-24 22:43:35 +01:00
Dirk
e9aaae0d46 remove any spdy + alpn output for STARTTLS
... as it's being done all other places
2017-11-24 22:39:18 +01:00
Dirk
9fe02e45fa Add robustness to STARTTLS+sockets
When testing servers which fall back after succeeding the STARTTLS handshake
to the underlying protocol (smtp, ftp etc.) there was often misleading output
=not available instead of notifying the user that there's a problem.

Now it is being tested in parse*serverhello() functions whether the first
by resembles a 5XX code and STARTTLS is supposed to be tested and then
passes an error code back to the caller , i.e. tls_sockets() and then
run_protocols(). Also other error code after the handshake are passed
better.

This is only an addition to the top level function run_protocols().
We might want to look into other top level functions too
2017-11-24 19:28:55 +01:00
Dirk
ad97b6fc4e Polish screen output "protos" not offered
In scenarios where --ssl-native was chosen AND the
openssl binary wasn't supporting the protocol
there were two warnings. This has been addressed.

Also tls_sockets() can return different values -- for now: 6 -- as
tested by the caller in run_protocols. In order to make
it more robust a fixme statement was added so that the
user becomes iat least aware of it.
2017-11-24 16:21:06 +01:00
Dirk
778a7cc12d Fix change of warning logic (no protos offered)
After changing the logic from $PROTOS_OFFERED
contaning each protocol detected to $PROTOS_OFFERED
contaning each protocol + colon yes or no there
wasn't a change for the sanity check that no protocols
are offered. This fixes it.
2017-11-24 15:15:50 +01:00
David Cooper
5c54f976e8
Add TLSv1.3 support for negotiated protocol and cipher
This PR adds TLSv1.3 support for the negotiated protocol and cipher in run_server_preference(). This mostly addresses #893, however, run_server_preference() will not work with a TLSv1.3-only server as it will fail when trying to determine whether the server has a cipher order.

Note that with this PR run_server_preference() will not always provide consistent results when testing a server that does not support TLSv1.3 using a version of OpenSSL prior to 1.1.1. If it is determined before running run_server_preference() (using run_protocols()) that the server does not support TLSv1.3, then run_server_preference() will using OpenSSL to determine the negotiated protocol and cipher. However, if it has not yet been determined that the server does not support TLSv1.3, then run_server_preference() will use tls_sockets(), which tries to simulate OpenSSL 1.1.1. Since the list of cipher lists sent will differ the negotiated cipher will sometimes differ. In addition, when a cipher suite that uses an ephemeral ECDH key is selected, the negotiated curve is sometimes different.
2017-11-21 16:14:04 -05:00
David Cooper
c65a352dcf Fix #901
This PR fixes #901. When $OPENSSL_TIMEOUT is set and mass testing is being performed, this PR changes find_openssl_binary() so that only child instances modify $OPENSSL to add the call to timeout.

This PR also changes the warning in case $OPENSSL_TIMEOUT is set and the timeout binary cannot be found so that the warning message is printed even if $WARNINGS is "off" or "batch".
2017-11-20 13:38:12 -05:00
Dirk
77fc111b13 put new socksend() into production
FIX #902.

Performance improvements LAN (just one shot):

```
	58 / 50
	52 / 44
	53 / 47
	143 / 137
	63 / 53
	48 / 43
```

WAN:

```
	89 / 77
	88/ 67
	234 / 231
	67 / 63
	84 / 81

```
2017-11-20 16:33:40 +01:00
Dirk
209b21c817 fix travis in cce48a34d0 2017-11-19 23:13:33 +01:00
Dirk
cce48a34d0 New socksend(), json flat: scan time
There's a new socksend() function which gives up to 8% performance
benefit (LAN) as is saves 3 sed and 1 tr command in every socksend call.
It has not been put in production yet as it shows a problem in ticketbleed
check and it is late and I call it a day ;-) and resolve that later (see #902).

For performance tests it's quite handy to also have in the flat json format
a footer with time consumed for each run. fileout_json_footer() was
patched accordingly.
2017-11-19 20:44:06 +01:00
Dirk Wetter
9949be750d
Merge pull request #900 from dcooper16/fix_heartbleed
Fix run_heartbleed()
2017-11-19 18:27:03 +01:00
Dirk
fd4e5469fe changed output in CAA and OCSP stapling
... from "--" to "not offered". Reason: e.g. on a white
terminal background it is hard to detect that there's
an output. Also "not offered" is more consistent with
the value of protocol checks.
2017-11-19 17:11:22 +01:00
Dirk
f20ce1a0f8 replace old school grep/sed/sort in certificate_info() 2017-11-19 17:02:24 +01:00
David Cooper
9db0a8f78c
Fix run_heartbleed()
When the server's response to the ClientHello message (i.e., the ServerHello, Certificate, ServerHelloDone) is split across more than one packet, the current call to tls_sockets() in run_heartbleed() only retrieves the first packet. As a result, sometimes when run_heartbleed() believes that it is reading the response to the Heartbleed payload it is actually reading the continuation of the response to the ClientHello message.

This PR fixes the problem by indicating in the call to tls_sockets() that the ephemeral key is needed. This causes tls_sockets() to continue requesting additional packets until it receives the ServerHelloDone.
2017-11-17 16:15:37 -05:00
Dirk
fb3aa6ad7f fix for filename output beginnng with .
There was an error in e450eb34e4
which addressed #846 which lead to the problem that an output filename
could start with a "." if no FNAME_PREFiX and if just --html, --csv
or --log or --json* was supplied.

This commit fixes the problem.
2017-11-17 22:09:56 +01:00
Dirk
98f4f28017 FIX #898
fix escaping of debugging output.  credits to @dcooper16
2017-11-17 20:56:18 +01:00
David Cooper
7219f16c2a
Yet another fix for OpenSSL 1.1.1
When the certificate signature algorithm is RSA-PSS and OpenSSL 1.1.1 is used $cert_sign_algo contains some trailing space characters, which causes the algorithm not to be recognized in the case statement. This PR fixes the problem by removing any trailing space characters from $cert_sign_algo.
2017-11-16 14:22:56 -05:00
Dirk
54b9119162 FIX #897 2017-11-16 01:07:26 +01:00
Dirk
7ec0d7ffb7 Polish #846, correct level for OCSP and GOST 2017-11-14 20:50:14 +01:00
Dirk
e450eb34e4 FIX #846 -- add output filename prefix
This commit adds the possibility to supply a output
file name prefix via --outprefix or FNAME_PREFIX
2017-11-14 19:41:25 +01:00
David Cooper
07d6aa8e60 Use tls_sockets() for run_heartbleed()
This PR changes run_heartbleed() to use tls_sockets() to send the ClientHello and to read the ServerHello.
2017-11-08 08:51:20 -05:00
David Cooper
fdfaa01946
Fix false positive in std_cipherlists()
This PR fixes a false positive in std_cipherlists(). Currently, sclient_success is not initialized (so it initially set to 0). If a server is being tested that only supports TLSv1.3, the --ssl-native option is not used, and run_protocols() is run before run_std_cipherlists(), then for many of the calls to std_cipherlists() no tests are run and so sclient_success remains at its initial value (0), which is treated as success (i.e., the server supports at least one of the ciphers in the list).

The reason this happens is that in the testing loop, the TLSv1.3 test is skipped if the list of ciphers doesn't include any TLSv1.3 ciphers (and only the "Strong encryption" test includes TLSv1.3 ciphers) and the tests for each of lower versions of SSL/TLS is skipped since it was already determined in run_protocols() that those versions weren't supported.
2017-11-07 11:53:49 -05:00
Dirk
5bd8cb08ba fix #892 (trailing dot in supplied hostname)
... and do minor updates to do bash internal functions in ``parse_hn_port()``
2017-11-05 22:41:11 +01:00
Dirk
000f957646 minor changes on TLS 1.3 protocol tests (see #890) 2017-11-05 20:30:18 +01:00
Dirk Wetter
b613f3fcf0
Merge pull request #889 from dcooper16/run_pfs_tls13
Add TLSv1.3 support for run_pfs()
2017-11-05 14:40:06 +01:00
David Cooper
a75617cfdb Add TLSv1.3 support for run_protocols()
This PR adds a check for TLSv1.3 support to run_protocols(), checking for support for the final version of TLSv1.3 (0x0304) as well as drafts 18, 19, 20, and 21 (0x7F12, 0x7F13, 0x7F14, and 0x7F15).
2017-11-02 17:22:04 -04:00
David Cooper
a102ee8fb6
Add TLSv1.3 support for run_pfs()
This PR adds TLSv1.3 support for run_pfs().
2017-11-02 11:28:09 -04:00
Dirk
9daec2a515 Add "auto" keyword to -oA/-oa (FIX #887)
File names are now auto-generated by using "-oA auto" / -oa "auto"
--similar to --csv and friends.

Also the formerly hidden switches --outFile and --outfile were added in the
help and in the manual.
2017-11-01 09:58:52 +01:00
Dirk
9b7000e87e Final check to make sure fileout and pr*warning correlates
Made sure that if fileout has a WARN or DEBUG flag it is
consistent with pr*warning. FIX #518
2017-10-31 12:23:16 +01:00
Dirk
278202ace9 FIX #848 -- determine tls date only when instructed
Former code implied a determination of the TLS time in
every call of tls_sockets() despite the fact that the
value is only needed at one point in the run.

This removes this behaviour by introducing another global
boolean switch TLS_DIFFTIME_SET which determines whether
the additional cost will be paid or not.

The gain in execution time is a bit meager though. At
most it seems it's 1-3 seconds.
2017-10-31 11:27:19 +01:00
Dirk
dca4da4736 FIX #884, FIX #885
For servers with client authentication one would need to supply a x509
certificate to check session resumption by ID or ticket. This is not (yet?)
supported in testssl.sh.

This commit fixes the misleading error message so that it is clear what the
problem is.
2017-10-30 18:41:19 +01:00
David Cooper
ad6cde996a
Add check for the Expect-CT header
This commit adds a check for the Expect-CT header to run_more_flags().
2017-10-30 11:48:48 -04:00
Dirk Wetter
b9723424e7
Merge pull request #882 from dcooper16/suppress_config_file_warn
Suppress more config file warnings
2017-10-30 08:41:55 +01:00
David Cooper
24c342b34b
Add TLSv1.3 support for run_cipher_match()
This PR adds TLSv1.3 support for run_cipher_match(). It also addresses issue #660 for run_cipher_match().
2017-10-27 13:52:09 -04:00
David Cooper
1addd74178
Suppress more config file warnings
This PR is a continuation of #833.

With additional testing with different options I encountered more places where $OPENSSL was printing "WARNING: can't open config file: /usr/local/etc/ssl/openssl.cnf" where testssl.sh was not suppressing the error message.

This PR redirects stderr to /dev/null or to $ERRFILE for several more calls to $OPENSSL in order to suppress these warning messages.
2017-10-27 13:07:04 -04:00
David Cooper
a25cbf5078
Fix problem with LibreSSL encryption names
For ciphers that use the ChaCha20-Poly1305 cipher, LibreSSL shows "Enc=ChaCha20-Poly1305" in the "openssl ciphers -V" command rather than "Enc=ChaCha20(256)" and for some GOST ciphers it shows "Enc=GOST-28178-89-CNT" rather than "Enc=GOST(256)". This causes a problem for neat_list() if information is being obtained from "$OPENSSL ciphers -V" rather than from the cipher-mapping.txt file.
2017-10-27 11:49:11 -04:00
David Cooper
06f842ae69 Use read_sigalg_from_file() helper function
This PR fixes several places where the read_sigalg_from_file() helper function isn't being used.
2017-10-27 10:34:04 -04:00
Dirk
a85ca3c250 FIX #765
Inconsistency in using optional and mandatory parameters in help and man page
2017-10-26 11:46:14 +02:00
Dirk
4e70ac6ad5 FIX #870 = testssl.sh -v/-b picks up wrong openssl binary 2017-10-22 23:41:17 +02:00
Dirk Wetter
f88e3d89f1 Merge pull request #826 from dcooper16/fix772
Complete fix of #772
2017-10-22 23:28:16 +02:00
Dirk Wetter
b16a86cf46 Merge pull request #827 from dcooper16/OpenSSL_111_fix
Another fix for OpenSSL 1.1.1
2017-10-22 23:27:50 +02:00
Tom Wassenberg
39bc207d0e
Correct small inconsistency in comment
The main parameter is referred to as "<URI>" everywhere, except for in one
comment, where it was "<host>". Made this consistent with other uses.
2017-10-20 21:36:41 +02:00
Dirk
e3b254d0a1 FIX #869 2017-10-20 19:58:20 +02:00
David Cooper
63fe5fa170 Complete fix of #772
PR #777 introduced a proposed solution to #772. This PR applies the proposed solution wherever it was not applied in #777.
2017-10-20 13:54:18 -04:00
David Cooper
0efaf9114f Another fix for OpenSSL 1.1.1
If testssl.sh is used with OpenSSL 1.1.1 and TLSv1.3 support is enabled, then the check for whether the server has a cipher order will always fail. The problem is that since the call to s_client doesn't specify a protocol a TLSv1.3 ClientHello will be sent. However, the call specifies a list of ciphers that doesn't include any TLSv1.3 ciphers. So, OpenSSL will fail with the error: "No ciphers enabled for max supported SSL/TLS version." The solution is to add the "-no_tls1_3" option.

This PR fixes the problem by taking advantage of the recently-added s_client_options() function. It adds a "-no_tls1_3" option whenever:
* $OPENSSL supports TLSv1.3
* The command line doesn't specify any protocol: -ssl2, -ssl3, -tls1, -tls1_1, -tls1_2, or -tls1_3.
* The command line includes the -cipher option
* The list of ciphers that will be sent doesn't include any TLSv1.3 ciphers.

Add TLSv1.3 support to run_cipher_per_proto()

Fix branch
2017-10-20 13:51:29 -04:00
Dirk Wetter
f212b609ab Merge pull request #867 from dcooper16/tls13_for_run_allciphers
Add TLSv1.3 support for run_allciphers()
2017-10-20 18:12:02 +02:00
David Cooper
ca7c8200eb Add TLSv1.3 support for run_server_preference()
This PR adds support for TLSv1.3 to run_server_preference(). It only provides partial support, as it only works if the support supports and earlier TLS protocol (in order to determine whether the server has a cipher order). It also will only show TLSv1.3 as the "Negotiated protocol" if $OPENSSL supports TLSv1.3.

This PR also fixes a bug in which the variable "proto" was defined as used as both a regular variable and as an array.
2017-10-20 11:40:19 -04:00
David Cooper
fb5c049fd7 Add TLSv1.3 support for run_allciphers()
This PR adds TLSv1.3 support for run_allciphers(). It also addresses issue #660 for run_allciphers().
2017-10-20 11:39:30 -04:00
Dirk
50fa1e74bd Merge branch '2.9dev' of github.com:drwetter/testssl.sh into 2.9dev 2017-10-20 16:45:59 +02:00
Dirk
d3795f1254 Add output options similar to nmap (FIX #861) 2017-10-20 16:32:57 +02:00
David Cooper
ebb99a53d2 Fix #871
This PR fixes the bug identified in issue #871.
2017-10-20 09:09:52 -04:00
Dirk
52e02d9d43 CAA work
This fixes #865 and improves #588. All CAA records are now shown,
also with old DNS binaries.
2017-10-18 18:43:54 +02:00
Dirk
4b187d6253 fix travis error in 8ceb1b5 2017-10-18 17:13:05 +02:00
Dirk
8ceb1b5ad8 use bash internal functions for #864 2017-10-18 15:25:43 +02:00
Dirk Wetter
76248493a0 Merge pull request #863 from dcooper16/run_server_preference_ssl3_bugfix
run_server_preference() bug fix
2017-10-18 13:42:54 +02:00
Frank Breedijk
0bb1767cc4 This should return it to normal 2017-10-18 08:22:51 +02:00
Frank Breedijk
4544f0f300 Make CAA record lookups resolve the entire DNS tree (Fixes #862) 2017-10-18 08:05:02 +02:00
David Cooper
ab9c70119f run_server_preference() bug fix
If run_server_preference() is performed
* against a server that supports SSLv3 and that does not have a cipher order; and
* using a version of OpenSSL that does not support SSLv3; and
* with the --mapping option set to "rfc" or "no-openssl"
then the "Negotiated cipher per proto" will not show the SSLv3 cipher since cipher[i] will be empty.
2017-10-17 13:28:15 -04:00
Dirk Wetter
4adc96b2f7 Merge pull request #856 from dcooper16/run_cipher_per_proto_tls13
Add TLSv1.3 support to run_cipher_per_proto()
2017-10-14 09:21:09 +02:00
Dirk Wetter
ac3ff349c9 Merge pull request #859 from dcooper16/rc4
run_rc4() and #660
2017-10-14 09:17:19 +02:00
David Cooper
605d0178ec run_rc4() and #660
This PR addresses issue #660 for run_rc4(), ensuring that support for RC4 ciphers is detected even if no RC4 ciphers are supported with the highest protocol that the server supports.
2017-10-13 16:50:36 -04:00
David Cooper
cbd50043ef Add TLSv1.3 support to run_cipher_per_proto() 2017-10-12 10:37:31 -04:00
David Cooper
7019f8b306 Add TLSv1.3 support for run_std_cipherlists()
This PR adds support for TLSv1.3 to run_std_cipherlists().

This PR also provides a partial fix for #660 - addressing the issue only for run_std_cipherlists(). Rather than testing the server once for each cipher list, it tries the server once for each protocol supported by the server. This makes the testing more robust, but adds significantly to the time it takes to perform the tests.
2017-10-12 10:36:39 -04:00
Dirk Wetter
22fd594334 Merge pull request #857 from dcooper16/use_bash_internal
Use bash internal functions
2017-10-12 11:34:20 +02:00
David Cooper
58d67b52a1 Fix #660 for run_sweet32()
This PR addresses issue #660 for run_sweet32(), detecting if 3DES is used, even if it isn't used with the highest protocol version supported by the server. As with PR #854, this increased robustness comes at the expense of taking addition time to run the test.
2017-10-11 15:41:05 -04:00
David Cooper
dab7c1bfc0 Use bash internal functions
This PR replaces more uses of awk with bash internal functions.
2017-10-11 15:20:49 -04:00
David Cooper
ea4e68c2ce Use bash internal functions
This PR replaces the use of sed or awk in a few places with bash internal functions.
2017-10-11 10:59:13 -04:00
David Cooper
a8ef95c2cc Don't maintain SOCK_REPLY_FILE in non-debug mode
In client_simulation_sockets() and tls_sockets(), don't work to create a SOCK_REPLY_FILE that contains the entire server's response (in cases where the response was spread across multiple packets) unless $DEBUG is at least 1.
2017-10-10 14:48:36 -04:00
David Cooper
3f342ec461 Fix typo
I believe there is a typo in the second definition of DEBUG_ALLINONE. If I run testssl.sh using the -x option for bash I get the following error:
     testssl.sh: line 12714: -false: command not found
2017-10-10 10:34:33 -04:00
Dirk
785e94912d replace grep -q by internal bash operator
and provide the alias SETX for DEBUG_ALLINONE
2017-10-09 15:13:46 +02:00
Dirk Wetter
985c21dfb7 Merge pull request #844 from dcooper16/has_server_protocol_fixes
has_server_protocol() fixes
2017-10-09 15:09:17 +02:00
Dirk Wetter
1758d18672 Merge pull request #842 from dcooper16/tls13_part1
Initial support for TLSv1.3
2017-10-09 14:53:32 +02:00
Dirk
6f896a057b add missing \n for html output 2017-10-08 22:03:29 +02:00