Commit Graph
100 Commits
Author SHA1 Message Date
David Cooper 8697e3417a Fix copy and paste error
This commit fixes a bug that was introduced by an accidental copy and paste.
2020-05-07 12:33:23 -04:00
David Cooper 46bf8c90ac Improve key exchange grading
This commit makes a couple of improvements to set_key_str_score().

It rates (finite-field) DH keys the same as RSA and DSA keys.

Second, in the case of a server that has more than one certificate, the current code sets $KEY_EXCH_SCORE based on the length of the public key in the last certificate that is parsed. This commit changes set_key_str_score() so that $KEY_EXCH_SCORE is set based on the weakest public key.

Note that there is still the issue that the key exchange score does not take into account any ephemeral keys used. However, that needs to be addressed by callling set_key_str_score() from run_fs() and run_logjam(), as certificate_info() cannot provide information about ephemeral keys.
2020-05-06 14:30:44 -04:00
David Cooper a8c9133fc6 Check for RSA exponent of 1
As suggested in #1576, set the grade cap to F if the RSA key has an exponent of 1.
2020-05-06 14:25:59 -04:00
David Cooper 9dba2a8c9c Fix #1576
This commit adds additional information to the "Server key size" line for a certificate if the subject public key is RSA, ECDSA, or DH.

For RSA it show the public exponent. For ECDSA, it shows the curve. For DH, it shows the group used, if it is a common prime.
2020-05-06 12:16:15 -04:00
David Cooper 329ba95d5b Fix #1433
This commit fixes #1433 by adding "@SECLEVEL=0" to the "$OPENSSL s_client" and "$OPENSSL ciphers" command lines if that option is supported. Adding this option configures OpenSSL to support some weak ciphers that it would not use in the default configuration.
2020-05-06 08:43:13 -04:00
David Cooper 84a82dbddc tls_sockets() return value
There is one place in parse_tls_serverhello() that returns 8 if the server's response is not well-formed TLS. However, there is no code in testssl.sh that is prepared to handle this return value. Every function except run_protocols() only distinguishes between 0, 2, and everything else. run_protocols(), however, gets confused if tls_sockets() returns a value that it is not expecting. So, this commit changes parse_tls_serverhello() to return 1 whenever the server's response can not be parsed.
2020-05-06 08:31:09 -04:00
David Cooper ae8a056afe Check server's response
There is code at the beginning of parse_tls_serverhello() that checks whether the server's response appears to consist of a sequence of messages of the form <protocol><content type><content>. However, at the moment the check is only performed if "$do_starttls" is false. This commit changes parse_tls_serverhello() so that the check is always performed.
2020-05-04 14:29:13 -04:00
David Cooper a5a28d2457 Improve LibreSSL 3.1.0 compatibility
This commit addresses two compatibility issues with LibreSSL 3.1.0, which has added client support for TLS 1.3.

The first issue is that LibreSSL has named the TLS 1.3 ciphers that it supports AEAD-AES256-GCM-SHA384, AEAD-CHACHA20-POLY1305-SHA256, and AEAD-AES128-GCM-SHA256, rather than using the OpenSSL names, which are TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, and TLS_AES_128_GCM_SHA256. (Draft versions of OpenSSL 1.1.1 names these ciphers TLS13-AES-256-GCM-SHA384, TLS13-CHACHA20-POLY1305-SHA256, TLS13-AES-128-GCM-SHA256.) There are several places where testssl.sh checks whether a cipher suite is a TLS 1.3 cipher by checking whether its OpenSSL name begins with "TLS_" (or "TLS13"). In order to work with LibreSSL 3.1.0, these checks also need to consider names that begin with "AEAD-" to be TLS 1.3 ciphers.

Second, in sub_session_resumption() there is code that adds "-no_ssl2" to the "$OPENSSL s_client" command line if that option is supported. If "-no_ssl2" is not supported, then other protocol information is added to the command line. I believe this code was written with the assumption that any version of OpenSSL that supports "-no_ssl2" does not support TLS 1.3. However, LibreSSL 3.1.0 supports both. So, this commit changes the code to add the "-no_ssl2" option only if TLS 1.3 is not supported.
2020-04-30 11:08:04 -04:00
David Cooper cb67d91417 Improve compatibility with LibreSSL 3.0.2 and earlier
This commit addresses two compatibility issues with LibreSSL.

First, with LibreSSL, "$OPENSSL s_client" does not support the "-curves" option, so the "-groups" option needs to be used instead. Note that with LibreSSL, the command line "$OPENSSL s_client -groups $curve -connect invalid." will not work, as it will complain "no port defined," but will not indicate whether the specified curve is supported. Adding a port number fixes that problem. (There does not seem to be a need to include a port number for other tests, such as whether the "-curves" option itself is supported.)

Second, including "-out -" in the command line for "$OPENSSL genpkey" causes LibreSSL to create a file with the name "-" if the algorithm is supported. This is not an issue at the moment, since LibreSSL's genpkey does not support X25519 or X448. However, both genpkey with both OpenSSL and LibreSSL uses stdout as the default output if no "-out" is specified, so the "-out -" is not necessary.
2020-04-30 10:37:12 -04:00
David Cooper 541d960924 Use $HAS_X25519 and $HAS_X448
generate_key_share_extension() and prepare_tls_clienthello() currently check the $OPENSSL version number to determine whether X25519 and X448 are supported. The commit changes these functions to use $HAS_X25519 and $HAS_X448.
2020-04-30 10:26:56 -04:00
David Cooper 3db9d74c21 Ticketbleed and TLS 1.3
run_ticketbleed() and sub_session_ticket_tls() each include one call to "$OPENSSL s_client". For each of these calls the expected response is a TLS 1.2 or earlier ServerHello. However, if $OPENSSL supports TLS 1.3, then a TLS 1.3 ClientHello will be sent.

This commit fixes this problem in two ways. For the call in run_ticketbleed(), "-no_tls1_3" is added to the command line if "$OPENSSL" supports TLS 1.3. For the call in sub_session_ticket_tls(), this commit changes the function so that the same ClientHello version is sent as will sent by run_ticketbleed() via sockets.
2020-04-29 10:13:22 -04:00
David Cooper 7f0c2e9137 Improve compatibility with OpenSSL 3.0
This commit fixes a couple of issues related to the use of testssl.sh with OpenSSL 3.0.0-alpha1.

First, when the command line includes an unknown option (e.g., -ssl2), OpenSSL 3.0.0-alpha responds with "Unknown option: -ssl2" rather than "Option unknown option -ssl2". This commit addresses this by making the check for "unknown option" case insensitve.

Second, the printing a DH key, OpenSSL 3.0.0-alpha1 labels the prime and the generator using "prime P:" and "generator G:" rather than just "prime:" and "generator:". This commit by changing testssl.sh to match on either string.
2020-04-23 15:20:50 -04:00
David Cooper bb1c649513 Fix run_logjam() in --ssl-native mode
This commit fixes a problem with run_logjam() when run in --ssl-native mode. If $OPENSSL does not support any DH export ciphers, then no test for such cipher is performed. However, the results of "test" is still checked, leading to testssl.sh incorrectly reporting that the server supports DH EXPORT ciphers.
2020-04-23 14:52:14 -04:00
David Cooper f5aa20ceb1 Extended run_server_preference()
This commit extends run_server_preference() to list every cipher supported by each protocol even in cases in which the server does not enforce a preference order.

For protocols where the server enforces a cipher order the list of supported ciphers is ordered by server preference (as now). For protocols where the server does not enforce a cipher order, the ciphers are listed by encryption strength (as run_cipher_per_proto() does).

In order to implement this, ciphers_by_strength() was extended to offer a non-wide mode.
2020-04-22 12:31:45 -04:00
David Cooper 04e51db402 Fix #1514
This commit is an attempt to fix #1514. The commit is mostly based on a suggestion at https://unix.stackexchange.com/questions/57940/trap-int-term-exit-really-necessary. Even with that change, it seemed that if testssl.sh were in the middle of executing run_cipher_per_proto() when it received a signal, it would not stop until that function had completed. This seems to have something to do with subshells. Changing the while loop in run_cipher_per_proto() seems to have fixed that issue. So, I also made similar changes to the while loops in prettyprint_local().
2020-04-02 08:03:45 -04:00
David Cooper b6050e68de Fix #1551
This commit fixes #1551 by changing get_cipher() to recognize RFC names that begin with SSL_*. It also modifies run_beast() so that it does not get stuck in an infinite loop if get_cipher() doesn't return a valid cipher name.
2020-04-01 13:34:29 -04:00
David Cooper 08d5146223 Align run_cipherlists() with pr_cipher_quality()
This commit modifies run_cipherlists() to align with pr_cipher_quality().

The biggest change made by this commit is that it breaks the current list of STRONG ciphers into two lists: one for AEAD ciphers that offer forward secrecy (STRONG) and one for AEAD ciphers that do not offer forward secrecy (GOOD).

The remaining changes are just minor tweaks:

* A few ciphers that use MD5 are moved from AVERAGE and 3DES to LOW.

* '!AECDH' was added to the OpenSSL description for LOW to catch one cipher in OpenSSL 1.0.2-chacha that offers no authentication that was being included in the LOW list.

This commit also changes sub_cipherlists() to change the output when a cipherlist with a rating of 6 is not present. There was a "FIXME" associated with this output, but it didn't matter before since there were no cipherlists with a rating of 6.
2020-04-01 11:27:24 -04:00
David Cooper 40dfd8b53b Handle GOST ciphers in pr_cipher_quality()
This PR modifes pr_cipher_quality() as proposed in #1548 so that GOST ciphers are handled correctly. It changes pr_cipher_quality() so that the OpenSSL name is used in cases in which no RFC name is defined. It also adds a case statement for GOST so that GOST ciphers (that do not use MD5 or Null encryption) are marked as pr_svrty_low (as they are in run_cipherlists) rather than just being assigned the default rating (5).
2020-04-01 11:18:50 -04:00
David Cooper 72dae035b5 Remove redundant entries
This commit removes two entries from a "case" test that were already covered by a previous entry.
2020-03-25 16:07:22 -04:00
David Cooper e15aea4790 Modify pr_cipher_quality to handle ARIA
This commit fixes the way pr_cipher_quality handles the OpenSSL names of some ARIA ciphers that either provide no authentication or that use CBC padding.
2020-03-25 15:57:00 -04:00
David Cooper d177a90bbe Adjust pr_cipher_quality ratings
This commit makes several changes to the way that ciphers are rated by pr_cipher_quality:

* It upgrades SEED ciphers to considered as strong as the corresponding AES ciphers.

* It downgrades ciphers that use AEAD, but that use a non-FS key exchange (TLS_DH_*, TLS_ECDH*, TLS_PSK_WITH_*) from best to good, thus giving them the same rating as AEAD ciphers that use static RSA (TLS_RSA_*).

* It downgrades some CBC ciphers to low (4) that are currently rated as neither good nor bad (5).

* It modifies the ratings created using OpenSSL names to provide the same ratings as those created using RFC names.
2020-03-25 15:28:08 -04:00
David Cooper 5ab73d1a1a Fix bug in setting DISPLAY_CIPHERNAMES
The permitted values for $DISPLAY_CIPHERNAMES are "rfc-only", "openssl-only", "openssl", and "rfc". However, get_install_dir() incorrectly sets $DISPLAY_CIPHERNAMES to "no-rfc" if it cannot find the $CIPHERS_BY_STRENGTH_FILE. ("no-rfc" is the string users would specify at the command line for the --mapping option, but not the value that $DISPLAY_CIPHERNAMES is set to internally).
2020-03-25 12:53:28 -04:00
David Cooper 58353d3522 Fix typo in emphasize_stuff_in_headers()
This commit fixes a typo in emphasize_stuff_in_headers() wherer ${yellow} was used rather than ${html_yellow} in the creation of the HTML output.
2020-03-06 14:25:07 -05:00
David Cooper 37dbe14def Fix printing percent characters
As noted in #1481, testssl.sh has a problem with printing percent ('%') characters.

At one point, the function out() was implemented as `/usr/bin/printf -- "${1//%/%%}"`. When this was the case, any '%' needed to be replaced with '%%' since '$1' was being used as the format string. This was changed, however, by https://github.com/drwetter/testssl.sh/commit/8a2fe5915a8d4f70ba423f98eff02a6646935aa9. Since the format string is now "%b" rather than '$1', the replacement is not needed anymore. Instead, the replacement now causes any '%' to be printed to be duplicated.

This problem does not happen very often, but does sometimes occur when a '%' character appears in a URI, such as in an HTTP redirect, a certificate revocation list, or an OCSP URI.
2020-03-06 08:28:52 -05:00
David Cooper 83e76a442b Fix handling of \n in strings 2020-02-27 13:59:05 -05:00
David Cooper b92f0de2c9 Fix HTML generation
This PR fixes two issues related to the generation of HTML files.

First, text that is to appear in the HTML file is first passed through html_reserved() to replace reserved characters with their corresponding entity names (e.g., '>' becomes '&gt;'). html_reserved() seems to work correctly on Ubuntu Linux, but it does not work as expected on MacOS. On MacOS, rather than converting '>' to '&gt;', it gets converted to '\&gt;', and the backslash is rendered by browsers.

This PR appears to fix the problem. However, given that the original version of html_reserved() was not portable, this revised version should be tested on multiple platforms.

I also noticed that in almost every case in which a string is passed to html_out(), it is first run through html_reserved(), but for some reason that is not the case in out() and outln(). I can't see any reason why html_reserved() is not called first in these two cases, so this PR adds in the calls.
2020-02-27 13:59:05 -05:00
David Cooper f342031844 Fix use of HSTS_MIN
This commit fixes two minor issues related to HSTS_MIN:

* If there is a misconfiguration the recommended max-age should be based on $HSTS_MIN rather than being hardcoded to 15552000 seconds = 180 days.

* If max-age is exactly $HSTS_MIN, testssl.sh shouldn't say that max-age is too short while also say that >= $HSTS_MIN seconds is recommended.
2020-02-20 14:17:49 -05:00
David Cooper 8d3640ca20 Fix #1504 by moving the description of the test out of the section that describes vulnerability tests. 2020-02-13 10:57:48 -05:00
David CooperandGitHub 6c88a26861 Wide output for cipher order
Since, in cases in which the server enforces a cipher order, both run_cipher_per_proto() and run_server_preference() list every cipher supported by the server for each protocol, there was a discussion at one point about eliminating run_cipher_per_proto() and extending run_server_preference().

This PR takes a step in that direction by providing the option to present the "Cipher order" in wide mode.
2020-02-12 11:05:20 -05:00
David CooperandGitHub 28d65247b0 Fix Shellcheck SC2197
This PR fixes one Shellcheck issue:

      In testssl_3.1dev_20200208.sh line 2395:
                HEADERVALUE="$(fgrep -Fai "$key:" $HEADERFILE | head -1)"
                               ^-- SC2197: fgrep is non-standard and deprecated. Use grep -F instead.

The man page for grep states that fgrep is the same a grep -F and that grep is deprecated. So, fgrep -F is just redundant.
2020-02-10 13:51:08 -05:00
David CooperandGitHub 2af5fa72d1 Minor code cleanup
This PR makes some minor changes to the code in order to clean it up a little.
2020-02-07 17:21:22 -05:00
David CooperandGitHub d916f048be neat_list() and export
Currently, the function neat_list() uses the variable "export", but does not define it. The result is that "export" variable in the calling function is used.

This PR fixes that by defining "export" as a local variable in neat_list() and by setting its value via a new parameter to the function.

This PR also removes a "FIXME" from run_rc4() since the problem has already been fixed.
2020-02-06 14:33:13 -05:00
David CooperandGitHub 3025d92ebf Fix public key length calculation
This PR fixes a minor bug in get_pub_key_size(). If the key size is being determined manually and length encoding requires 4 bytes, then the current code computes the length incorrectly. This is a very insignificant bug, since does not apply to RSA or ECC keys, and the key would have to be at least 16 megabytes long for it to require 4 bytes to encode.

This PR also cleans up get_pub_key_size() a bit by replacing `i=$i+...` with `i+=...` and by enclosing math in `$(( ... ))`.
2020-02-04 14:55:53 -05:00
David CooperandGitHub f756080a01 Fix typos
I noticed two places that had a single "&" rather than "&&". I didn't try to do any testing, but these certainly seem like typos.
2020-02-04 09:52:44 -05:00
David Cooper 46c05c6732 Fix client simulation
replace ciphers with ch_ciphers and sni with ch_sni in client simulation data file.
2020-01-31 10:52:50 -05:00
David CooperandGitHub aba544b188 TLS 1.3 post-handshake messages
This PR adds support for post-handshake messages when using sockets with TLS 1.3 connections. If a TLS 1.3 connection is established and the connection is to remain open after tls_sockets() finishes, then after the client's Finished message is sent the master secret and the application traffic keys are computed. This PR also adds two new functions to send and receive application data over a TLS 1.3 connection.

This PR also includes two proofs-of-concept for the use of the new functions. receive_app_data() is called immediately after the client's Finished message is sent. Some server's will send new session tickets immediately after the handshake is complete. If they do, then the code will decrypt and parse the session ticket messages.

This PR also modifies service_detection() to try using sockets if the server only supports TLS 1.3 and $OPENSSL does not support TLS 1.3. After the handshake is complete, this code sends an HTTP GET request and reads the response. The code is fairly slow and it doesn't always work. However, since it is only used in cases in which $OPENSSL cannot work, it can't hurt to try using sockets.
2020-01-30 15:20:25 -05:00
David CooperandGitHub 5d2978fdbe Minor code cleanup
This PR cleans up the code in a few places by using "case" statements in a few places where the value of a variable is checked multiple times.
2020-01-30 14:00:46 -05:00
David CooperandGitHub 19db41742f Suppress shellcheck issue SC2178
Due to a bug, the shellcheck program will complain if a variable is defined as an array but is later used as an ordinary string, even if the two uses are locally defined variables in different contexts. The error message is:

   SC2178: Variable was used as an array but is now assigned a string.

While the warnings are not highlighting any actual problems in testssl.sh, this PR gets rid of the warnings by renaming a few variables.
2020-01-30 13:53:33 -05:00
David Cooper 1ea631addd Speedup AES-GCM
The implementation of AES-GCM in #1473 is much slower than the original version, even when the authentication tag is not being computed. This PR modifies the code in gcm() in order to significantly speed up the encryption/decryption time (when authentication tags are not being computed).
2020-01-29 12:12:04 -05:00
David CooperandGitHub 7516c01315 TLS 1.3 Finished messages
This PR adds processing of the Finished messages in TLS 1.3 handshakes. It also addresses some shellcheck issues.

If in debug mode, the HMAC of the transcript hash of the handshake context ($msg_transcript) is computed and compared against the Finished message sent by the server.

If the full server response is parsed and the connection with the server is not to be closed when tls_sockets() completes, then the TLS 1.3 handshake is completed by creating the client Finished message and sending it to the server.
2020-01-28 11:15:03 -05:00
David CooperandGitHub b8d414b432 Reorganize TLS 1.3 key derivation
This PR reorganizes the code for deriving TLS 1.3 symmetric keys in order to facilitate implementing the full key schedule. For example, rather than having a single function to derive the handshake traffic keys, this PR creates one function to derive the handshake secret and a separate function to derive the handshake traffic keys. The second function has been generalized so that it can derive either client or server traffic keys. Separating into two functions also makes the handshake_secret available for later use to derive the master secret and then the application traffic secrets and the application traffic keys.

This PR also changes where there message transcript is created, a message transcript will also be needed to derive the application traffic secrets. This PR includes the code to add the messages to the initial message transcript that will be needed for the input to the application traffic secret derivation function.
2020-01-27 09:52:15 -05:00
David CooperandGitHub 351bb7a4e8 Full AEAD cipher implementations
RFC 8446 specifies cipher suites that use three symmetric encryption algorithms, all of which are Authenticated Encryption with Associated Data (AEAD) algorithms. In each of these algorithms when data is encryption an authentication tag is created, which allows the recipient to verify that the data has not been modified. The authentication may also cover some additional data that was not encrypted.

The current implementations of these algorithms in testssl.sh decrypt the ciphertext, but do not check that the authentication tag is correct (which involves the recipient computing the correct tag for the received data and then comparing it to the provided tag). While testssl.sh can get away with not checking authentication tags when receiving data, the ability to compute authentication tags is needed in order to send encrypted data as TLS servers would reject any encrypted data that did not have a correct authentication tag. Being able to send encrypted data is necessary to be able to complete the TLS 1.3 handshake.

This PR replaces the current implementations of the symmetric encryption algorithms with full implementations of each of the algorithms. These full implementations include the ability to encrypt data for sending, and can also verify the authentication tag when decrypting data. Since the Bash implementations of these algorithms is very slow, the decryption code is designed to only compute and check authentication tags in debug mode.

While the implementation of the code to compute authentication tags for AES-CCM was based on NIST Special Publication 800-38C, I was not able to implement the code for AES-GCM or Poly1305 from their specifications (NIST Special Publication 800-38D and RFC 8439, respectively). So, I would very much like to thank the implementers of https://github.com/mko-x/SharedAES-GCM and https://github.com/floodyberry/poly1305-donna. The implementations of AES-GCM and Poly1305 in the PR were developed by translating the C code in https://github.com/mko-x/SharedAES-GCM and https://github.com/floodyberry/poly1305-donna into Bash. I don't understand what that code is doing, but it seems to work. :-)

I have only tested this code on a computer with a 64-bit operating system. While I have not tested it, I believe that the decryption code will work with 32-bit integers if not in debug mode (i.e., if not trying to compute the authentication tags). I also believe that the AES-CCM code for computing authentication tags will work with 32-bit integers. However, AES-GCM and Poly1305 code for computing authentication tags will definitely only work on systems that have 64-bit integers. So, on systems that do not have 64-bit integers, encryption will not work for AES-GCM or ChaCha20-Poly1305, and decryption will not work for these algorithms if in debug mode.
2020-01-24 15:26:13 -05:00
David CooperandGitHub 43d83b27d4 Simplify code to determine draft TLS 1.3 version
This PR simplifies the code for determining which draft version of TLS 1.3 a server is offering by making use of a simple regular expression and $BASH_REMATCH rather than looping through every possible draft version.
2020-01-22 13:48:27 -05:00
David CooperandGitHub cce57c4613 Fix run_ssl_poodle()
PR #1463 changed run_ssl_poodle() to only run the test if it is known that the server supports SSLv3. However, support for SSLv3 may be unknown at the time run_ssl_poodle() is run (e.g., if the server supports TLS 1 and SSLv3, and run_ssl_poodle() is the first test performed). So, run_ssl_poodle() should perform testing unless it is known that SSLv3 is not supported.
2020-01-22 11:20:34 -05:00
David CooperandGitHub d49d96ae40 Undo copy and paste error
This PR removes what appears to be a copy and paste error introduced by #1463.
2020-01-22 11:14:55 -05:00
David CooperandGitHub 4b6bdf8cdf More polishing of http basic auth
* Replace "! -z" with "-n"
* Replace "openssl' with "$OPENSSL"
* Redirect stderr output of $OPENSSL to /dev/null to supress "WARNING: can't open config file: /usr/local/etc/ssl/openssl.cnf" message (see #833)
* Remove unnecessary spaces from $GET_REQ11 string.
2020-01-16 13:41:27 -05:00
David CooperandGitHub 477b113fe6 Add missing variable declarations
derive-handshake-traffic-keys() uses the variables `derived_secret`, `server_write_key`, and `server_write_iv`, but they are not declared as local variables of the function. This PR fixes that.
2020-01-14 13:53:36 -05:00
David CooperandGitHub f181efb352 Don't write to /dev/stdout
As noted in #1273, there are some environments that will not allow writing to /dev/stdout. PR #1277 was an attempt to address that problem (along with an unrelated problem), but it appears that work on #1277 has been abandoned.

At the moment, "/dev/stdout" is only used as a parameter to asciihex_to_binary_file (in fact, most calls to asciihex_to_binary_file specify "/dev/stdout" as the file parameter). This PR removes the file parameter from asciihex_to_binary_file (and so renames it asciihex_to_binary). In most cases, this just means removing "/dev/stdout" as a parameter to the function. In the few cases in which a parameter other than "/dev/stdout" was provided to asciihex_to_binary_file, this PR just uses a redirect (">" or ">>") to accomplish the same result as providing the output file to asciihex_to_binary_file().

Note that #1273 and #1277 raised the issue of trying to write to /tmp, and this PR does not attempt to address that.
2020-01-14 09:10:23 -05:00
David CooperandGitHub 855758b3af Fix run_server_preference() in --ssl-native mode
This PR fixes two problems that occur when testing a server that supports TLSv1.3 using OpenSSL 1.1.1 in --ssl-native mode.

First, when testing whether the server has a cipher order, the value of $sclient_success is checked after each call to tls_sockets(), but $sclient_success. As the goal is just to verify that the connection was successful (and didn't downgrade), $? can be checked rather than $sclient_success. [When not in --ssl-native mode, this problem is masked since $sclient_success is set to 0 earlier in the function.]

The second problem is that line 6646 tries to copy "$TEMPDIR/$NODEIP.parse_tls13_serverhello.txt", but this file is currently only created (on line 6287) if tls_sockets() is used to determine the negotiated protocol. This PR fixes the problem by also populating "$TEMPDIR/$NODEIP.parse_tls13_serverhello.txt" when OpenSSL is used to determine the negotiated protocol.
2020-01-13 10:51:34 -05:00
David Cooper 420fa73f5a Fix Safari 13.0 Client Simulation
The ciphersuites string for Safari 13.0 ends with a colon (':'). which causes OpenSSL to reject the command line when client simulation testing is performed in --ssl-native mode. This PR fixes the problem by removing the trailing colon.
2020-01-13 10:31:20 -05:00
David Cooper 7c1b8139b2 Minor tweak to run_tls_fallback_scsv()
If the server is known not to support TLS 1.3 (as well as TLS 1.2, TLS 1.1, and TLS 1), then mention TLS 1.3 in the list of not supported protocols. While lack of TLS 1.3 support is not part of the reason that no fallback is possible, it is part of the reason that the result is reported as prln_svrty_high.
2019-12-27 09:33:08 -05:00
David Cooper a0b2fb5d56 Minor tweak to run_tls_fallback_scsv()
Don't report "OK" if the server may be TLS 1.3-only or SSLv3-only, as one is very good and one is very bad.
2019-12-27 09:33:08 -05:00
David Cooper 3f25bcc6d4 Minor tweaks to run_tls_fallback_scsv() 2019-12-27 09:33:08 -05:00
David Cooper e457ff8cc1 Improve check for $low_proto in run_tls_fallback_scsv()
If $high_proto is set to something other than SSLv3, support for SSLv3 will not have been determined by determine_optimal_sockets_params(), but it may have been determined later (e.g., by run_protocols()). So, this commit changes the loop to always check for SSLv3 support (without calling "$OPENSSL s_client" if $HAS_SSL3 is false). The check for whether the fallback test can be performed is moved until after the loop
2019-12-27 09:33:08 -05:00
David Cooper 00f613f62d WIP: Don't call s_client for unsupported protocol versions
This PR fixes a couple of places where "$OPENSSL s_client" is called with "-ssl3" even if SSLv3 is not supported.

The fix in ciphers_by_strength() is easy, as the issue only occurs if "$using_sockets" is true. If SSLv3 (or TLSv1.3) is not supported, then testing using "$OPENSSL s_client" is skipped and all of the supported ciphers are found using tls_sockets().

The fix for run_tls_fallback_scsv() is more complicated. While it is easy to avoid calling "$OPENSSL s_client" with "-ssl3" if SSLv3 is not supported, it is not easy to determine the correct message to present to the user if support for SSLv3 (and possibly also TLSv1.3) is unknown.

For the case in which $high_proto cannot be set, I believe that I have covered all of the possibilities, but an not sure if the correct message/rating is used in every case.

For the case in which it is not possible to determine whether SSLv3 is the $low_proto, more could be done. If $high_proto is TLS 1.1 or TLS 1, then this PR is okay, as it is possible that SSLv3 would be the fallback protocol, but there is no way to tell. However, it seems unlikely that a server would support TLS 1.2 and SSLv3, but not TLS 1.1 or TLS 1. So, perhaps if $high_proto is TLS 1.2 and the server does not support TLS 1.1 or TLS 1, it should just be assumed that SSLv3 is not supported, even if it cannot be tested.
2019-12-27 09:33:08 -05:00
David CooperandGitHub ce4f923c6b Fix TLS 1.3 key derivation
At the moment, testssl.sh does not correctly derive the handshake traffic keys in the case that the server responds to the initial ClientHello with a HelloRetryRequest. The handshake traffic keys are computed incorrectly because the wrong messages are provided to derive-handshake-traffic-keys() for input to the Transcript-Hash calculation:

* TLS_CLIENT_HELLO is not updated to contain the value of the second ClientHello, and so the first ClientHello is being provided to derive-handshake-traffic-keys() as both the first and the second ClientHello.

* In middlebox compatibility mode the server may send a dummy ChangeCipherSpec message immediately after the HelloRetryRequest. Since it is part of the server's response to the initial ClientHello, the ChangeCipherSpec message is included in the $hrr that is sent to derive-handshake-traffic-keys(), but it should not be included in the computation of the Transcript-Hash.

This PR fixes the above two problems by updating TLS_CLIENT_HELLO when a second ClientHello is sent and by removing any ChangeCipherSpec message (140303000101) from the end of the server's initial response.
2019-12-23 15:37:26 -05:00
David Cooper 0b94a14614 Indent subsequent rows of HTTP security headers by two spaces
When printing a long HTTP security header, this commit causes every row after the first one to be indented by two additional spaces. In the case of very long headers, this extra indentation makes it a little easier for readers to see where the next security header begins.
2019-12-03 16:19:01 -05:00
David CooperandGitHub 725fdc11cb Fix formatting of HTTP security headers
When printing out HTTP security headers, run_security_headers() uses out_row_aligned_max_width(), since some headers are very long and need to be wrapped. At the moment, however, the first line is too long. The problem is that while "$header $HEADERVALUE" is printed in the indented area, only $HEADERVALUE is passed to out_row_aligned_max_width().

This PR fixes the problem by passing "$header $HEADERVALUE" to out_row_aligned_max_width() so that the the first line is wrapped at the correct place.
2019-12-03 15:38:16 -05:00
David Cooper 9cb95e9f70 Align cipher rating numbers
Currently sub_cipherlists() and pr_cipher_quality() use different numbers for the same cipher quality ratings. sub_cipherlists() uses:

   -2 = pr_svrty_critical, -1= pr_svrty_high, 0 = pr_svrty_low, 1 = pr_svrty_good, 2 = pr_svrty_best

while pr_cipher_quality() uses:

   1 = pr_svrty_critical, 2 = pr_svrty_high, 3 = pr_svrty_medium, 4 = pr_svrty_low
   5 = neither good nor bad, 6 = pr_svrty_good, 7 = pr_svrty_best

This PR changes sub_cipherlists() (and run_cipherlists()) to use the same numbers for cipher quality as pr_cipher_quality(). It does not change any of the ratings assigned to ciphers by run_cipherlists() or pr_cipher_quality(), so the two are still not in alignment. But, hopefully using the same numbering in both functions will make it a bit easier to compare them and bring them into alignment.
2019-11-25 10:07:33 -05:00
David CooperandGitHub 2394dba9b2 Fix indentation
This PR fixes some indentation issues. The PR is a bit long, but it only makes changes to indentation (except for one comment line, where a trailing space character is removed).
2019-11-21 11:11:30 -05:00
David CooperandGitHub 084bf8fa75 Fix #1385
This PR fixes #1385.

sub_session_resumption() returns 3 when $CLIENT_AUTH is true. However, the comment at the beginning of the function indicates that 6 will be returned. run_server_defaults() is prepared to handle a return value of 6 (to indicate client auth), but is not expecting 3 as a possible return value.
2019-11-20 09:22:52 -05:00
David CooperandGitHub b15b39a5cb Support new OpenSSL/LibreSSL versions
This PR enhances support for the latest versions of OpenSSL and LibreSSL.

The development version of OpenSSL at https://github.com/openssl/openssl/ is version 3.0.0-dev. So, checks for OpenSSL versions need to support this version as well. At the same time, the latest versions of LibreSSL are 3.0.0, 3.0.1, and 3.0.2, so version number alone will no longer be sufficient to distinguish between OpenSSL and LibreSSL.

In addition to checks for these new version numbers, this PR addresses a couple of other issues:

 - In LibreSSL, the "$OPENSSL ciphers" command will not accept any protocol version other than "-tls1" as a parameter (and even including "-tls1" as an option is described as "deprecated").  So, this PR ensures that "$OPENSSL ciphers" is not passed any protocol version option other than "-tls1" is LibreSSL is being used.
- In OpenSSL 3.0.0-dev, the "$OPENSSL dgst" can no longer be used to compute HMACs, but a new "$OPENSSL mac" function has been created. So, this PR changes hmac() to use "$OPENSSL mac" with OpenSSL 3.0.0-dev.

Note that I have not tested the modified version of sub_session_resumption(). I am just assuming that OpenSSL 3.0.0-dev works the same as OpenSSL 1.1.1 and that all versions of LibreSSL work the same as OpenSSL 1.1.0 and earlier.
2019-11-14 14:24:09 -05:00
David CooperandGitHub aab7e028c2 Enhance run_protocols() in --ssl-native mode
When tls_sockets() is used for run_protocols(), for each protocol version the results will indicate whether the server responded to a ClientHello for that protocol version with (1) a ServerHello for that same protocol version, (2) a ServerHello for a different (hopefully lower) protocol version, or (3) a handshake failure.

Currently, however, run_prototest_openssl() does not distinguish between cases in which the server responds with a ServerHello for a different (hopefully lower) protocol version and cases in which the server responds with a handshake failure. This PR changes run_prototest_openssl() so that it distinguishes between these two cases (as long as $OPENSSL supports the protocol version specified in the ServerHello).

Making use of the additional information provided by run_prototest_openssl(), this PR also modifies run_protocols() to check that version negotiation was performed correctly even if $using_sockets is false.

Note that one special case needed to be addressed. If an SSLv3-only server is being tested using an $OPENSSL that does not support SSLv3, then $latest_supported must not be set to SSLv3. In the case of a server like this, it is possible that support for SSLv3 will be determined by determine_optimal_sockets_params(), which will cause run_protocols() to report that the server supports SSLv3, even though $OPENSSL does not support SSLv3 and testing is being performed in --ssl-native mode. If $latest_supported were set, then later tests in run_protocols() would incorrectly report a version negotiation failure, even though the failure to connect was a result of a limitation of $OPENSSL rather than a fault of the server.
2019-11-13 10:46:51 -05:00
David CooperandGitHub a7fe481904 Don't ignore first call to $OPENSSL s_client
run_prototest_openssl() currently calls "$OPENSSL s_client" twice, once with $PROXY and once without. The problem is that the results of the first call are just ignored. This commit changes run_prototest_openssl() so that the attempt without $PROXY is only tried if the first attempt was unsuccessful.
2019-11-07 13:12:41 -05:00
David CooperandGitHub 8e729d1396 Missing line break
If --ssl-native is being used and the server supports SSLv2, but does not support any SSLv2 ciphers, there is a missing line break after the warning message is printed.
2019-11-07 13:03:42 -05:00
David CooperandGitHub c607bf4d92 Check stderr for "no cipher list"
run_prototest_openssl() currently checks only stdout for the string "no cipher list", which is an indication that the server supports SSLv2, but no ciphers for that protocol. However, the output that includes "no cipher list" is sent to stderr.
2019-11-07 13:01:21 -05:00
David CooperandGitHub 9d97db85fc Fix typos in comment 2019-11-07 12:57:58 -05:00
David CooperandGitHub 54fad800c0 Fix issue with run_protocols() in --ssl-native mode
This PR fixes a minor problem with run_protocols() in "--ssl-native" mode if $OPENSSL does not support TLS 1.3. Currently, the warning message that $OPENSSL does not support a protocol is printed when run_prototest_openssl() is called. This causes a problem for the output if $OPENSSL does not support TLS 1.3, since the run_prototest_openssl() is called before the results for TLS 1.2 are printed. The result is something like this:

 SSLv2      not offered (OK)
 SSLv3      not offered (OK)
 TLS 1      offered (deprecated)
 TLS 1.1    offered (deprecated)
Local problem: /home/cooper/Desktop/testssl.sh/bin/openssl.Linux.x86_64 doesn't support "s_client -tls1_3"
 TLS 1.2    offered (OK)
 TLS 1.3     NPN/SPDY   not offered
 ALPN/HTTP2 http/1.1 (offered)
2019-11-06 15:58:38 -05:00
David CooperandGitHub 7add991156 Fix #1366
When processing a command line for parallel mass testing, create_mass_testing_cmdline() did not account for the newer shortened versions of the output file options: -oj, -oJ, -oC, and -oH. This caused the command line for the child processes to be incorrect.
2019-11-05 16:10:34 -05:00
David CooperandGitHub 2810c70163 Address comments in #1205
This commit addresses the comments in #1205. If a server only supports TLS 1.3, then it is not considered an issue if the server does not enforce a cipher order. However, if the server does not support a cipher order for TLS 1.2 and below, then that is an issue, even if the server does support a cipher order for TLS 1.3.
2019-10-28 16:15:38 -04:00
David Cooper beec1a7e1e Use results of determine_optimal_sockets_params() 2019-10-28 15:02:49 -04:00
David Cooper 3ea1b1b884 WIP: Separate server preference test (cipher order) for TLS 1.3
This PR is an attempt to fix #1163 by running separate tests for a server cipher order preference to TLSv1.3 and for SSLv3 - TLSv1.2.

If the server supports TLSv1.3, then a test is performed to determine whether the server enforces a cipher order to TLSv1.3. A separate test is performed for SSLv3 - TLSv1.2 unless it is known that the server does not support any of these protocols.

If the server enforces a cipher order for SSLv3 - TLSv1.2, but not for TLSv1.3, then cipher_pref_check() is not called for TLSv1.3, since cipher_pref_check() is intended to show the cipher order that the server enforces. As TLSv1.3 will be the negotiated protocol if it is supported, the negotiated cipher for TLSv1.3 will already be presented.

This PR still has one major flaw, which may create a problem when testing a TLSv1.3-only server. If run_protocols() is run before run_server_preference(), then everything will be okay, as run_server_preference() will be able to determine that SSLv3 - TLSv1.2 are not supported. However, if run_server_preference() is run by itself, run_server_preference() will not know that SSLv3 - TLSv1.2 are not supported and so it will try to determine whether the server enforces a cipher preference order for these protocols. The attempt to connect to the server will fail, but at the moment run_server_preference() doesn't know whether the failure is because the server does not support SSLv3 - TLSv1.2 or because the server supports at least one of these protocols, but does not support any ciphers in $list_fwd. At the moment, run_server_preference() incorrectly flags an error.

One option would be to perform additional tests against the server in this case to determine the reason for the connection failure. Another option would be to have some code that is always run earlier, such as determine_optimal_proto(), test whether a server that supports TLSv1.3 supports any earlier protocols (SSLv3 - TLSv1.2).
2019-10-28 15:02:49 -04:00
David CooperandGitHub 8a0f94f561 Use $jsonID rather than literal string
In run_drown(), $jsonID is set to "DROWN" and most calls to fileout() are of the form

     fileout "$jsonID" ...

However, one call is written as

    fileout "DROWN" ...

This PR changes this one call to be consistent with the others. This does not change the functionality of the program.
2019-10-28 12:43:29 -04:00
David CooperandGitHub 42c8769983 Fix do_starttls initialization bug
At the moment, $do_starttls is initialized to true in initialize_globals() and then it is set to true again in parse_cmd_line() if the --starttls command line option is used. Presumably the intention was to set $do_starttls to false in initialize_globals().
2019-10-28 10:15:05 -04:00
David CooperandGitHub be073e6134 Fix parse_tls_serverhello() bug
This PR fixes a minor bug in parse_tls_serverhello(). In some cases the server's entire response is not retrieved. In these cases, it is possible that the response from the server ends with a portion of a handshake message.

The loop at the beginning of parse_tls_serverhello() extracts the various handshake and alert messages from the server's response. If it gets to the end of the response, and what is at the end is not a complete message, it should just ignore that fragment and break out of the loop. At the moment, however, parse_tls_serverhello() just continues in the loop rather than breaking out. This has not been a problem up to now, since $msg_len is usually set to a positive value from a previous iteration of the loop, which causes the loop to end.

In the case of the server identified in #1353, however, $msg_len is 0 and so the continue rather than break results in an endless loop.
2019-10-28 10:06:21 -04:00
David CooperandGitHub 73edf6fa8e Fix client simulation bug
This PR fixes a bug in modify_clienthello() that occurs when client simulation is being performed, the ClientHello contain an SNI extension, and $SNI is empty. In the case, modify_clienthello() should just skip over the SNI extension and not include one in the modified ClientHello. However, the code currently only skips over the 2-byte extension type. The result being that the remainder of the extension is included in the modified ClientHello. This PR fixes the problem by ensuring the $offset is advanced whether or not $SNI is empty.
2019-10-23 11:03:52 -04:00
David CooperandGitHub 877d444300 Warn if bad version negotiation detected
There are a few places where testssl.sh sends a TLS 1.2 (or TLS 1.3) ClientHello and expects the server to respond with a ServerHello as long as it supports TLS 1.2 (or TLS 1.3) or earlier.

run_protocols() performs a fairly thorough check for a server's ability to handle version negotiation, but the problem may also be caught by determine_optimal_sockets_params(), if the server rejects a TLS 1.2 ClientHello even though it supports some earlier protocol version.

In the future, we could try to make use of $OPTIMAL_SOCKETS_PROTO in order to make testssl.sh work a bit better with servers (if any still exist) that don't handle version negotiation correctly. At the moment, though, this PR just prints a warning to the user that the server is buggy, and that this may lead to problems in the scan. It doesn't call fileout() to add anything to the JSON/CSV output, since run_protocols() should already be doing that.
2019-10-07 10:21:04 -04:00
David CooperandGitHub 30b93d4c72 Use determine_optimal_sockets_params() in run_protocols()
This PR modifies run_protocols() to use the information collected by determine_optimal_sockets_params(). If it has already been determined that a protocol is supported, then no test is run. run_protocols() will still run a test for a protocol even if it has been determined that the server does not support that protocol. The reason for running the test is to verify that the server handles version negotiation correctly. This could be a TLSv1 server that rejects a TLSv1.2 or TLSv1.3 ClientHello, or it could happen in the opposite direction. At one point there was a server that would respond to an SSLv3 ClientHello with a TLSv1.2 ServerHello.

This PR required a couple of changes to determine_optimal_sockets_params() so that additional information could be passed to run_protocols(). If the server supports TLS 1.3, then run_protocols() needs to know which version (RFC 8446, draft 28, draft 27, etc.) rather than just that TLS 1.3 is supported. If the server supports TLS 1.2, but not TLS 1.3, then run_protocols() needs to know about at least one TLS 1.2 cipher that the server supports so that it can form a TLS 1.3 ClientHello that has no more than 128 ciphers and that should result in the server returning a TLS 1.2 ServerHello.
2019-10-04 16:55:09 -04:00
David CooperandGitHub 9b3ab29550 Modify check for TLS13_ONLY
In a PR that I'm developing to to use the results of determine_optimal_sockets_params() in run_protocols() I add specific versions of TLS 1.3 to PROTOS_OFFERED (e.g., tls1_3_rfc8446, tsl1_3_draft28). If that PR is accepted, then the current check for TLS 1.3-only will no longer work. So, this commit changes the way that the check for TLS 1.3-only is performed in order to avoid problems if the other PR is merged.
2019-10-03 16:18:51 -04:00
David CooperandGitHub 4f462eb718 Simplify determine_sizelimitbug()
This PR takes advantage of the testing done by determine_optimal_sockets_params() in order to simplify determine_sizelimitbug().

By the time that determine_sizelimitbug() is called, determine_optimal_sockets_params() has already determined whether TLSv1.2 ClientHello with 128 ciphers (including 00FF) sent by tls_sockets() works, and it has set TLS12_CIPHER to a list of exactly 128 ciphers (including 00FF) that works with the server. So, determine_sizelimitbug() doesn't have to check whether the server supports TLSv1.2 and no longer needs to send tests using 127 or 128 ciphers. determine_sizelimitbug() can just perform one test with 129 ciphers, if the server supports TLSv1.2, and use the results to set $SERVER_SIZE_LIMIT_BUG.
2019-10-02 13:21:08 -04:00
David Cooper ca29015e9c Use OpenSSL for determine_optimal_proto()
This PR reverts determine_optimal_proto() to use OpenSSL again rather than tls_sockets().

The primary reason for this is that the primary purpose of determine_optimal_proto() is to set OPTIMAL_PROTO, which is only used with $OPENSSL s_client. So, the best way to determine what works best on the $OPENSSL s_client command line is to use $OPENSSL s_client.

In most cases, determine_optimal_proto_sockets_helper() would set OPTIMAL_PROTO to an acceptable value, but it might not always do so. For example, suppose that a server
* supports different cipher suites with different protocols, 
* supports TLSv1.2, but only with cipher suites not supported by $OPENSSL, but
* supports TLSv1.1 with at least one cipher suite supported by $OPENSSL.

In the above case, determine_optimal_proto_sockets_helper() would set OPTIMAL_PROTO to "-tls1_2", but testing using $OPENSSL would result in OPTIMAL_PROTO being set to "-tls1_1".

Using $OPENSSL for determine_optimal_proto() also allows for edge cases to be detected earlier:
* If the server only supports TLSv1.3, and $OPENSSL does not support TLSv1.3, then the code in this PR will detect that (rather than waiting until run_protocols() is executed).
* The code in this PR can also detect if the server only supports SSLv3 (and possibly also SSLv2), but $OPENSSL does not support SSLv3.
* This code can also detect the (rare) case in which connections using $OPENSSL succeed, but connections using tls_sockets() fail.

[Note also that in the current code, if $all_failed is true, then a message may be printed that $OPENSSL is not IPv6 aware, even if testing was performed using tls_sockets() rather than $OPENSSL.]
2019-10-02 13:08:52 -04:00
David CooperandGitHub 644d7c839e Update
This commit addresses TLSv1.3 servers that do not support session tickets by that support session resumption by ID, but only with TLSv1.2 or earlier.
2019-10-01 16:25:51 -04:00
David CooperandGitHub 0fe60e82a8 Fix sub_session_resumption()
This PR fixes an issue with sub_session_resumption() when using OpenSSL 1.1.1.

As noted in #1335, some servers will return a session ticket for TLSv1.2, but not for TLSv1.3.

OpenSSL 1.1.1 does not support the "-no_ssl2" option, and so when using OpenSSL 1.1.1 sub_session_resumption() adds $OPTIMAL_PROTO to the $OPENSSL s_client command line. When determine_optimal_proto_sockets_helper() is called, $OPTIMAL_PROTO will generally be set to "-tls1_2" (or "-tls1_1" or "-tls1") unless the server is a TLSv1.3-only server. As a result  sub_session_resumption() will specify that same protocol on the command line if OpenSSL 1.1.1 is being used.

If "--ssl-native" is used, however, then determine_optimal_proto() will set $OPTIMAL_PROTO to "-tls1_3" if the server supports TLSv1.3 (and doesn't use STARTTLS). Similarly, if the version of determine_optimal_proto() in #1336 is used, then $OPTIMAL_PROTO will usually be empty. In either case, sub_session_resumption() will send a TLSv1.3 ClientHello, even if the server only supports session tickets for TLSv1.2 and below.

This PR appears to fix the problem. This PR makes no changes when using a version of OpenSSL that supports "-no_ssl2". When using a version of OpenSSL that does not support "-no_ssl2", however, rather than using $OPTIMAL_PROTO, this PR has sub_session_resumption() use whatever protocol version the server connected with when $sessticket_lifetime_hint was set.
2019-10-01 15:48:02 -04:00
David Cooper 11486d3bca Reset OpenSSL dependent variables
There is a problem if a TLSv1.3-only server is tested using the OpenSSL 1.0.2-chacha binary and $OSSL_SHORTCUT is true.

$HAS_NO_SSL2 is set to true when find_openssl_binary() is called with OpenSSL 1.0.2-chacha. /usr/bin/openssl does not have the -no_ssl2 option, but the second call to find_openssl_binary(), after setting $OPENSSL to /usr/bin/openssl, does not set $HAS_NO_SSL2 to false. So, later calls to $OPENSSL s_client include the -no_ssl2 option, resulting in connection failures.

This PR fixes the problem by modifying find_openssl_binary() to ensure that every OpenSSL-dependent variable is set by this function.
2019-10-01 08:51:15 -04:00
David Cooper 8ec4087121 Fix session ticket issues
This PR fixes two issues with finding session tickets when using OpenSSL 1.1.1.

First, if OpenSSL connects to the server using TLSv1.3 and it receives more than one Post-Handshake New Session Ticket, then the "TLS session ticket lifetime hint" will appear more than once in $TMPFILE. This will cause the line to appear more than once in $sessticket_lifetime_hint, which causes problems when trying to extract the $lifetime and $unit from $sessticket_lifetime_hint.

This PR fixes the first problem by changing the awk expression in the lines that set sessticket_lifetime_hint so that only the first line with "session ticket lifetime" is extracted.

The second issue is that some servers (e.g., google.com) return a session ticket for TLSv1.2, but not for TLSv1.3. For such servers, testssl.sh will miss the session ticket if $OPTIMAL_PROTO is empty or "-tls1_3" and the --ssl-native flag is not set.

This PR addresses the second issue with the changes in lines 9047 - 9053 -- the code that is intended to provide a last chance to find a session ticket.

If $OPENSSL supports TLSv.1.3 and the server returns session tickets for TLSv1.3 connections, then the session ticket would have already been found by get_server_certificate(), since get_server_certificate() uses $OPENSSL for TLSv1.3 if $OPENSSL supports TLSv1.3. So, in such circumstances, the code in liens 9047 - 9053 should not try again with TLSv1.3. So, if $OPENSSL supports TLSv1.3 and $OPTIMAL_PROTO is empty or is set to "-tls1_3" (either of which would result in a TLSv1.3 ClientHello), the "$OPENSSL s_client" call is changed to specify -no_tls1_3 rather than $OPTIMAL_PROTO.

The code on line 9047 is also changed to only make this final try is $TLS13_ONLY is false. If $TLS13_ONLY is true, then either:
* $OPENSSL does not support TLSv1.3 and the connection attempt would fail anyway; or
* $OPENSSL supports TLSv1.3, in which case any session ticket would have been found by get_server_certificate(), since get_server_certificate() uses $OPENSSL for TLSv1.3 if $OPENSSL supports TLSv1.3.

In either case, there is no reason to try again to find a session ticket.
2019-09-30 16:50:40 -04:00
David CooperandGitHub 6d1d74a7a4 Fix #1328
This PR fixes #1328 by removing any '\r' characters that appear in the command line read from the file provided to the `--file` option.
2019-09-24 15:53:32 -04:00
David Cooper 5cebf17287 Add more checks to determine_optimal_sockets_params()
This commit adds some more checks to determine_optimal_sockets_params(). These additional checks will almost never need to be run, and so will not slow down the typical run of testssl.sh, but adding them will provide information that can be useful for other parts of testssl.sh.

These additional checks will only be run if the server does not support TLSv1.3 and a TLSv1.2 ClientHello is not successful. This means that either:
   * The server is not an TLS/SSL enabled server.
   * The server only supports SSLv2.
   * The server supports some protocol in SSLv3 - TLSv1.1, but does not handle version negotiation correctly.

Adding these additional checks helps in at least the following ways.

If determine_optimal_proto() (assuming it is reverted to using OpenSSL) is unable to connect to the server using OpenSSL, it will be possible determine whether the problem is the the server does not support TLS/SSL or that a different version of OpenSSL is needed to test the server.

If the code in #1205 for run_server_preference() is unable to connect when checking for a cipher order, the reason for the failure will be known, making it possible to determine the correct response to the failure.
2019-09-23 15:57:07 -04:00
David Cooper ce9bc79422 Add determine_optimal_sockets_params()
This PR adds a new helper function that is run just prior to determine_optimal_proto() and that determines the what information tls_sockets() should include in a ClientHello.

For a TLSv1.3 ClientHello, determine_optimal_sockets_params() determines whether tls_sockets() should use 0x33 or 0x28 are the extension number for the key_share extension. 0x33 should be used with servers that support RFC 8446 or drafts 23-28. 0x28 should be used with servers that support drafts 18-22.

For a TLSv1.2 ClientHello, determine_optimal_sockets_params() determines what cipher list tls_sockets() should send. For most servers, the list of ciphers in $TLS12_CIPHER works best. But, there are some servers that do not support any ciphers in $TLS12_CIPHER, but do support one or more ciphers in $TLS12_CIPHER_2ND_TRY.
2019-09-23 15:56:08 -04:00
David Cooper 80a725541b Allow TLS12_CIPHER to be changed
In some rare cases a server does not support any of the ciphers in $TLS12_CIPHER, but does support at least one cipher in $TLS12_CIPHER_2ND_TRY. In such cases, TLS12_CIPHER should be changed to $TLS12_CIPHER_2ND_TRY so that subsequent tests using $TLS12_CIPHER will succeed.
2019-09-23 15:54:44 -04:00
David CooperandGitHub 25d68544ed More run_protocol() fixes
This PR fixes a few issues with run_protocol():

* In the case that the call to `tls_sockets "03" "$TLS12_CIPHER"` had a return value of 2, the code determining what results to print was looking at `$DETECTED_TLS_VERSION`. However, the value of this variable was set by the later call to `tls_sockets "04" "$tls13_ciphers_to_test"`. This caused incorrect results in the case of a server that supports TLSv1.3 and TLS1.1 (or earlier), but not TLSv1.2. This PR saves the value of `$DETECTED_TLS_VERSION` in `$tls12_detected_version` and then uses this variable later rather than `$DETECTED_TLS_VERSION`.

* When running in debug mode with a server that does not support TLSv1.3, testssl.sh was printing

           TLS 1.3     -- downgradednot offered and downgraded to a weaker protocol"

This PR fixes the output by not printing the "--downgraded"

* As noted in #1329, run_protocols() was treating a downgrade from TLSv1.2 as less bad if the server supports TLSv1.3. This PR changes this code back to treat any downgrade from TLSv1.2 as equally bad.

* In order to be consistent with the TLSv1.3 test, this PR changes the TLS1.2 test output to say "not offered and downgraded to a weaker protocol" if a TLSv1.2 ClientHello results in a downgraded connection.
2019-09-23 11:26:40 -04:00
David CooperandGitHub 76fb81112b Fix run_protocols()
This PR fixes a problem in run_protocols() that was introduced by https://github.com/drwetter/testssl.sh/commit/7ec3c6ab997b012b1518903137d84043cf9825a8.

https://github.com/drwetter/testssl.sh/commit/7ec3c6ab997b012b1518903137d84043cf9825a8 changes run_protocols() to perform the initial testing for TLSv1.3 support before testing for TLSv1.2 support. The problem with this is that the code for testing TLSv1.3 makes use of the results of the TLSv1.2 testing.

In the current code, Line 5183 looks at the value of $subret to determine whether the TLSv1.2 ClientHello resulted in a successful connection. However, $subet has not yet been set (it has just been initialized to 0 at the beginning of the function). Since $subret will always be 0, the code will try to extract a cipher from $TEMPDIR/$NODEIP.parse_tls_serverhello.txt. This may work, since $TEMPDIR/$NODEIP.parse_tls_serverhello.txt may have been populated by a prior function call, but this is not how the code was intended to work.

This PR fixes the problem by doing the TLSv1.2 testing before the TLSv1.3 testing is done. It still waits until both have been tested, however, before outputting the results, so that the output for TLSv1.2 can be modified depending on whether TLSv1.3 is supported.
2019-09-20 17:37:11 -04:00
David CooperandGitHub 2a141ab0f4 Fix typo in check of $COLOR
In emphasize_stuff_in_headers() there is a check of the $COLOR value, which checks whether $COLOR is 2. It should check whether $COLOR is 2 or higher.
2019-06-24 14:49:20 -04:00
David Cooper e2042e99d6 Initial support for certificate compression
draft-ietf-tls-certificate-compression specifies a new TLS extension that allows a client to indicate support for receiving the server's certificates in compressed form. This PR adds initial support for that extension to testssl.sh. It was developed based on an announcement that facebook.com has implemented support for the extension.

This PR does not add a test for a server's support for the new extension, it just adds code to parse_tls_serverhello() that will parse a compressed certificate message if one is present in the server's response. So, in practice, the code in this PR will not actually be used until additional code has been added that sends a ClientHello with the compress_certificate extension.

The code in this PR can be tested by using the --devel option and by changing line 19347 from

     tls_sockets "$TLS_LOW_BYTE" "$HEX_CIPHER" "ephemeralkey"

to

     tls_sockets "$TLS_LOW_BYTE" "$HEX_CIPHER" "all+" "00,1b, 00,03, 02, 00,01"

testssl.sh can then be called using

     testssl.sh --debug 3 --devel 04 "13,01" facebook.com

Note that this PR adds a test for zlib support to find_openssl_binary(). The test uses $OPENSSL zlib in the same way it is used in parse_tls_serverhello(). The test asks $OPENSSL zlib to uncompress the compressed version of the string "zlib" and the checks to see whether the decompression was performed correctly. There is already a test for zlib support in run_crime(), but I did not check to see whether the check in run_crime() could be replaced with a check of the new $HAS_ZLIB variable.
2019-06-06 12:17:32 -04:00
David CooperandGitHub 0d2b955e21 Check pwnedkeys.com database
This PR adds a check of whether the server's public key appears in the https://pwnedkeys.com database.
2019-05-22 10:11:34 -04:00
David CooperandGitHub b081f5fffc Mark only TLSv1.3 final as pr_svrty_best
This PR changes run_protocols() so that, when using tls_sockets(), support for TLSv1.3 is only marked as pr_svrty_best() if the final (RFC 8446) version is supported. It also changed run_protocols() so that support for TLSv1.3 is marked as pr_svrty_best() if OpenSSL is used (i.e., if the --ssl-native option is specified).

One potential issue is that the --ssl-native version assumes that if OpenSSL supports TLSv1.3 it supports the final (RFC 8446) version of the protocol. If the tester is using a development version of OpenSSL 1.1.1 rather than the final version, then the protocol test will actually be indicating whether the server supports the same draft version of OpenSSL as the $OPENSSL being used to perform the tests.
2019-04-29 15:28:46 -04:00
David CooperandGitHub a1289d1ec3 The -outfile, -oa, -outFile, and -oA options should accept a directory
Currently the -outfile, -oa, -outFile, and -oA assume that <fname> being provided is to be used as a filename, unless it is "auto." However, all of the individual options (e.g., --logfile) allow for a directory name to be provided instead of a file name.

This PR changes the handling of the -outfile, -oa, -outFile, and -oA options so that if a directory name is provided, the files are created in that directory.
2019-04-25 14:55:13 -04:00
David CooperandGitHub 66d15c6a03 Fix #1243
This PR fixes #1243 by modifying create_mass_testing_cmdline() to handle the --outfile, -oa, --outFile, and -oA options in the case that the filename provided is not "auto." It also modifies create_mass_testing_cmdline() so that in the case of serial testing -oj and -oJ are treated the same as --jsonfile and --jsonfile-pretty, respectively.
2019-04-25 14:10:09 -04:00
David CooperandGitHub 2f4ce4a276 Fix shellcheck issue SC2128
This PR addresses the following issues raised by shellcheck:

In ../github/testssl_2.9dev_20190409b.sh line 1133:
if [[ "$BASH_VERSINFO" == 3 ]]; then
       ^-- SC2128: Expanding an array without an index only gives the first element.

In ../github/testssl_2.9dev_20190409b.sh line 4301:
          tmpfile_handle $FUNCNAME.dd
                         ^-- SC2128: Expanding an array without an index only gives the first element.

In ../github/testssl_2.9dev_20190409b.sh line 4388:
     tmpfile_handle $FUNCNAME.dd
                    ^-- SC2128: Expanding an array without an index only gives the first element.

In ../github/testssl_2.9dev_20190409b.sh line 4657:
     tmpfile_handle $FUNCNAME$1.txt
                    ^-- SC2128: Expanding an array without an index only gives the first element.

In ../github/testssl_2.9dev_20190409b.sh line 5327:
     tmpfile_handle $FUNCNAME.${debugname}.txt
                    ^-- SC2128: Expanding an array without an index only gives the first element.

In ../github/testssl_2.9dev_20190409b.sh line 5943:
          tmpfile_handle $FUNCNAME.byID.log $tmpfile || \
                         ^-- SC2128: Expanding an array without an index only gives the first element.

In ../github/testssl_2.9dev_20190409b.sh line 5944:
          tmpfile_handle $FUNCNAME.byticket.log $tmpfile
                         ^-- SC2128: Expanding an array without an index only gives the first element.

In ../github/testssl_2.9dev_20190409b.sh line 12410:
     tmpfile_handle $FUNCNAME.dd $SOCK_REPLY_FILE
                    ^-- SC2128: Expanding an array without an index only gives the first element.

In ../github/testssl_2.9dev_20190409b.sh line 13164:
               tmpfile_handle $FUNCNAME.dd
                              ^-- SC2128: Expanding an array without an index only gives the first element.

In ../github/testssl_2.9dev_20190409b.sh line 13284:
     tmpfile_handle $FUNCNAME.dd $SOCK_REPLY_FILE
                    ^-- SC2128: Expanding an array without an index only gives the first element.

In ../github/testssl_2.9dev_20190409b.sh line 13388:
     tmpfile_handle $FUNCNAME.dd $SOCK_REPLY_FILE
                    ^-- SC2128: Expanding an array without an index only gives the first element.
--
In ../github/testssl_2.9dev_20190409b.sh line 13801:
               [[ "$DEBUG" -ge 1 ]] && echo $tls_hello_ascii >$TEMPDIR/$FUNCNAME.tls_hello_ascii${i}.txt
                                                                       ^-- SC2128: Expanding an array without an index only gives the first element.
2019-04-11 12:05:10 -04:00
David CooperandGitHub 9d2061fdf9 Fix #1223
This PR fixes #1223 by checking whether the stapled OCSP response from the server is an error message.

Another way to fix #1223 would be to just change line 8510 to:
```
if grep -a "OCSP Response Status" <<< "$ocsp_response_status" | grep -q successful || \
     [[ "$ocsp_response" =~ Responder\ Error: ]]; then 
```
However, I believe this alternative would lead to confusing results, testssl.sh would print

     offered, error querying OCSP responder (tryLater)

I'm not sure whether it makes sense to say "offered" when the stapled response that is provided is just an error message, but I think it is important to make clear that the error response was received from the TLS server, and that it wasn't testssl.sh that tried querying the OCSP responder.
2019-04-01 14:21:45 -04:00
David CooperandGitHub 9ec70fa4d9 determine_optimal_proto_sockets_helper() speedup and bug fix
There is currently a bug in determine_optimal_proto_sockets_helper(). In two places there is code of the form:

   tls_sockets ...
   if [[ $? -eq 0 ]]; then
        ...
   elif [[ $? -eq 2 ]]; then
        ...
   fi

This code does not work as intended since the second check ("elif [[ $? -eq 2 ]]") is actually comparing the results of the first check to 2 rather than the results of the call to tls_sockets().

This PR fixes that problem and also speeds up the code. Since tls_sockets() sets $DETECTED_TLS_VERSION to the protocol version that was negotiated, there is no need to scan $TEMPDIR/$NODEIP.parse_tls_serverhello.txt for this information.
2019-03-13 16:17:50 -04:00