Commit Graph

2486 Commits

Author SHA1 Message Date
Dirk 6b8f6f82c3 Fix comments in run_logjam, replace awk by bash internal match expr
``run_logjam()`` contained in certain cases additional quotes
and a typo where only the word comment ended up in JSON/CSV.

Instead of ``$(awk '/Master-Key: / { print $2 }' "$2")`` the
admittedly performance sensitive function ``sclient_connect_successful()``
contains now a bash internal match according to #997 . First
tests didn't show much benefit (only default run with mostly
sockets was tested).

Unit tests showed no problems so far, but coverage is low.
2018-03-28 19:46:55 +02:00
Dirk 1924c9a0a6 Connectivity problems, man page update
See previous commit

This commit finally fixes #1005 so that either a --ssl-native scan
terminates on the next (defined) occasion if there are network connectivity
problems. It introduces another set of variables (MAX_OSSL_FAIL vs. NR_OSSL_FAIL).
As "openssl s_client connect" is sometimes still being used without --ssl-native
it also shortens the wait for regular scans if an outage is encountered.
To make things easier bot sets (incl. *_SOCKET_FAIL) of variables are independent.

For the seldom case that somebody uses --ssl-native with client checks an exception
had to be made as otherwise only MAX_OSSL_FAIL client check would be performed.
This hasn't been understood yet...

As sometimes HTTP header requests (over OpenSSL) fail repeatedly in a way that an empty
reply is returned, the same strategy of detecting problems is applied here,
using MAX_HEADER_FAIL and NR_HEADER_FAIL.

All three detection mechanisims share a new function connectivity_problem().
2018-03-28 17:48:04 +02:00
Dirk baabf2a61e Connectivity problems, readd some CVEs+CWEs
This commit finally fixes #1005 so that either a --ssl-native scan
terminates on the next (defined) occasion if there are network connectivity
problems. It introduces another set of variables (MAX_OSSL_FAIL vs. NR_OSSL_FAIL).
As "openssl s_client connect" is sometimes still being used without --ssl-native
it also shortens the wait for regular scans if an outage is encountered.
To make things easier bot sets (incl. *_SOCKET_FAIL) of variables are independent.

For the seldom case that somebody uses --ssl-native with client checks an exception
had to be made as otherwise only MAX_OSSL_FAIL client check would be performed.
This hasn't been understood yet...

As sometimes HTTP header requests (over OpenSSL) fail repeatedly in a way that an empty
reply is returned, the same strategy of detecting problems is applied here,
using MAX_HEADER_FAIL and NR_HEADER_FAIL.

All three detection mechanisims share the new function connectivity_problem().

In addition unit tests showed that some vulnerability checks lost their
CVEs+CWEs whcich have been readded. For ROBOT a CVE was added (F5)
2018-03-28 17:28:55 +02:00
Dirk 991521957d housekeeping and FIX #406 2018-03-28 11:46:53 +02:00
Dirk Wetter 653827cfe1
Merge pull request #1024 from dcooper16/fix_client_simulation
Fix client simulations
2018-03-27 09:10:55 +02:00
David Cooper 72e7259ab2
Fix client simulations
parse_tls_serverhello() checks $TLS_CLIENT_HELLO for a supported_versions extension, and if it contains one, checks that the negotiated version is listed in that extension. However, while $TLS_CLIENT_HELLO is always set in socksend_tls_clienthello() it is not set by client_simulation_sockets() (or any of the functions that client_simulation_sockets() calls). As a result, when the server's response to a client simulation is parsed, parse_tls_serverhello() may compare the negotiated version against the supported_versions extension from a ClientHello message from a previous test.

This PR fixes the problem by having client_simulation_sockets() set $TLS_CLIENT_HELLO.
2018-03-26 09:43:51 -04:00
Dirk 224d851f7b workaround for missing date format conversion (OpenBSD) in http date 2018-03-24 19:37:05 +01:00
Dirk 7c015c5a13 Dealing with OpenSSL connect problems (part1, framework)
In addition to 080840f655 and to
address #1005 eventually this is the first part of dealing
with connectivity problems. As opposed to the commit mentioned
above this is the counterpart for openssl which is useful for
just normal usage and for using --ssl-native.

It adds another global MAX_OSSL_FAIL (preset to 2) representing
a threshold how many unsucessful openssl connections are needed to
quit the whole scan. It should again reduce scan time in those
cases.

This is the framework part in sclient_connect_successful() which
is mainly commented out. The hooks will follow soon.
2018-03-22 16:59:57 +01:00
Dirk 4a24454880 FIX #1021 and make testssl.sh work again ;-( 2018-03-21 15:00:16 +01:00
Dirk ffd89f8dea Address 4xx after STARTTLS handshake seems to have succeeded
In certain cases (see e.g. #939) the server side "kind of" falls back
from a TLS channel to the underlying plaintext STARTTLS with a 5xx
or 4xx.

This PR adds the detection of 4xx for all protocols (if STARTTLS has been
used) and labels the detection as "likely not av" (and keeps the warning). This
leads to two JSON/CSV objects in those cases.

The detection isn't quite perfect as the issue #939 shows the first
TLS message fragment resembles a TLS alert.

As noticed it also removes the recommendation to increase the
debug level if the level is already sufficient.
2018-03-21 12:36:25 +01:00
Dirk 50f7c93046 missing HSTS in JSON had wrong severity
FIX #1010 where a missing HSTS header lead to HIGH severity
whereas screen output was on info level.

In order to emphasize the importance of HSTS a missing header
flag will now be slighly penalized. To be in line with HSTS=0
the latter needed to be readjusted to medium as well.

As suggested by krissi for better readability a missing HSTS
header will now be 'not offered' instead of '--' on screen.
2018-03-21 10:08:38 +01:00
Dirk Wetter 83b09067a4
Merge pull request #1017 from dcooper16/fix1013
Fix #1013
2018-03-21 09:35:35 +01:00
Dirk Wetter 27b217b301
Merge pull request #1020 from dcooper16/ossl_ver_appendix
Fix OSSL_VER_APPENDIX
2018-03-19 16:52:20 +01:00
David Cooper abff65bf99
Fix OSSL_VER_APPENDIX
This PR changes find_openssl_binary() so that $OSSL_VER_APPENDIX contains everything from $OSSL_VER that appears after $OSSL_VER_MAJOR.$OSSL_VER_MINOR.
2018-03-19 11:35:53 -04:00
David Cooper 0f7a49e2a3 Fix #1013
This PR fixes the issue raised in #1013. It primarily does this in two ways:

* In calls to `$OPENSSL s_client` that specify ciphers, the TLSv1.3 ciphers are provided separately using the `-ciphersuites` option. Then, the `s_client_options()` function manipulates the command-line options as necessary based on the version of OpenSSL being used.

* Calls to `$OPENSSL ciphers` were replaced with calls to `actually_supported_ciphers()`, which calls `$OPENSSL ciphers`. `actually_supported_ciphers()` modifies the parameters for the call to `$OPENSSL ciphers` as necessary, based on the version of OpenSSL being used.
2018-03-19 11:26:31 -04:00
Dirk Wetter a7d195fb89
Merge pull request #1018 from dcooper16/fix1015
Fix #1015
2018-03-19 16:15:49 +01:00
Dirk Wetter 2c8bdcd205
Merge pull request #1019 from dcooper16/ossl_ver
Extracting OpenSSL version information
2018-03-19 16:13:47 +01:00
David Cooper aa72fa839c
Extracting OpenSSL version information
When using testssl.sh with the current development branch of OpenSSL 1.1.1, determine_trust() incorrectly reports that "/openssl <= 1.0.2 might be too unreliable to determine trust." The problem is that find_openssl_binary() is not correctly extracting the version information. The version is "1.1.1-pre3-dev," but find_openssl_binary() extracts:

   OSSL_VER_MAJOR: 1
   OSSL_VER_MINOR: 1.13
   OSSL_VER_APPENDIX: -pre-dev

This PR fixes the problem and also eliminates the use of external functions in extracting version information for $OSSL_VER.

Note that this code makes a change from the current code. Currently, $OSSL_VER_APPENDIX is intended to contain anything from $OSSL_VER that comes after $OSSL_VER_MAJOR.$OSSL_VER_MINOR. For example, if $OSSL_VER is 1.1.0-dev, then  $OSSL_VER_APPENDIX is "-dev". In this PR, the "-dev" is dropped and so does not appear in $OSSL_VER_MAJOR, $OSSL_VER_MINOR, or $OSSL_VER_APPENDIX. The reason for this is that testssl.sh is only using $OSSL_VER_APPENDIX in cases in which $OSSL_VER_MAJOR.$OSSL_VER_MINOR is 0.9.8 to determine whether 0.9.8a - 0.9.8l or 0.9.8m - 0.9.8z. So, it seems that testssl.sh isn't interested in things such as "-dev" or "-pre3-dev".

If desired, this PR could be changed to that $OSSL_VER_APPENDIX contains everything in $OSSL_VER that appears after $OSSL_VER_MAJOR.$OSSL_VER_MINOR.
2018-03-19 10:59:26 -04:00
David Cooper 4c946bfd49
Fix #1015
This PR fixes #1015 by adding underscore ('_') to the list of characters that may appear in a domain name label. a178f3e183 already addressed this for the check of whether the Common Name field in a certificate is a DNS name. This PR fixes it for checks of whether a domain name is a wildcard name and whether the server's host name is a wildcard match against a name in a certificate.
2018-03-19 09:40:15 -04:00
Dirk a178f3e183 relax host+domain names to contain underscore, see #1015 2018-03-18 23:38:23 +01:00
Dirk c91a771465 FIX #1009 and leftover fix for #988
In cases where HTTP/2 was the only ALPN protocol, the
JSON output was missing the finding as the additional
protocol was empty.

This fix tests whether there's an additional protcol
and only calls fileout() if there's one.
2018-03-17 20:37:00 +01:00
Dirk 4eb6ab5976 better naming of variables from previous commit 2018-03-17 19:44:40 +01:00
Dirk 377bde14cc Fixes ssl-native and LOGJAM: no common primes
When testssl.sh was using -ssl-native and the local openssl version
didn't support any DH ciphers the deliberately lazy approach was to
not complete the second part of the test: check for common primes.
This is a FIX for issue #1016.
2018-03-17 15:40:28 +01:00
Dirk Wetter f6dc501aed
Merge pull request #1014 from dcooper16/tls13_cipher_pre_check
Check for TLSv1.3 in OpenSSL
2018-03-15 21:51:44 +01:00
Dirk Wetter 15ca770980
Merge pull request #1012 from dcooper16/tls13_draft26
TLS 1.3 drafts 25 and 26
2018-03-15 21:51:02 +01:00
David Cooper 2c1c9f33aa
Check for TLSv1.3 in OpenSSL
This PR adds a missing check for local TLSv1.3 support in cipher_pref_check(). If the "--ssl-native" mode is being used and $OPENSSL does not support TLSv1.3, then a warning should be displayed indicating that TLSv1.3 cannot be checked rather than displaying the same results as if the server did not support TLSv1.3.
2018-03-15 11:58:11 -04:00
David Cooper f7ab400167
TLS 1.3 drafts 25 and 26
Drafts 25 and 26 of TLS 1.3 were issued in early March. Neither made any change that is relevant to testssl.sh.

This PR adds checks for these two drafts.
2018-03-14 09:55:18 -04:00
Dirk Wetter 107d2dd12e
Merge pull request #1007 from dcooper16/revert_read_sigalg_from_file
Revert change to read_sigalg_from_file()
2018-03-12 21:38:44 +01:00
David Cooper 819e4505f1 Revert change to read_sigalg_from_file()
The implementation of read_sigalg_from_file() was changed on January 29 in 88cd5528e7. The new implementation does not work correctly in cases in which read_sigalg_from_file() is called with $TMPFILE as as parameter.

The current implementation of the function is:
```
read_sigalg_from_file() {
     local hostcert_txt="${1//pem/txt}"

     [[ -r "$hostcert_txt" ]] || $OPENSSL x509 -noout -text -in "$1" 2>/dev/null >$hostcert_txt
     awk -F':' '/Signature Algorithm/ { print $2; exit; }' $hostcert_txt
}
```

When called using $TMPFILE (/tmp/testssl.XXXXXX/tempfile.txt), hostcert_txt is set to $TMPFILE, and since this file exists and is readable, the next line does nothing and the final line tries to read the signature algorithm from $TMPFILE rather than from a parsed version of the certificate.

This PR reverts read_sigalg_from_file() to its previous implementation, at least as a temporary solution.
2018-03-12 15:29:10 -04:00
Dirk Wetter f718592960
Merge pull request #1006 from dcooper16/X448
Add X448 support in TLS 1.3
2018-03-12 19:43:35 +01:00
David Cooper 714933d607 Add X448 support in TLS 1.3
Support for X448 was recently added to the development branch of OpenSSL 1.1.1. This PR adds an X448 key pair to etc/tls_data.txt (that was generated using OpenSSL 1.1.1) and adds X448 to the supported_groups extension for TLS 1.3 ClientHello messages.
2018-03-12 13:40:22 -04:00
Dirk 9618ed993d Better parsing of HPKP
Previously testssl.sh ran into problems when both Public-Key-Pins
and Public-Key-Pins-Report-Only were present in the header.

Now if both are present only Public-Key-Pins is evalutated and a
hint is given. If Public-Key-Pins-Report-Only is only present
a hint is given and those are evaluated.

If more than 1 from either Public-Key-Pins-Report-Only or
Public-Key-Pins is detected only the first one will be processed
and a warning will be displayed.
2018-03-03 21:40:46 +01:00
Dirk 1011ef1399 polish 2018-03-02 21:07:33 +01:00
Dirk 2e5dd0439a document variable for previous commit 080840f 2018-03-02 20:57:06 +01:00
Dirk 080840f655 fix uninitialized variables, proper exit if fail2van and friends
This commit improves assessments of server which decides after
a few connects either to drop all tcp packets or send a tcp
reset, see #1005 . It adds another global MAX_SOCKET_FAIL
(preset to 2) representing a threshold how many bash socket
connections are needed to quite the whole scan. It dramatically
reduces scan time and makes sure that the user is clear that
the scan had a problem and the results at maximum partly useful
(CSV, HTML, JSON). It also writes a valid JSON file.
See also #769.

It also fixes non-initialized variables in ``fileout_json_finding()``
and ``fileout()``
2018-03-02 20:39:07 +01:00
Dirk b1b30b3fb8 better matching of empty server banners
.. which could have led to borken JSON too
2018-03-01 16:01:05 +01:00
Dirk 8c9fdc6cc9 Fix HPKP regression where it stated that only one SPKI was used 2018-03-01 15:29:28 +01:00
Dirk b5fcc00031 reflect previous commit of changed treatment of --severity
... and some minor polishing
2018-03-01 15:13:55 +01:00
Dirk d344a1ee04 JSON(PRETTY) + CSV output when scan interrupted, WARN severity
If a severity level >= INFO was supplied,JSON* and CSV output were
not writing error conditions in fileout(). As those indicate a problem
of the assessment, this has been fixed so that if any severity is
supplied to testssl.sh it will always write WARN conditions
to file when detected.

Also if the scan was interrupted on the client side where previously
a scanTime of 0 was written to the log file now it will raise a warning
that the scan was interrupted.

See also #769. Error conditions after starting a scan on the server side
(TCP Reset, DROP after a while) still won't be treated accurately in the file output
with an error message but the resulting JSON should be still valid.
2018-03-01 13:47:34 +01:00
Dirk abc956344a Address/FIX #876 (empty addcmd2 variable) 2018-02-27 13:06:29 +01:00
Dirk Wetter 3cf591f513
Merge pull request #1002 from danielchodusov/2.9dev
Using generic HAProxy name
2018-02-22 14:49:32 +01:00
Daniel Chodusov 6497157957
Using generic HAProxy name
This prime appears to be not only in HAProxy 1.5 but as well in the newer versions. The test result will return incorrect response message, when testing on the newer HAProxy versions (ie. 1.5 is detected but 1.8 is installed).
2018-02-22 13:57:02 +01:00
Dirk 415f98b655 Some improvements added. bash >= 3.2 is required 2018-02-19 11:55:12 +01:00
Dirk 5add07a798 make sure bash is at least 3.2, see #999 2018-02-19 11:31:03 +01:00
Dirk f812403292 Workaround for missing date conversion in OpenBSD
OpenBSD doesn't seem to be able to convert from $(date)
output to a more readable format as performed on notBefore
and notAfter from x509(1). Thus an exemption is made and
just display the format like x509(1) shows.

Also newer OpenBSD come with a newer LibreSSL version,
thus the pattern when to display a warning when the
"too unreliable to determine trust" warning is displayed.
2018-02-16 23:47:48 +01:00
Dirk Wetter 6f0a9aee11
Merge pull request #994 from dcooper16/extended_tls_sockets
TLSv1.3 draft 24
2018-02-15 21:18:45 +01:00
Dirk Wetter 022dd256e9
Merge pull request #993 from dcooper16/extra_extensions_bugfix
Bugfix for extra extensions
2018-02-15 21:16:55 +01:00
David Cooper 6536c21bec TLSv1.3 draft 24
This commit adds support for TLSv1.3 draft 24. The only change between draft 23 and draft 24 is that draft 24 requires the version number of the record layer to be "0303" for a second ClientHello, whereas draft 23 allowed the version number for both the initial and second ClientHello to be "0301".

The current text reads:

   legacy_record_version  This value MUST be set to 0x0303 for all
      records generated by a TLS 1.3 implementation other than an
      initial ClientHello (i.e., one not generated after a
      HelloRetryRequest), where it MAY also be 0x0301 for compatibility
      purposes.
2018-02-15 14:29:17 -05:00
David Cooper 41839ab221
Bugfix for extra extensions
If extra extensions are provided to socksend_tls_clienthello(), the socksend_tls_clienthello() needs to determine what extensions were provided so that it doesn't add any of these extensions a second time. The code that was looping through the extra extensions to get the extension IDs was using the value for the length of the string. This commit fixes the error.
2018-02-15 14:13:04 -05:00
Dirk e564783fec Addding private CAs
As per @krissi's suggestion this is a smart addition to add privat CAs
to all certificate stores -- during runtime only. The switch --add-ca or
--add-CA expects an argument of one additional CA file or a comma separated
lists of them. The enviroment file ADDITIONAL_CA_FILES can be used
alternatively.

This fixes eventually #230.
2018-02-15 19:28:22 +01:00