Commit Graph

2582 Commits

Author SHA1 Message Date
Dirk Wetter 488cb03c60
Merge pull request #1078 from dcooper16/stapled_ocsp_revocation_check
Check stapled OCSP response for revocation status
2018-06-28 20:25:46 +02:00
David Cooper b5595a9205
Check stapled OCSP response for revocation status
In cases in which the server offers a stapled OCSP response, this commit extracts the OCSP response and then checks the response for the status of the server's certificate. The check is performed in the same way as when the certificate includes an OCSP URI and the "--phone-out" option is set, except that the OCSP response is received from the TLS server rather than coming directly from the OCSP responder. Since this only involves additional processing of data that testssl.sh is already receiving, the check is performed whether or not the "--phone-out" flag is set.
2018-06-28 14:15:55 -04:00
Dirk 4aabc3f999 Merge branch 'skip_offensive' into 2.9dev 2018-06-26 13:10:18 +02:00
Dirk 33cf1d524c Fine tuning if Jac2NL's commit of IDS evasion
Reduce the offensive tests to 4: the others are "just" / mostly cipher
based checks which should not cause an IDS to block. (This maybe
subject to reconsider at a later time.)

Added a switch --ids-friendly

Updated VULN_COUNT accordingly

Added this (including PHONE_OUT to env debugging output)

Added help()

Manual section added
2018-06-26 13:04:30 +02:00
Dirk 01f177199c Merge branch 'Jac2NL-2.9dev' into skip_offensive 2018-06-26 12:36:01 +02:00
Dirk 3ce22ed8df Merge branch '2.9dev' of https://github.com/Jac2NL/testssl.sh into Jac2NL-2.9dev 2018-06-26 12:35:29 +02:00
Dirk 87f0cda234 Fix #1071 : typos and link in docu 2018-06-25 18:31:55 +02:00
Jacco de Leeuw 7098b7be50 If the environment variable OFFENSIVE has been set to false, skip a number of checks that have offensive characteristics. 2018-06-24 17:30:36 +02:00
Dirk Wetter 0355145576
Merge pull request #1074 from dcooper16/fix615
Fix #615
2018-06-22 14:18:23 +02:00
David Cooper 30624ddb75
Fix #615
This PR fixes #615 for the case in which tls_sockets() is used by splitting the list of CBC ciphers into two lists, each with fewer than 128 ciphers and then testing each list separately.

For the --ssl-native case, no changes were needed. Even though $cbc_ciphers contains 154 ciphers, no version of OpenSSL supports all of these ciphers, and so the actual ClientHello sent by every version of OpenSSL contains fewer than 128 ciphers.

I did, however, add the -no_ssl2 flag to the "$OPENSSL s_client" command to prevent OpenSSL from sending an SSLv2-compatible ClientHello. As is noted in a comment in run_server_preference(), "the supplied openssl will send an SSLv2 ClientHello if $SNI is empty and the -no_ssl2 isn't provided."
2018-06-21 11:23:30 -04:00
Dirk aa0f33e984 Update RFC section in ~/doc with soon to be TLS 1.3 RFC
See PR #1072, title taken from
https://github.com/ietf/draft-ietf-tls-tls13/blob/master/rfc8446.xml
(maybe subject to change).

Todo: Handle the obsolted ones, maybe by adding "obsolete"
2018-06-20 09:41:51 +02:00
Dirk Wetter 8e144e8e6f
Merge pull request #1072 from dcooper16/rfc8446
TLS 1.3 will be RFC 8446
2018-06-20 09:25:10 +02:00
David Cooper af8f98a04d TLS 1.3 will be RFC 8446
According to https://www.theregister.co.uk/2018/06/19/ietf_calls_for_formal_tls_1_0_1_1_deprecation, TLS 1.3 will be published as RFC 8446. This seems to be confirmed by https://www.rfc-editor.org/auth48/rfc8446 and https://github.com/ietf/draft-ietf-tls-tls13.

This PR updates comments that refer to draft-ietf-tls-tls13 to instead refer to RFC 8446. It also makes minor changes to other comments related to TLS 1.3 drafts.

NOTE: This PR is a bit premature as it may be several weeks before RFC 8446 is actually published.
2018-06-19 14:19:34 -04:00
Dirk f2944205b9 Fix output error when $HEADERFILE was empty 2018-06-19 10:27:42 +02:00
Dirk d23241b8cd Fix to-be-expired-soon certificate
The certificate from self-signed.badssl.com was about to expire
which raises a MEDIUM type issue in testssl.

This commit does a workaround for this, so that those certificates
will be ok in Travis CI.

(Same problem exists in 2.9.5)
2018-06-13 14:30:35 +02:00
Dirk b6cc01a558 Fix line feeds in vulnerabilty output when running in wide mode
This commit is a FIX for #1069, thus when running in
wide mode it corrects an additional line feed which
happened sometimes.

As @dcooper16 pointed out it also cleans up the needless
if-statements in run_rc4(), run_lucky13() and run_beast().

It also inserts for wide mode lines a blank so the alignment
is not at the left border anymore (check for leftovers
needed).
2018-06-13 13:25:02 +02:00
Dirk Wetter 1be25f9cf7
Merge pull request #1067 from dcooper16/revocation_checking_errors
Fix false "revoked" results for CRL and OCSP checking
2018-06-05 10:33:42 +02:00
David Cooper 02d1071b9c Reduce redundant code
Move some checks into functions so that the code doesn't have to be repeated.
2018-06-01 13:35:05 -04:00
David Cooper 998c2aa1f8
Fix false "revoked" results for CRL and OCSP checking
This PR fixes problems with check_revocation_crl() sometimes reporting that a certificate is revoked even when it isn't, and with check_revocation_ocsp() sometimes reporting "error querying OCSP responder" even if the OCSP responder provided a good response. The most common reason for this to happen is that OpenSSL cannot validate the server's certificate (even without status checking). PR #1051 attempted to get status checking to work even in cases in which the server's certificate could not be validated. This PR instead addresses the problem by not checking status if determine_trust() was unable to validate the server's certificate.

In some cases the server's certificate can be validated using some, but not all of the bundles of trusted certificates. For example, I have encountered some sites that can be validated using the Microsoft and Apple bundles, but not the Linux or Mozilla bundles.

This PR introduces GOOD_CA_BUNDLE to store a bundle that could be used to successfully validate the server's certificate. If there is no such bundle, then neither check_revocation_crl() nor check_revocation_ocsp() is run. When check_revocation_crl() and check_revocation_ocsp() are called, the status checks within them closely match the validation check in determine_trust(), which helps to ensure that if the check fails it is because of the status information.

As noted in #1057, at least one CA provides incorrect information when the CRL is downloaded, so validation could fail for a reason other than the certificate being revoked. So, this PR adds a check of the reason that validation failed and only reports "revoked" if the validation failed for that reason.

As noted in #1056, it is not possible to perform an OCSP query without access to the certificate issuer's public key. So, with this PR check_revocation_ocsp() is only called if the server's provided at least one intermediate certificate (i.e., the issuer's certificate, which contains the issuer's public key).
2018-05-29 16:39:46 -04:00
Dirk Wetter 626e0fc65a
Merge pull request #1066 from dcooper16/ocsp_error_responses
OCSP error handling
2018-05-25 22:28:05 +02:00
David Cooper 5e7f1b75f6
Check for HTTP errors
Added back in check for HTTP error codes.
2018-05-25 10:51:22 -04:00
David Cooper feb48c0295
OCSP error handling
This PR improves the handling of error responses when checking status using OCSP. It can handle a few types of errors:

* When the responder just returns an error (e.g., "Responder error: unauthorized").

* When the response cannot be verified (e.g., invalid signature, expired certificate).

* When the response is valid ("Response verify OK"), but there is a problem with the response for the individual certificate (e.g., information is too old, or status is "unknown").
2018-05-24 17:00:27 -04:00
Dirk c21ed3212e add a small hint how to use the tuning vars 2018-05-24 13:09:56 +02:00
Dirk 770f3ea832 fix typo in cmd line arg for -oA 2018-05-24 10:48:25 +02:00
Dirk Wetter ec57b64339
Merge pull request #1064 from dcooper16/ocsp_checking
OCSP improvements
2018-05-23 22:38:50 +02:00
David Cooper 0e27cfeadd
OCSP improvements
This PR fixes two issues with OCSP checking. First, the syntax for specifying a host header changed in OpenSSL between versions 1.0.2 and 1.1.0. With OpenSSL 1.0.2-chacha, 1.0.2o, and LibreSSL the syntax needs to be "-header HOST <hostname>". With OpenSSL 1.1.0h and 1.1.1 the syntax needs to be "-header HOST=<hostname>". I have not been able to test other versions of OpenSSL 1.0.2 or 1.1.0, but am assuming that all versions of OpenSSL 1.1.0 use the same syntax as 1.1.0h.

This PR also fixes a typo in the case of an error, which was causing $code to be set to "empty ocsp response" if the response was not empty rather than if it was empty.
2018-05-23 14:32:51 -04:00
Dirk f426a3bb21 Merge branch 'crl_ocsp' into 2.9dev
OCSP revocation checks ran with a 100% success rate for the Alexa ~Top 1000
(basis: all hosts supporting TLS and having an OCSP URI)
2018-05-23 15:09:03 +02:00
Dirk Wetter d578adb05c
Merge pull request #1062 from dcooper16/certificate_validity_dates
Fix extraction of certificate validity dates
2018-05-23 10:51:29 +02:00
David Cooper d272aff336
Fix extraction of certificate validity dates
The current code for extracting the validity dates for certificates assumes that the strings "Not Before" and "Not After" will appear exactly once in the pretty-print of the certificate. In most cases that works. However, there are a few server certificates that include the private key usage period extension, which also includes "Not Before" and "Not After" times. The result is that the current code does not correctly extract the start date and end date from any certificates that have private key usage period extensions.

This PR fixes the problem and also speeds up extraction of the dates by only using Bash internal functions.

The pretty-print of a certificate begins as follows:
   Certificate:
       Data:
           Version: 3 (0x2)
           Serial Number: ...
       Signature Algorithm: ...
           Issuer: ...
           Validity
               Not Before: ... GMT
               Not After : ... GMT
           ...

The code in this PR extracts the start date by first removing from the certificate everything that comes before "Not Before: ". It looks for the shortest string that includes ""Not Before: " in order to ensure it is not getting the date from the private key usage period extension. After that, the longest string that begins with "GMT" is removed so that only the notBefore date remains. 

The part that removes the string up to "Not Before: " actually looks for the first instance of "Not Before: " that comes after the "Validity". This is to protect against the unlikely possibility that the string "Not Before: " appears somewhere in the issuer's name.

The extraction of the notAfter date works similarly. It first looks for the first instance of "Not After :" that appears after both "Validity" and "Not Before: " and then takes the date string that appears immediately afterwards, with the assumption that the date string ends in "GMT".
2018-05-21 15:14:05 -04:00
Dirk 92eaa3e7d3 Merge #1047 before processing #1051 2018-05-18 22:39:19 +02:00
Dirk Wetter 31b26da691
Merge pull request #1060 from dcooper16/multicomponent_ocsp_uris
HTTP URLs with multiple components in path
2018-05-18 21:40:56 +02:00
David Cooper 6152592973
HTTP URLs with multiple components in path
In some cases the OCSP URI contains multiple components in the path (e.g., http://www.example.com/OCSP/myOCSPresponder).

This PR changes check_revocation_ocsp() to remove all components in the path, rather than just the final component, when extracting the host name from the URI for the host header.
2018-05-18 15:35:10 -04:00
Dirk 7d36734a96 Handle host header in OCSP request properly
My previous commit added a host header but didn't properly
format the host header (trailing slashes / path). This commit
corrects that so that the 305 times HTTP 400 in #1056
should now be gone (TBC), including Google CA responders.

One issue which needs to be addressed (same as in CRL
revocation checks): Not trusted certificates (zhanqi.tv,
taken from my Alexa scans) fail for obvious reasons.
2018-05-18 20:30:37 +02:00
Dirk 82aae158ba Minor additions to OCSP revocation check
Error from OCSP responder is now being displayed (and logged to JSON, ...)

Whole replay is kept in $tmpfile for debugging purposes

JSON output added for OCSP responderi query failures

Furtermore wget was replaced by "type -p" and grep by fgrep.
2018-05-16 19:52:10 +02:00
Dirk fac65d82b0 Add host header to OCSP Request
Some OCSP responder seems to need a host header, see e.g.
https://blog.ivanristic.com/2014/02/checking-ocsp-revocation-using-openssl.html .

This commit adds this header.

It addresses not all errors though. E.g. "https://testssl.sh" is fine now,
"https://google.com" still returns "Code=400,Reason=Bad Request" which
needs further investigation,

Also this commit gives a warning if the OCSP request fails (fileout needs
to be added)
2018-05-16 15:51:45 +02:00
Dirk Wetter 567cc05fca
Merge pull request #1055 from dcooper16/ocsp_checking
Add OCSP checking
2018-05-16 11:11:00 +02:00
David Cooper 79c9536d47
Add OCSP checking
This commit adds the option to query the OCSP server(s) specified in a server's certificate for the certificate's revocation status.
2018-05-15 16:50:57 -04:00
Dirk Wetter e68b1ce025
Merge pull request #1054 from dcooper16/fix1053
Fix #1053
2018-05-15 17:54:09 +02:00
David Cooper a84d970f3d
Fix #1053
It appears that #1053 was caused by a typo that was introduced by 39647d1703.
2018-05-15 11:43:39 -04:00
Dirk Wetter be743e0c9b
Merge pull request #1052 from dcooper16/draft28
TLS 1.3 draft 28
2018-05-07 10:58:49 +02:00
Dirk Wetter a787ba0ec5
Merge pull request #1048 from dcooper16/ldap_get_failure
ldap_get() should fail if curl fails
2018-05-07 10:55:54 +02:00
David Cooper 1cfe4072b6
TLS 1.3 draft 28
https://github.com/tlswg/tls13-spec/wiki/implementations now lists a server that supports TLS 1.3 draft 28, so this PR adds supports for drafts 27 and 28.

Since run_protocols() now checks for 11 different drafts of TLS 1.3 in addition to the final version, performing a separate test for each draft had become far too time consuming. So, this PR rewrites the check for TLS 1.3 versions in run_protocols() so that the number of tests is proportional to the number of drafts that the server supports rather than the number of drafts that testssl.sh can check for.
2018-05-04 15:05:49 -04:00
David Cooper 87a7fa5df6 ldap_get() should fail if curl fails
ldap_get() incorrectly returns 0 if curl indicates an error.
2018-05-02 10:43:11 -04:00
Dirk Wetter b6c527598b
Merge pull request #1049 from dcooper16/dont_offer_compression
Don't offer TLS compression by default
2018-05-02 11:08:51 +02:00
Dirk Wetter 4b39039209
Merge pull request #1050 from dcooper16/remove_extra_space
Remove extra space in BEAST output
2018-05-02 11:00:01 +02:00
David Cooper 76eca42ca0
Remove extra space in BEAST output
testssl.sh was inserting two spaces between the CBC ciphers detected by OpenSSL and those detected to tls_sockets(). This PR fixes the problem.

This issue was previously fixed by 87fe0c15da, but that fix was accidentally removed by the next commit: f3dc53f554.
2018-05-01 09:40:20 -04:00
David Cooper 85e7fd3a08
Don't offer TLS compression by default
There is at least one server that will fail under some circumstances if the ClientHello offers a compression method other than null.

In OpenSSL 1.1.0 and 1.1.1, s_client will not offer any other compression methods unless the "-comp" option is provided. However, in earlier versions of OpenSSL, s_client will by default offer the DEFLATE compression method, however, this can be disabled using the "-no_comp" option.

This PR addresses the flaw in this server by having s_client_options() add a "-no_comp" option to the command line if "-no_comp" is supported and the test doesn't require offering compression.

Since run_crime() requires compression to be offered, run_crime() was changed to always add "-comp" to the command line, and then s_client_options() was changed to remove "-comp" from the command line, if that option isn't supported.
2018-04-30 16:47:12 -04:00
Dirk 470f8b62e6 jsonID HTTP status code as variable 2018-04-28 22:38:53 +02:00
Dirk 168eec447a Merge branch 'crl_ocsp' into 2.9dev 2018-04-28 22:36:27 +02:00
Dirk 0851599a4e Polish existing CRL revocation check code
* Extra function for ldap_get()
  * Hint when curl is not installed and LDAP URI is encountered
  * Rename jsonID cert_cRLDistributionPoints to cert_crlDistributionPoints
  * Fix trailing _ in jsonID

Open/to be clarified:

   * Proxy for curl / proxy needs to come from testssl.sh
   * Proxy support for HTTP bash socket GET
   * cert_CRLrevoked comes before cert_cRLDistributionPoints
   * Unit tests

Still open: OCSP
2018-04-28 22:25:43 +02:00