When run_rc4() is run with the "--show-each" option, but without the "--wide" option, a list of all RC4 ciphers is printed, without any distinction between those that are supported by the server and those that are not. This is the same issue I noted in #332 for run_pfs().
In run_pfs(), the displayed output was corrected, but all ciphers were still being added to $pfs_ciphers, so the list of supported PFS ciphers sent to fileout() was incorrect.
This PR fixes both issues.
When certificate_info() is trying to determine what type of public key the server has so that it can determine whether the key size is acceptable, it sometimes looks at $cert_sig_algo rather than $cert_key_algo. This PR fixes that and also adds support for DSA public keys.
The dec2hex() was actually converting from hex to decimal. Since it was only being used in one place, and wasn't really needed there, I just deleted it.
Revised parse_tls_serverhello() to more carefully check the response for errors, and to provide for more flexibility (e.g., if handshake messages are split across multiple fragments).
The new test in PR #346 sends a TLSv1.4 ClientHello, so socksend_tls_clienthello() needs to include the signature algorithms extension if $tls_low_byte >= 3 rather than only if it is equal to 3.
One server I am testing responds to an SSLv3 ClientHello with TLSv1.2. If tls_sockets is being used, then testssl.sh responds with "#FIXME: downgraded. still missing a test case here." This PR fixes that, and in general checks the responses in run_protocols() more closely.
If tls_sockets is being used and the connection fails even though the server supports an earlier version of SSL/TLS, then it flags an error. If tls_sockets returns 2, then it verifies that $DETECTED_TLS_VERSION is equal to the highest version number supported by the server (that is also less than the version number in the ClientHello).
In addition, in order to test servers' support for version negotiation, it adds a new test that sends a TLSv1.4 ClientHello and verifies that the server responds with the highest version number that it supports. (This test only runs if both $using_sockets and $EXPERIMENTAL are true and server actually supports some version of SSL/TLS other than SSLv2.)
Changed to only include the signature algorithms extension for TLSv1.2, since RFC 5246 says:
Note: this extension is not meaningful for TLS versions prior to 1.2.
Clients MUST NOT offer it if they are offering prior versions.
However, even if clients do offer it, the rules specified in [TLSEXT]
require servers to ignore extensions they do not understand.
Inclusion of the extension for TLS 1.1 didn't seem to cause any harm, but it seems better to follow the RFC and not include it for TLSv1.0 or TLSv1.1.
RFC 7685 notes that there is at least one TLS implementation that hangs if the client sends a ClientHello with a TLSCiphertext.length between 256 and 511 bytes, and so the padding extension was defined in order to get around this bug. (OpenSSL s_client includes this extension when the -bugs option is used.) So, I changed socksend_tls_clienthello() to include the padding extension if the CLientHello would have a length between 256 and 511 bytes, making the padding extension just large enough to make the ClientHello 512 bytes.
I also fixed a typo (a missing "0x") in the check for whether any ECC ciphers are included in the Client Hello.
In doing some work on cipher_pref_check() I noticed that it was failing on SSLv2 since the call to "$OPENSSL s_client" includes SNI. I've also noticed in my testing that "$OPENSSL s_client" will not connect to an SSLv2-only server unless the "-ssl2" flag is included. So, I carefully checked each call to "$OPENSSL s_client" in the program (other than in run_allciphers and run_cipher_per_proto, since those functions are already addresses in PR #341) to see whether they would inappropriate fail with an SSLv2-only (or SSLv3-only) server.
As a general rule, if the call doesn't currently include the protocol, then I added "-ssl2" if $OPTIMAL_PROTO is "-ssl2", indicating that the server only supports SSLv2, and I removed any $SNI if a protocol is specified if a protocol is specified and it is either SSLv2 or SSLv3.
I tested it on an SSLv2-only server, and the results are much better. I also tested it on a collection of other servers, none of which support SSLv2, and the results are the same as with the current code.
The only thing I haven't been able to test is how the revised code works when the "--starttls" option is used. I don't believe the changes I made would cause anything to break in that case, but I also don't think code will work any better in that case, if the server only supports SSLv2. Of course, since no server should support SSLv2 (let alone only SSLv2), it shouldn't really be an issue.
One thing that I did not change, but that I do not understand; why does determine_optimal_proto() try the protocols in the order "-tls1_2 -tls1 -ssl3 -tls1_1 -ssl2" rather than "-tls1_2 -tls1_1 -tls1 -ssl3 -ssl2"? Doesn't the current ordering imply that TLS v1.0 and SSLv3 are better than TLS v1.1?
Versions of OpenSSL prior to 1.1.0 ignore the options "-tls1_1" and "-tls1_2". So, a call of the form "$OPENSSL ciphers -tls1_2 -V 'ALL:COMPLEMENTOFALL:@STRENGTH' would list all supported ciphers (including SSLv2 ciphers), not just ciphers appropriate for TLS1.2.
This changes the code to use "-tls1" instead of "-tls1_1" or "-tls1_2" if a version of OpenSSL other than 1.1.0 is being used.
I changed the code to use the global $HAS_SSL2 rather than $sslv2_locally_supported.
I don't think there's a need to use $HAS_SSL3 in run_allciphers(), since the call to "$OPENSSL s_client" for non-SSLv2 ciphers does not specify a protocol. It's also not needed in run_cipher_per_proto(), since there is already a call to locally_supported() before anything further is done with a protocol.
This PR adds the signature algorithms, heartbeat, session ticket, and next protocol extensions to the client hello message created by socksend_tls_clienthello() for TLS 1.0 and above. It also adds the supported elliptic curves and ec points format extensions if the client hello message includes any ECC cipher suites.
I tested this version against several servers with $EXPERIMENTAL set to true and get the same results as with the current code with $EXPERIMENTAL set to false.
This PR addresses two problems related to SSLv2 in run_allciphers() and run_cipher_per_proto().
In run_cipher_per_proto(), the call to "$OPENSSL s_client" is changed to that $SNI is not included if $proto is -sslv2 or -sslv3. As noted in a comment within run_prototest_openssl(), "newer openssl throw an error if SNI is supplied with SSLv2" and "SSLv3 doesn't have SNI (openssl doesn't complain though -- yet)."
run_allciphers() will sometimes incorrectly report that a server supports an SSLv2 cipher, even if the server does not support SSLv2 at all. The problem occurs if there is a supported SSLv3 cipher suite that has the same OpenSSL name as an SSLv2 cipher suite (e.g., RC4-MD5). Since the call to "$OPENSSL s_client" only uses the OpenSSL name, but the results report both the name, hexcode, and RFC cipher suite name, both the SSLv2 and SSLv3 cipher suites are reported as being supported (e.g., 0x04=RC4-MD5=TLS_RSA_WITH_RC4_128_MD5 and x010080=RC4-MD5=SSL_CK_RC4_128_WITH_MD5). This PR fixes the problem by testing SSLv2 cipher suites separately from non-SSLv2 cipher suites.
The last line of neat_list currently uses $HEXC as the parameter to show_rfc_style(), but it should use $hexcode. At the moment using $HEXC instead of $hexcode makes no difference, since hexcode="$1" and in all calls to neat_list() the first parameter is $HEXC. However, this bug could create problems in the future since neat_list() will misbehave if the value of the first parameter (hexcode) isn't the same as $HEXC.
This PR makes basically the same changes to run_cipher_per_proto() as I previously made to run_allciphers(). The main difference is that in this function, round 0 consists of a single call to "$OPENSSL s_client" with "-cipher" including all of the locally supported ciphers. The reason for the difference is that in run_allciphers() its saves time to assume the server supports at least one cipher suite. In the case of run_cipher_per_proto(), however, it is likely that the server will not support some protocols at all, so its usually faster to start with a single call to "$OPENSSL s_client" that tests whether the server supports the protocol at all.
The run_allciphers() function currently works by calling "$OPENSSL s_client" once for each cipher suite supported by $OPENSSL. In the case of "OpenSSL 1.0.2-chacha (1.0.2e-dev)" that means 195 calls to "$OPENSSL s_client" even though servers tend to only support a small fraction of these cipher suites.
This PR produces the same output as the current run_allciphers() with fewer calls to "$OPENSSL s_client", which results in the function running faster (usually much faster). The basic idea behind the revised function is to test cipher suites in blocks. If $OPENSSL supports 195 cipher suites, then it group these cipher suites into 4 blocks of 64 (with the final block being smaller). It makes one call to "$OPENSSL s_client" with cipher suites 1-64, and if it fails, then it knows that none of these 64 cipher suites are supported by the server and it doesn't need to perform any more tests on these 64 cipher suites. If it succeeds, then it breaks the 64 cipher suites into 4 blocks of 16 and calls "$OPENSSL s_client" with each of those blocks. The blocks of 16 that are successful are broken into blocks of 4, and for each of the successful blocks of 4 the individual cipher suites are tested.
For testssl.sh and www.google.com the number of calls to "$OPENSSL s_client" is reduced from 195 to 88. For github.com the number of calls is reduced to 56!
I haven't made any changes to run_cipher_per_proto yet, but if this PR is accepted I can make the same changes in that function.
Thanks,
David
* SHOW_EACH_C has now the correct logic
* pr_litemagenta ==> pr_warning
* fileout WARN according to pr_warning then changed appropiately
* some global vars in "" to avoid unneccessary shell expansion
* HAS_SSL2/HAS_SSL3 now works more reliably
* warning added in cipher order if ssl2/ssl3 is not supported by openssl
This corrects the indentation within determine_trust() when there are multiple certificates and the output for "Chain of trust (experim.)" takes up more than one lines.
In addition, it fixes the ID field of the JSON output for entries related to the certificate. At the moment, each ID string begins with a blank space. This changes it to remove the space if there is one certificate and to add "Server Certificate #X" at the beginning of each ID if there is more than one certificate.
Perhaps there's a better way than just using, for example, "Server Certificate #1 key_size" as a way to distinguish multiple "key_size" entries in the JSON file. This is just one idea, and it can certainly be changed if those who intend to use the JSON output prefer something else.
- minor output fixes for BEAST
- >4096 bit RSA keys labled in litemangenta now as it could have compatibility probs
- -V 0x.. or -V 0X.. gives at least a warning
The number of .pem files in $INSTALL_DIR/etc is currently hard-coded into determine_trust. This modifies the code so that the number of files can be changed without having to change the code.
This should fix issue #278. I'm not sure whether openssl verify will ever print out more than one error, so to be safe, I wrote the code to handle the possibility that it might; if there is more than one error, it just takes the first and ignores the rest.
* no color code in files
* rc4 ciphers were missing
* NODE was missing
* calling of NODEIP/PORT was not neccessary
* default naming of files similar to $LOGFILE
- careful regression tests for this, point open: speed
- test for more TLS extensions
- heartbleed() does now before a check whether heartbeat is available to save time
- breach simplyfied (and doesn't have to be killed in seldom cases)
- tmpfiles are only being erased after exit not after each function
- user agent is testssl -- unless --sneaky is chosen
- global host vars are now being resetted to prevent side effects
- tls version in record layer is now always 1
- used ERRFILE wherever possible
- smaller code cleanups
- revamped BEAST a bit: availablity of higher protocols lead now to yellow color, see #208
- Fixed error in BEAST (no higher protos led to no message)
- made BEAST it faster: one check for protocol ssl3+tls1 upfront, see #208
CBC cipher selection is not so easy using the openssl tool alone. Selecting the cipher based on the string CBC occuring in it would be right if it’s
about the RFC name of the cipher but not so with the openssl naming. Since CBC ciphers are not going to be continued anyway, I think it’s safe to take
a static list. However, it’s easy to extract it from the cipher list in openssl-rfc.mapping.html, but we certainly don’t want to require that file to
be shipped all the time.
- "--file" works now fine with equal sign
- fixed load balancer issue where header request stalled and testssl.sh consequently too
- http_date needed to be changed too because of that
- needed to estimate then the http_date when request was killed (HAD_SLEPT)
will Mr. Spock like this??
- fixed load balancer issue where header request for breach test stalled and thus an error was displayed
- code improvements
- labeled TLS_FALLBACK_SCSV as experimental, to be improved in next release (remarks in code)
- removed experimental from FREAK check
- separated headerfile from errorfile, TLS handshake oids were sometimes misinterpreted as IPv4 addreses in header
- bumped up rc version
- linefeeds
- freebsd 9 supports now also colors with setaf, Darwin?
- correct indentation of help
- improved parsing in command line so that where a distinct option is required it is also tested in the 1st place
- removed -q in help (deprecated as we might want to use it for other things in the future)
- fix: if $PWD/openssl was a dir it bailed out
- cleanup of fatal errors ==> provide ONE function
- reverting #27. Catch is the functions are being initiated at a fixed time instead of while calling. This conflicts with the --color option which is done late. Other solution?
- FIX: ipv6 address in rDNS was ..umm err ....missing some chars
- rough ipv6 address detection (fixes single colon in "further ip addresses")
- FIX: facebook has EC certificate but signing algo is not EC
- FIX for wrong openssl location in banner
* at least exit with -250 or worse if a problem occurs (rest still undefined, needs to be fixed, see #145/#100)
* renamed all top level tests in "run_" for better code
* SSLv2 + STARTTLS protocol check always uses sockets now
* STARTTLS protocol now returns over sockets the TLS time (if available)
* few LibreSSL output oddities fixes
* output corrections for STARTTLS
* additional path for binaries (we change the path soon but leave both in the code for now)
* FIX: if request w/o SNI didn't succeed it resulted in an ugly openssl error message
* FIX#51 (we try to initialize GOST engine before showing the banner)
* FIX for screwed up output for fixed ciphers (FREAK, LOGJAM), see also #126
* GOST support now doesn't complain if MY confif file aleady exists (minor fix)
- NEW: Varnish and Squid header detected
- NEW: option --ip=one is a shortcut and means just test the first ip
- CSP Report-Only in security headers
- New: Varnish and Squid header detected, OWA header
- all single tests in bold now
- no support for TLS 1.2 spits out "NOT ok" as it is not ok
- Medium ciphers and DES ciphers are not having aNULL and aDH ciphers anymore and have different colors --> ratings
- http-date is now in http header(), tls_time in server_defaults()
- http header reply is indented to same row as server defaults
- http status code is displayed clearly now
- BUGFIX: IPv6 address wasn't displayed
- cleanup
- application banner now in two lines if needed
- try a second time to get a http header if first one fails
- fix: case where % sign in ip address made prinf hiccup (sanitized)
- fix: $url was in some functions empty
- fixed bug where some headers were displayed twice
cipher lists now wth plural ending
added Liferay-Portal + X-OWA-Version for application banner
new http_header (still leaving old one in)
readability improvements
- option long changed to wide
- PFS now is per default not wide
- PFS comes after standard cipher lists
- debug output improved (in terms of privacy and additional info)
- little bit more robust for strange keysize and dh bits
- added ecdsa-with-SHA256 to Signature Algorithm
- FIX: no TLS1+SSL3 resulted in no output for BEAST
- FIX#92
- FIX for TLS time (difftime was too small for local clock skew)
- warning for freebsd/macosx w/o ports need now a "yes"
- TLS 1.0 not offered is not bold anymore
- output weirdness fixed for cipher order in spdy
- FIX: 2 occurrances of OPENSSL calls had a hostname instead of an IP address
- FIX: starttls protocol correctly displayed
- NEW added duplicate detection for header flags
- NEW: added four GOST cipher to standard socket handshake
- recommends if openssl 1.0.2 is used and results were strange and IIS6 --> run wqith openssl 1.0.1
- declared some global vars as readonly
to =< 1.0.1) finding the right protocol before
- hints for IIS6+openssl 1.0.2 non-conformity #99
- version bumped up to 2.4rc2
- better formatting for BSD in cipher order
- FIX: 2x bug for cipher order + sslv2
- preambel revisited
- http date
- cipher list in preferences
- GET_REQ11 now closes the connection
- openssl_age comes afeter the banner so that help doesn't need to go thru this
- uname -s ==> SYSTEM
- X-Powered-By is easy to remove (PHP, ASP.NET), thus labelled as yellow
- same X-AspNet-Version (version # itself is brown)
- better addressed address resolution failures ;-)
- bumped up version to 2.4rc1
- feature: integrated TLS+HTTP time into server defaults
- NEW: option: -U/vulnerable
- moved explanation for BREACH into result
- FREAK and CCS are not labled experimental anymore
- unifying of get request headers
- readability of help
- introducng a variable name LONG which for certain funcs shows broad output with hexc, cipher, KX, etc.
- FIX: regression not showing security headers
- introducing VULN_THRESHLD
(timeout was faster then socket resply)
- FIX: CORS header not labeled as green
- NEW: Now also STARTTLS works with all cmd line options and is absolutely doing the same stuff!
(integrated starttls() into parse_hn_port() )
- option --mx needed to be changed because of starttls
- regression fix: exec for socket doesn't play nice with stderr redirect
(probably bash bug)
- added some env options to cmd line as long args (--assuming-http,--ssl_native,
--color, debug, --sneaky, --warnings)
- threw away getent as it doesn't work under Linux && not network && localhost
(replaced by grep)
- SSL-POODLE is not labeled anymore experimental
- HB+CCS are called while checking STARTTLS but given a hint that its not yet supported
- added more env vars to debug output
- cleanups
- FIX regression: capitalized/all lowercase headers weren't detected
- if socksend is blocked (IDS) output looks better and is reported as test didn't succeed
- no secure cookie or Httponly will be marked as brown
- tput color yellow is now brown
- logic of some ENV variables changed (attention!)
- included some ENV as long options (not in the help yet)
- decentralized http check for breach
- if openssl is not executable it bails out better now
- help function now exits
Note that due to the refactoring of some status messages, the output will be slightly different (more verbose) than previous versions
Moved specific status messages to http_header()
Moved specific status messages to breach()
Moved specific status messages to ccs_injection()
Moved specific status messages to heartbleed()
Moved specific status messages to renego()
Moved specific status messages to crime()
Moved specific status messages to tls_poodle()
Moved specific status messages to freak()
Moved specific status messages to beast()
Added some more documentation for functions
Fixed typos in help
Created new function main:
This is the main function of testssl.sh
Refactored major part of the original main function
Created new function startup:
Parses the startup options
Created new function intialize_globals:
Initializes all used global variables
Created new function scanning_defaults:
Sets default scanning options when only one parameter (URI) is given
TODO: Refactor more/duplicate parts of functions
Note: For the new functions, fixed spaces (4) are used instead of tabs
- FIX for SNI output as it doensn';t make sense for non HTTP servives
- lines for RC4 and PFS shortenedA
- display all MX records to test before testing
- removed LOCERR, added CCS_MAX_WAITSOCK, HEARTBLEED_MAX_WAITSOCK
* internal renaming of color functions ( --> pr_*)
* new color switches (tput)
* $COLOR is treated as integer not string
* for some issues color adjusted accordingly (red --> brown/yellow)
- so is heartbleed
- FIX: shopt is removed in rc4 as most of the bash shells segfault here (bug!)
- not tested anymore for HTTP within starttls, instead displaying here a line
Idea is to bail out per default (with WARNINGS=off) this makes batch processing possible
as often testssl.sh hangs for minutes or endless on non-SSL ports.
- NEW: CN, SAN
- NEW: OCSP URI
- NEW: CRL distr point
- NEW: Issuer
- NEW: expiration
- NEW: signature algo
- renamed cmdline --simple_preference to --server_defaults
- now we have a TEMPDIR where all files are written toA
- function or handling/removing TMPFILE
- BUGFIX: HTTP specific vuln. won't be checked if service is not http (we still
check crime and also spdy => gmail has spdy for pop and imap)
- Feature: service detection: HTTP, IMAP, POP, SMTP
- alignment in rDNS output corrected
- minor cleanup / improvements
- significant code improvement of hex-byte parser <-> socket sender
- BUGFIX: BSD now doesn't put an extra \n if rfc map file is missing
- bumped to 2.1rc3, hoping that'll be the last
- even clearer warning upon old openssl version (MacOSX!)
- oparoz hexdump patch
- heartbleed doenst do a precheck anymore --> just sockets as it may lead to false negatives
if the client was complied with it disabled (FreeBSD)
