Merge branch 'master' into openss2rfc_rfc2openssl

Conflicts:
	testssl.sh

bin/Readme.md

@@ -1,13 +1,26 @@
 
-
-**Note: Further new improved binaries (1.0.2i, compiled from a snapshot from Peter Mosman's fork: https://github.com/drwetter/openssl) will be uploaded soon.**
-
-**Probably we discontinue the rarely used ones at github as it is not very approrpriate. Main site for binaries see https://testssl.sh/**
-
-
 Binaries
 ========
 
+All the precompiled binaries provided here have extended support for
+everything which is normally not in OpenSSL or LibreSSL -- 40+56 Bit,
+export/ANON ciphers, weak DH ciphers, weak EC curves, SSLv2 etc. -- all the dirty
+features needed for testing. OTOH they also come with extended support
+for new / advanced cipher suites and/or features which are not in the 
+official branch like (old version of the) CHACHA20+POLY1305 and CAMELLIA 256 bit ciphers.
+They also have IPv6 support, see below.
+
+The (stripped) binaries this directory are all compiled from my openssl
+snapshot (https://github.com/drwetter/openssl) from Peter Mosman's openssl
+fork (https://github.com/PeterMosmans/openssl). Thx a bunch, Peter!
+
+Compiled Linux and FreeBSD binaries so far come from Dirk, other
+contributors see ../CREDITS.md .
+
+**I discontinued to upload the not commonly used binaries at github ** (ARM7l, Darwin.i386 and all except one kerberos compiles) **as it is not very appropriate to use github especially for those. The main site for all 
+binaries is https://testssl.sh/openssl-1.0.2i-chacha.pm.ipv6.contributed/, also see the tarball @
+https://testssl.sh/openssl-1.0.2i-chacha.pm.ipv6.Linux+FreeBSD.tar.gz**
+
 The binaries here have the naming scheme ``openssl.$(uname).$(uname -m)``
 and will be picked up from testssl.sh if you run testssl.sh directly
 off the git directory. Otherwise you need ``testssl.sh`` to point to it 
@@ -18,19 +31,6 @@ The Linux binaries with the trailing ``-krb5`` come with Kerberos 5 support,
 they won't be picked up automatically as you need to make sure first they
 run (see libraries below).
 
-All the precompiled binaries provided here have extended support for
-everything which is normally not in OpenSSL or LibreSSL -- 40+56 Bit,
-export/ANON ciphers, weak DH ciphers, weak EC curves, SSLv2 etc. -- all the dirty
-features needed for testing. OTOH they also come with extended support
-for new / advanced cipher suites and/or features which are not in the 
-official branch like (old version of the) CHACHA20+POLY1305 and CAMELIA 256 bit ciphers.
-
-The binaries in this directory are all compiled from an OpenSSL 1.0.2 fork
-from Peter Mosmans (https://github.com/PeterMosmans/openssl). Thx a bunch, 
-Peter!
-
-Compiled Linux and FreeBSD binaries so far come from Dirk, other contributors see ../CREDITS.md .
-
 
 Compiling and Usage Instructions
 ================================
@@ -62,6 +62,12 @@ If you want to compile OpenSSL yourself, here are the instructions:
      git clone https://github.com/PeterMosmans/openssl
      cd openssl
 
+or use my repo:
+
+    git clone https://github.com/drwetter/openssl
+    cd openssl
+
+
 2.) configure the damned thing. Options I used (see https://github.com/drwetter/testssl.sh/blob/master/utils/make-openssl.sh)
 
 **for 64Bit including Kerberos ciphers:**
@@ -92,7 +98,10 @@ If you want to compile OpenSSL yourself, here are the instructions:
     enable-seed enable-camellia enable-idea enable-rfc3779 no-ec_nistp_64_gcc_128 \
     -static experimental-jpake -DOPENSSL_USE_BUILD_DATE 
 
-IPv6 support would need additionally the patch from ``fedora-dirk-ipv6.diff``. This doesn't give you the option of an IPv6 enabled proxy yet. It is good practice to compile those binaries with ``-DOPENSSL_USE_IPV6`` as later on you can tell them apart by``openssl version -a``.
+IPv6 support would need additionally the patch from ``fedora-dirk-ipv6.diff`` (included already
+in my branch).  This doesn't give you the option of an IPv6 enabled proxy yet.
+It is good practice to compile those binaries with ``-DOPENSSL_USE_IPV6`` as
+later on you can tell them apart by``openssl version -a``.
 
 Four GOST [1][2] ciphers come via engine support automagically with this setup. Two additional GOST 
 ciphers can be compiled in (``GOST-GOST94``, ``GOST-MD5``) with ``-DTEMP_GOST_TLS`` but as of now they make 
@@ -110,7 +119,7 @@ If you don't have / don't want Kerberos libraries and devel rpms/debs, just omit
 
 6.) ``./apps/openssl ciphers -V 'ALL:COMPLEMENTOFALL' | wc -l`` lists for me
 * 193(+4 GOST) ciphers including kerberos 
-* 177(+4 GOST) ciphers without kerberos
+* 179(+4 GOST) ciphers without kerberos
 
 as opposed to ~110 from Ubuntu or Opensuse. 
 

testssl.sh

@@ -203,7 +203,6 @@ TLS_EXTENSIONS=""
 GOST_STATUS_PROBLEM=false
 DETECTED_TLS_VERSION=""
 PATTERN2SHOW=""
-SOCKREPLY=""
 SOCK_REPLY_FILE=""
 HEXC=""
 NW_STR=""
@@ -260,7 +259,6 @@ TLS_LOW_BYTE=""
 HEX_CIPHER=""
 
                                              # The various hexdump commands we need to replace xxd (BSD compatibility)
-HEXDUMPVIEW=(hexdump -C)                     # This is used in verbose mode to see what's going on
 HEXDUMP=(hexdump -ve '16/1 "%02x " " \n"')   # This is used to analyze the reply
 HEXDUMPPLAIN=(hexdump -ve '1/1 "%.2x"')      # Replaces both xxd -p and tr -cd '[:print:]'
 
@@ -1511,23 +1509,6 @@ socksend() {
      sleep $2
 }
 
-
-#FIXME: This is only for HB and CCS, others use still sockread_serverhello()
-sockread() {
-     local -i ret=0
-     local ddreply
-
-     [[ "x$2" == "x" ]] && maxsleep=$MAX_WAITSOCK || maxsleep=$2
-
-     ddreply=$(mktemp $TEMPDIR/ddreply.XXXXXX) || return 7
-     dd bs=$1 of=$ddreply count=1 <&5 2>/dev/null &
-     wait_kill $! $maxsleep
-     ret=$?
-     SOCKREPLY=$(cat $ddreply 2>/dev/null)
-     rm $ddreply
-     return $ret
-}
-
 openssl2rfc() {
      local rfcname=""
      local -i i
@@ -6089,9 +6070,10 @@ tls_sockets() {
 run_heartbleed(){
      local tls_proto_offered tls_hexcode
      local heartbleed_payload client_hello 
-     local -i n ret
+     local -i n ret lines_returned
      local -i hb_rounds=3
      local append=""
+     local found_500_oops=false
 
      [[ $VULN_COUNT -le $VULN_THRESHLD ]] && outln && pr_headlineln " Testing for heartbleed vulnerability " && outln
      pr_bold " Heartbleed"; out " (CVE-2014-0160)                "
@@ -6175,39 +6157,45 @@ run_heartbleed(){
           socksend "$client_hello" 1
 
           debugme outln "\nreading server hello"
-          sockread 32768
+          sockread_serverhello 32768
           if [[ $DEBUG -ge 4 ]]; then
-               echo "$SOCKREPLY" | "${HEXDUMPVIEW[@]}" | head -20
+               hexdump -C "$SOCK_REPLY_FILE" | head -20
                outln "[...]"
                outln "\nsending payload with TLS version $tls_hexcode:"
           fi
+          rm "$SOCK_REPLY_FILE"
 
           socksend "$heartbleed_payload" 1
-          sockread 16384 $HEARTBLEED_MAX_WAITSOCK
+          sockread_serverhello 16384 $HEARTBLEED_MAX_WAITSOCK
           [[ $? -eq 3 ]] && append=", timed out"
 
+          lines_returned=$(hexdump -ve '16/1 "%02x " " \n"' "$SOCK_REPLY_FILE" | wc -l | sed 's/ //g')
           if [[ $DEBUG -ge 3 ]]; then
                outln "\nheartbleed reply: "
-               echo "$SOCKREPLY" | "${HEXDUMPVIEW[@]}"
+               hexdump -C "$SOCK_REPLY_FILE" | head -20
+               [[ $lines_returned -gt 20 ]] && outln "[...]"
                outln
           fi
-          lines_returned=$(echo "$SOCKREPLY" | "${HEXDUMP[@]}" | wc -l | sed 's/ //g')
 
           if [[ $lines_returned -gt 1 ]]; then
                if [[ "$STARTTLS_PROTOCOL" == "ftp" ]] || [[ "$STARTTLS_PROTOCOL" == "ftps" ]]; then
                     # check possibility of weird vsftpd reply, see #426
-                    saved_sockreply[n]=$SOCKREPLY
+                    saved_sockreply[n]="$(hexdump -ve '1/1 "%.2x"' "$SOCK_REPLY_FILE")"
-                    debugme out "${saved_sockreply[n]}"
+                    [[ $n -eq 1 ]] && grep -q '500 OOPS' "$SOCK_REPLY_FILE" && found_500_oops=true
+                    rm "$SOCK_REPLY_FILE"
+                    #debugme out "${saved_sockreply[n]}"
                     #TMPFILE="${saved_sockreply[n]}"
                     close_socket
-                    tmpfile_handle "$FUNCNAME,$n.txt"
+                    #tmpfile_handle "$FUNCNAME,$n.txt"
                else
+                    rm "$SOCK_REPLY_FILE"
                     pr_svrty_critical "VULNERABLE (NOT ok)"
                     fileout "heartbleed" "NOT ok" "Heartbleed (CVE-2014-0160): VULNERABLE (NOT ok)$append"
                     ret=1
                     break
                fi
           else
+               rm "$SOCK_REPLY_FILE"
                pr_done_best "not vulnerable (OK)"
                fileout "heartbleed" "OK" "Heartbleed (CVE-2014-0160): not vulnerable (OK)$append"
                ret=0
@@ -6220,7 +6208,7 @@ run_heartbleed(){
           # This is the robust approach. According to a few tests it could also suffice # to check for "500 OOPS" only.
           # Checking for the same socket reply DOES NOT suffice -- server can be idle and return the same memory
           if [[ "${saved_sockreply[1]}" == "${saved_sockreply[2]}" ]] && [[ "${saved_sockreply[2]}" == "${saved_sockreply[3]}" ]] \
-               && echo "${saved_sockreply[1]}" | grep -q '500 OOPS'; then
+               && "$found_500_oops"; then
                pr_done_best "not vulnerable (OK)$append"
                [[ $DEBUG -ge 1 ]] && out ", successful weeded out vsftpd false positive"
                fileout "heartbleed" "OK" "Heartbleed (CVE-2014-0160): not vulnerable (OK)$append"
@@ -6234,7 +6222,7 @@ run_heartbleed(){
           # for the repeated tries we did that already
           #TMPFILE="$SOCKREPLY"
           close_socket 2>/dev/null
-          tmpfile_handle $FUNCNAME.txt
+          #tmpfile_handle $FUNCNAME.txt
      fi
      outln "$append"
 
@@ -6249,6 +6237,9 @@ ok_ids(){
 
 #FIXME: At a certain point heartbleed and ccs needs to be changed and make use of code2network using a file, then tls_sockets
 run_ccs_injection(){
+     local tls_proto_offered tls_hexcode ccs_message client_hello byte6 sockreply
+     local -i retval ret lines
+
      # see https://www.openssl.org/news/secadv_20140605.txt
      # mainly adapted from Ramon de C Valle's C code from https://gist.github.com/rcvalle/71f4b027d61a78c42607
      [[ $VULN_COUNT -le $VULN_THRESHLD ]] && outln && pr_headlineln " Testing for CCS injection vulnerability " && outln
@@ -6308,41 +6299,45 @@ run_ccs_injection(){
      socksend "$client_hello" 1
 
      debugme outln "\nreading server hello"
-     sockread 32768
+     sockread_serverhello 32768
      if [[ $DEBUG -ge 4 ]]; then
-          echo "$SOCKREPLY" | "${HEXDUMPVIEW[@]}" | head -20
+          hexdump -C "$SOCK_REPLY_FILE" | head -20
           outln "[...]"
           outln "\npayload #1 with TLS version $tls_hexcode:"
      fi
+     rm "$SOCK_REPLY_FILE"
 
 # ... and then send the a change cipher spec message
      socksend "$ccs_message" 1 || ok_ids
-     sockread 2048 $CCS_MAX_WAITSOCK
+     sockread_serverhello 2048 $CCS_MAX_WAITSOCK
      if [[ $DEBUG -ge 3 ]]; then
           outln "\n1st reply: "
-          out "$SOCKREPLY" | "${HEXDUMPVIEW[@]}" | head -20
+          hexdump -C "$SOCK_REPLY_FILE" | head -20
 # ok:      15 | 0301    |  02 | 02 | 0a
 #       ALERT | TLS 1.0 | Length=2 | Unexpected Message (0a)
 #    or just timed out
           outln
           outln "payload #2 with TLS version $tls_hexcode:"
      fi
+     rm "$SOCK_REPLY_FILE"
 
      socksend "$ccs_message" 2 || ok_ids
-     sockread 2048 $CCS_MAX_WAITSOCK
+     sockread_serverhello 2048 $CCS_MAX_WAITSOCK
      retval=$?
 
      if [[ $DEBUG -ge 3 ]]; then
           outln "\n2nd reply: "
-          printf -- "$SOCKREPLY" | "${HEXDUMPVIEW[@]}"
+          printf -- "$(hexdump -C "$SOCK_REPLY_FILE")"
 # not ok:  15 | 0301    | 02 | 02  | 15
 #       ALERT | TLS 1.0 | Length=2 | Decryption failed (21)
 # ok:  0a or nothing: ==> RST
           outln
      fi
+     sockreply=$(cat "$SOCK_REPLY_FILE" 2>/dev/null)
+     rm "$SOCK_REPLY_FILE"
 
-     byte6=$(echo "$SOCKREPLY" | "${HEXDUMPPLAIN[@]}" | sed 's/^..........//')
+     byte6=$(echo "$sockreply" | "${HEXDUMPPLAIN[@]}" | sed 's/^..........//')
-     lines=$(echo "$SOCKREPLY" | "${HEXDUMP[@]}" | count_lines )
+     lines=$(echo "$sockreply" | "${HEXDUMP[@]}" | count_lines )
      debugme echo "lines: $lines, byte6: $byte6"
 
      if [[ "$byte6" == "0a" ]] || [[ "$lines" -gt 1 ]]; then