Add client simlation data and provide howto

While we are thankful that Ivan Ristic permitted to use the client
data from SSLlabs, it became of bit outdated now (see #1158). Also
as sslhaf [1] was used, the data comes from HTTP traffic only.

This is a start to address it. It provides data from Android 9
(connecting to the play store, so that it is sure we don't capture
a ClientHello from an application having an own TLS stack.

Also it provides documentation how to grab data yourself, and
provide it back to testssl.sh.

Aim is at least for testssl.sh 3.0 to add Android 8 and OpenSSL 1.1.1 (@drwetter).

My hope others can assist with  Safari on OSX 11 and 12. Java 10 and 11,
and a recent Opera and Edge version. (Firefox and Chrome are out of
date too)

Mail clients to follow later.

[1] https://github.com/ssllabs/sslhaf
This commit is contained in:
Dirk 2019-04-18 10:06:01 +02:00
parent e768ab3f7b
commit 5f047db92f
4 changed files with 104 additions and 1 deletions

View File

@ -34,4 +34,5 @@ If you want to test against e.g. a company internal CA you want to avoid warning
* ``client-simulation.txt`` as the name indicates it's the data for the client simulation. Use
``~/utils/update_client_sim_data.pl`` for an update. Note: This list has been manually
edited to sort it and weed it out.
edited to sort it and weed it out. In addition the file named ``client-simulation.wiresharked.txt``
provides more client data and some instructions how to generate it yourself

View File

@ -193,6 +193,27 @@
requiresSha2+=(false)
current+=(true)
names+=("Android 9.0 (native)")
short+=("android_90")
ciphers+=("TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:DES-CBC3-SHA")
ciphersuites+=("")
sni+=("$SNI")
warning+=("")
handshakebytes+=("1603010246010002420303d6259dca682ab368c7e095da7189996da830514896063d4acdc83cb5d2c2568d2041a787bf8dd3d7a1ceda514a6606f1068432a13063ea320fd7e7b367af47ecae00220a0a130113021303c02bc02fc02cc030cca9cca8c013c014009c009d002f0035000a010001d77a7a00000000001e001c0000196c68332e676f6f676c6575736572636f6e74656e742e636f6d00170000ff01000100000a000a0008aaaa001d00170018000b00020100002300000010000e000c02683208687474702f312e31000500050100000000000d00140012040308040401050308050501080606010201001200000033002b0029aaaa000100001d00203e67895a11e9ce5c69df2995782adaddb7a03ef30b245000ca332d5940ecff20002d00020101002b000b0aeaea0304030303020301001b00030200026a6a0001000029010500e000da001c9941f6b101f853f370851e583bd22e03150fc67298947270c6058707fe1670efe590d777a34b9e2e2d0ec6aa8d0ddc375c2535934c75c9623d1a271f735417fdd9190dae7f4c8541c262f8fbfeee2e820f54f59f68e78503f5c093f6084037be22c20dad3d057f64dc73f2dd45948e27c707f3f2107b32040a21fa9c1273e7797aaf5a5bc8994e9eafc4bd43b2951e10f952564a910f146344ec6d0c49f75fc6a070c75f0ffdd84fe9e10f77c23f1062e90f9e1e396eddb84d8ac00bf7ac87c557622dd18c54bbc229268699c60434648b279dd86e996baee9d1c155002120235d43319c7d5bb4725a52fa782468cd2280bd622c40a36296b354759f6d4389")
protos+=("-no_ssl3 -no_ssl2")
tlsvers+=("-tls1_3 -tls1_2 -tls1_1 -tls1")
lowest_protocol+=("0x0301")
highest_protocol+=("0x0304")
service+=("HTTP,FTP,SMTP,POP,IMAP")
minDhBits+=(-1)
maxDhBits+=(-1)
minRsaBits+=(-1)
maxRsaBits+=(-1)
minEcdsaBits+=(-1)
curves+=("X25519:secp256r1:secp384r1")
requiresSha2+=(false)
current+=(true)
names+=("Chrome 27 Win 7")
short+=("chrome_27_win7")
ciphers+=("ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-CAMELLIA256-SHA:DHE-DSS-CAMELLIA256-SHA:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:ECDH-RSA-AES256-SHA:ECDH-ECDSA-AES256-SHA:CAMELLIA256-SHA:AES256-SHA:ECDHE-ECDSA-RC4-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-RC4-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-CAMELLIA128-SHA:DHE-DSS-CAMELLIA128-SHA:DHE-DSS-RC4-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:ECDH-RSA-RC4-SHA:ECDH-RSA-AES128-SHA:ECDH-ECDSA-RC4-SHA:ECDH-ECDSA-AES128-SHA:SEED-SHA:CAMELLIA128-SHA:RC4-SHA:RC4-MD5:AES128-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:EDH-DSS-DES-CBC3-SHA:ECDH-RSA-DES-CBC3-SHA:ECDH-ECDSA-DES-CBC3-SHA:DES-CBC3-SHA")

View File

@ -0,0 +1,51 @@
# This is a file which content has been manually created from Wireshark.
# The content needs to be added to client-simulation.txt which other part
# comes from the SSLlabs client API via update_client_sim_data.pl
# The whole process is done manually.
#
# Instructions how to add a client simulation:
# * Start wireshark at the client / router. Best is during capture to filter for the target you want to contribute.
# * Make sure you create a bit of encrypted traffic to a target of your choice 1) .
# * Make sure the client traffic is specific: For just "Android" do not use a browser.
# * Stop the recording.
# * If needed sort for ClientHello.
# * Look for the ClientHello which matches the source IP + destination IP you had in mind.
# * Retrieve "handshakebytes" by marking the Record Layer --> Copy --> As a hex stream.
# * Figure out "protos" and "tlsvers" by looking at the supported_versions TLS extension (43=0x002b). May work only on modern clients.
# * Adjust "lowest_protocol" and "highest_protocol" accordingly.
# * Get "curves" from at the supported groups TLS extension 10 = 0x00a. Omit GREASE.
# * Retrieve "alpn" by looking at the alpn TLS extension 16 (=0x0010).
# * Review TLS extension 13 (=0x000d) whether any SHA1 signature algorithm is listed. If not "requiresSha2" is true
# * Leave "maxDhBits"/"minDhBits" and "minRsaBits"/"maxRsaBits" at -1, unless you know for sure what the client can handle
# * For "handshakebytes" mark the Cipher Suites --> Copy --> As a hex stream, remove any leading GREASE ciphers (?a?a) and
# supply it to ~/utils/hexstream2cipher.sh
# * Figure out the services by applying a good piece of logic
# * Before submitting a PR: test it yourself! You can also watch it again via wireshark
#
#
# 1) Attention: if you want to contribute it contains the target hostname (SNI)
names+=("Android 9.0 (native)")
short+=("android_90")
ciphers+=("TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:DES-CBC3-SHA")
ciphersuites+=("")
sni+=("$SNI")
warning+=("")
handshakebytes+=("1603010246010002420303d6259dca682ab368c7e095da7189996da830514896063d4acdc83cb5d2c2568d2041a787bf8dd3d7a1ceda514a6606f1068432a13063ea320fd7e7b367af47ecae00220a0a130113021303c02bc02fc02cc030cca9cca8c013c014009c009d002f0035000a010001d77a7a00000000001e001c0000196c68332e676f6f676c6575736572636f6e74656e742e636f6d00170000ff01000100000a000a0008aaaa001d00170018000b00020100002300000010000e000c02683208687474702f312e31000500050100000000000d00140012040308040401050308050501080606010201001200000033002b0029aaaa000100001d00203e67895a11e9ce5c69df2995782adaddb7a03ef30b245000ca332d5940ecff20002d00020101002b000b0aeaea0304030303020301001b00030200026a6a0001000029010500e000da001c9941f6b101f853f370851e583bd22e03150fc67298947270c6058707fe1670efe590d777a34b9e2e2d0ec6aa8d0ddc375c2535934c75c9623d1a271f735417fdd9190dae7f4c8541c262f8fbfeee2e820f54f59f68e78503f5c093f6084037be22c20dad3d057f64dc73f2dd45948e27c707f3f2107b32040a21fa9c1273e7797aaf5a5bc8994e9eafc4bd43b2951e10f952564a910f146344ec6d0c49f75fc6a070c75f0ffdd84fe9e10f77c23f1062e90f9e1e396eddb84d8ac00bf7ac87c557622dd18c54bbc229268699c60434648b279dd86e996baee9d1c155002120235d43319c7d5bb4725a52fa782468cd2280bd622c40a36296b354759f6d4389")
protos+=("-no_ssl3 -no_ssl2")
tlsvers+=("-tls1_3 -tls1_2 -tls1_1 -tls1")
lowest_protocol+=("0x0301")
highest_protocol+=("0x0304")
alpn+=("h2,http/1.1")
service+=("HTTP,FTP,SMTP,POP,IMAP")
minDhBits+=(-1)
maxDhBits+=(-1)
minRsaBits+=(-1)
maxRsaBits+=(-1)
minEcdsaBits+=(-1)
curves+=("X25519:secp256r1:secp384r1")
requiresSha2+=(false)
current+=(true)

30
utils/hexstream2cipher.sh Executable file
View File

@ -0,0 +1,30 @@
#!/bin/bash
hs="$1"
len=${#hs}
echo "# ciphers: $((len/4))"
mapfile="etc/cipher-mapping.txt"
[ -s $mapfile ] || mapfile="../$mapfile"
[ -s $mapfile ] || exit 255
cip=""
first=true
for ((i=0; i<len ; i+=4)); do
printf "%02d" "$i"
echo -n ": ${hs:$i:4}"
grepstr="0x${hs:$i:2},0x${hs:$((i+2)):2}"
echo -n " --> $grepstr --> "
cip=$(grep -i ${grepstr} $mapfile | awk '{ print $3 }')
echo $cip
if "$first"; then
ciphers="$cip"
first=false
else
ciphers="$ciphers:$cip"
fi
done
echo
echo $ciphers