Improve handling of short-lived certificates (#3097) [3.2 backport]

Backport of the short-lived certificate handling from 3.3dev (commit
26a75cf7) to the 3.2 branch, as the issue was reported against 3.2.4.

Certificates with a short validity period (e.g. Let's Encrypt's 6-day
"shortlived" profile, now GA) always tripped the days2warn expiry
thresholds and were shown in red on the "Certificate Validity (UTC)"
line, even seconds after issuance. That red is misleading for a cert
that is intentionally short-lived.

Detect short-lived certificates by their validity period (notAfter -
notBefore) using a new DAYS_VALID_SHORTLIVED threshold (default 10 days,
per the CA/Browser Forum BR 1.6.1 "Short-lived Subscriber Certificate"
definition, which also covers the LE 6-day profile). For such certs:

  - show them as good (not red) with a "short-lived cert (N days)" remark, and
  - warn (HIGH) only when less than 24h of validity is left, and only for
    certificates whose total lifetime exceeds 24h.

Regular certificates are unaffected. The threshold is overridable via the
DAYS_VALID_SHORTLIVED env var (documented in doc/testssl.1.md next to
DAYS2WARN1/2) and added to CHANGELOG.md. The 3.2 man/HTML docs are
Ronn-NG generated, so the entry was added to match that format.

CHANGELOG.md

@@ -7,6 +7,7 @@
 
 ### Features implemented / improvements in 3.2
 
+* Detect short-lived certificates (validity period <= `DAYS_VALID_SHORTLIVED`, default 10 days) and no longer flag them red merely for their short lifespan; warn only when less than 24h is left (#3097)
 * Rating (SSL Labs), as of 3.2.2 version 2009r
 * Extend Server (cipher) preference: always now in wide mode instead of running all ciphers in the end (per default)
 * Remove "negotiated cipher / protocol"

doc/testssl.1

@@ -411,6 +411,8 @@ DAYS2WARN1 is the first threshold when you'll be warning of a certificate expira
 .IP "\[ci]" 4
 DAYS2WARN2 is the second threshold when you'll be warning of a certificate expiration of a host, preset to 30 (days)\. For Let's Encrypt this value will be divided internally by 2\.
 .IP "\[ci]" 4
+DAYS_VALID_SHORTLIVED is preset to 10 (days)\. Certificates with a total validity period (notAfter \- notBefore) of this many days or fewer are treated as intentionally short\-lived (see "Short\-lived Subscriber Certificate" in the CA/Browser Forum Baseline Requirements, e\.g\. Let's Encrypt's 6\-day profile)\. They are not being warned of for their short lifespan\. A warning is issued only when less than 24 hours of validity are left, and only for certificates whose total validity period is more than 24 hours\.
+.IP "\[ci]" 4
 TESTSSL_INSTALL_DIR is the derived installation directory of testssl\.sh\. Relatively to that the \fBbin\fR and mandatory \fBetc\fR directory will be looked for\.
 .IP "\[ci]" 4
 CA_BUNDLES_PATH: If you have an own set of CA bundles or you want to point testssl\.sh to a specific location of a CA bundle, you can use this variable to set the directory which testssl\.sh will use\. Please note that it overrides completely the builtin path of testssl\.sh which means that you will only test against the bundles you point to\. Also you might want to use \fB~/utils/create_ca_hashes\.sh\fR to create the hashes for HPKP\.

doc/testssl.1.html

@@ -497,6 +497,7 @@ Rating automatically gets disabled, to not give a wrong or misleading grade, whe
   <li>HPKP_MIN is preset to 30 (days). If you want warnings sooner or later for HTTP Public Key Pinning you can change this</li>
   <li>DAYS2WARN1 is the first threshold when you'll be warning of a certificate expiration of a host, preset to 60 (days). For Let's Encrypt this value will be divided internally by 2.</li>
   <li>DAYS2WARN2 is the second threshold when you'll be warning of a certificate expiration of a host, preset to 30 (days). For Let's Encrypt this value will be divided internally by 2.</li>
+  <li>DAYS_VALID_SHORTLIVED is preset to 10 (days). Certificates with a total validity period (notAfter - notBefore) of this many days or fewer are treated as intentionally short-lived (see "Short-lived Subscriber Certificate" in the CA/Browser Forum Baseline Requirements, e.g. Let's Encrypt's 6-day profile). They are not being warned of for their short lifespan. A warning is issued only when less than 24 hours of validity are left, and only for certificates whose total validity period is more than 24 hours.</li>
   <li>TESTSSL_INSTALL_DIR is the derived installation directory of testssl.sh. Relatively to that the <code>bin</code> and mandatory <code>etc</code> directory will be looked for.</li>
   <li>CA_BUNDLES_PATH: If you have an own set of CA bundles or you want to point testssl.sh to a specific location of a CA bundle, you can use this variable to set the directory which testssl.sh will use. Please note that it overrides completely the builtin path of testssl.sh which means that you will only test against the bundles you point to. Also you might want to use <code>~/utils/create_ca_hashes.sh</code> to create the hashes for HPKP.</li>
   <li>MAX_SOCKET_FAIL: A number which tells testssl.sh how often a TCP socket connection may fail before the program gives up and terminates. The default is 2. You can increase it to a higher value if you frequently see a message like <em>Fatal error: repeated repeated TCP connect problems, giving up</em>.</li>

doc/testssl.1.md

@@ -399,6 +399,7 @@ Except the environment variables mentioned above which can replace command line
 * HPKP_MIN is preset to 30 (days). If you want warnings sooner or later for HTTP Public Key Pinning you can change this
 * DAYS2WARN1 is the first threshold when you'll be warning of a certificate expiration of a host, preset to 60 (days). For Let's Encrypt this value will be divided internally by 2.
 * DAYS2WARN2 is the second threshold when you'll be warning of a certificate expiration of a host, preset to 30 (days). For Let's Encrypt this value will be divided internally by 2.
+* DAYS_VALID_SHORTLIVED is preset to 10 (days). Certificates with a total validity period (notAfter - notBefore) of this many days or fewer are treated as intentionally short-lived (see "Short-lived Subscriber Certificate" in the CA/Browser Forum Baseline Requirements, e.g. Let's Encrypt's 6-day profile). They are not being warned of for their short lifespan. A warning is issued only when less than 24 hours of validity are left, and only for certificates whose total validity period is more than 24 hours.
 * TESTSSL_INSTALL_DIR is the derived installation directory of testssl.sh. Relatively to that the `bin` and mandatory `etc` directory will be looked for.
 * CA_BUNDLES_PATH: If you have an own set of CA bundles or you want to point testssl.sh to a specific location of a CA bundle, you can use this variable to set the directory which testssl.sh will use. Please note that it overrides completely the builtin path of testssl.sh which means that you will only test against the bundles you point to. Also you might want to use `~/utils/create_ca_hashes.sh` to create the hashes for HPKP.
 * MAX_SOCKET_FAIL: A number which tells testssl.sh how often a TCP socket connection may fail before the program gives up and terminates. The default is 2. You can increase it to a higher value if you frequently see a message like *Fatal error: repeated TCP connect problems, giving up*.

testssl.sh

@@ -218,6 +218,7 @@ HPKP_MIN=${HPKP_MIN:-30}                # >=30 days should be ok for HPKP_MIN, p
      HPKP_MIN=$((HPKP_MIN * 86400))     # correct to seconds
 DAYS2WARN1=${DAYS2WARN1:-60}            # days to warn before cert expires, threshold 1
 DAYS2WARN2=${DAYS2WARN2:-30}            # days to warn before cert expires, threshold 2
+DAYS_VALID_SHORTLIVED=${DAYS_VALID_SHORTLIVED:-10}  # validity period (notAfter-notBefore) <= this many days => "short-lived", see CA/Browser Forum BR 1.6.1 and #3097
 VULN_THRESHLD=${VULN_THRESHLD:-1}       # if vulnerabilities to check >$VULN_THRESHLD we DON'T show a separate header line in the output each vuln. check
 UNBRACKTD_IPV6=${UNBRACKTD_IPV6:-false} # some versions of OpenSSL (like Gentoo) don't support [bracketed] IPv6 addresses
 NO_ENGINE=${NO_ENGINE:-false}           # if there are problems finding the (external) openssl engine set this to true
@@ -9161,6 +9162,7 @@ certificate_info() {
      local indent=""
      local days2warn2=$DAYS2WARN2
      local days2warn1=$DAYS2WARN1
+     local cert_is_shortlived=false
      local provides_stapling=false
      local caa_node="" all_caa="" caa_property_name="" caa_property_value=""
      local response=""
@@ -9822,12 +9824,32 @@ certificate_info() {
           days2warn1=$((days2warn1 / 2))
      fi
 
+     # A short-lived certificate has a validity period (notAfter - notBefore) at or below
+     # DAYS_VALID_SHORTLIVED. These (e.g. Let's Encrypt's 6-day "shortlived" profile) are
+     # intentionally short, so the normal days2warn thresholds would always flag them red.
+     # For those we only warn when the cert is nearly expired (< 24h left), see #3097.
+     [[ $diffseconds -gt 0 ]] && [[ $diffseconds -le $((secsaday*DAYS_VALID_SHORTLIVED)) ]] && cert_is_shortlived=true
+
      debugme echo -n "(diffseconds: $diffseconds)"
      if ! [[ "$($OPENSSL x509 -checkend 1 2>>$ERRFILE <<< "$hostcert")" =~ \ not\  ]]; then
           pr_svrty_critical "expired"
           expfinding="expired"
           expok="CRITICAL"
           set_grade_cap "T" "Certificate expired"
+     elif "$cert_is_shortlived"; then
+          # An intentionally short-lived cert (e.g. Let's Encrypt's 6-day profile) shouldn't be
+          # flagged red just for its short lifespan. Warn only when it is about to expire (< 24h
+          # left) and only if its total lifetime is more than 24h -- otherwise the 24h rule would
+          # flag such a cert red for its whole life. The "short-lived cert" remark signals intent.
+          if [[ $diffseconds -gt $secsaday ]] && \
+             ! [[ "$($OPENSSL x509 -checkend $secsaday 2>>$ERRFILE <<< "$hostcert")" =~ \ not\  ]]; then
+               pr_svrty_high "short-lived cert, expires < 24h"
+               expfinding+="short-lived cert, expires < 24h"
+               expok="HIGH"
+          else
+               pr_svrty_good "short-lived cert ($days2expire days)"
+               expfinding+="short-lived cert ($days2expire days)"
+          fi
      else
           # low threshold first
           if [[ "$($OPENSSL x509 -checkend $((secsaday*days2warn2)) 2>>$ERRFILE <<< "$hostcert")" =~ \ not\  ]]; then
@@ -21492,6 +21514,7 @@ HPKP_MIN: $HPKP_MIN
 CLIENT_MIN_FS: $CLIENT_MIN_FS
 DAYS2WARN1: $DAYS2WARN1
 DAYS2WARN2: $DAYS2WARN2
+DAYS_VALID_SHORTLIVED: $DAYS_VALID_SHORTLIVED
 
 HEADER_MAXSLEEP: $HEADER_MAXSLEEP
 MAX_WAITSOCK: $MAX_WAITSOCK