Revives and rebases #1809 by @tosticated (Jim Blankendaal) onto 3.3dev. When --phone-out is set, run_hsts now queries https://hstspreload.org/api/v2/status and reports whether the domain is on the browser HSTS preload list (preloaded/pending/rejected/unknown), cross-referenced with the served header, the same-domain check and the bulk flag.
Addresses the review comments on #1809: the API-response matching uses native bash string matching instead of forking grep, the JSON quoting is handled inside check_hsts_preloadlist_match() so callers pass plain values, and the value arrays use 'local -a'. The output decision table is kept as-is (per maintainer feedback). Adds t/53_hsts_preload.t. Original design and decision table by @tosticated.
- parsing output from old dig versions (Mac) works for almost every svc_key
- for old dig versions: double lined RR work (but output is not nice yet)
- cleaned up comments
- separate function https_rr_raw_parser()
- commented output from claude.ai for ech for later interpretation
- get_mx_record() has a warning when get_https_rrecord returned != 0
- dig needs to be called with $DIG_R
- basic parsing for alpn on Mac should be fine now
- case statement filled with moste of the functions
- port function tested + added, but not called yet
- ipv4hint function tested + added but not called yet
- ipv6hint function tested + added but not called yet. Doesn't do compression of ipv6 address yet
- stub functions dohpath+ech
This is just to assist debugging of the runners, so that
we can grab in a case needed the screen and stderr .
* there's a script t/03_debug.t.DISABLED which needs to be renamed then
* it utilises IPC::Run3
- also showing the PATH is added for both runners
- Readme amended accordingly
Adds X-Permitted-Cross-Domain-Policies (already highlighted in emphasize_stuff_in_headers() but never reported), Origin-Agent-Cluster, Document-Policy, Clear-Site-Data, Reporting-Endpoints, Report-To and NEL to run_security_headers(), all presence-only/INFO, matching how COOP/COEP/CORP were added in #2619.
When a port was appended to the domain (e.g. "--mx example.com:25"), the suffix was passed straight into the MX DNS lookup, so no MX records were found. Strip a trailing :port off the domain before the lookup and use it as the port to test. Also fix the no-MX message, which printed $1 (the run date) instead of the domain, plus a "records(s)" typo.
First implemented and tested working is decode_https_rr_alpn().
Also we use the svk params in a case statement to decipher the
hexstream better.
The hexstream ($line) has now no blanks anymore. They seem to be
arbitrary.
Variables need to be declared in get_https_rrecord() .
- quote vars (hoping it'll resolve the Mac runner issue)
- make sure CNAMEs are properly parsed
- end get_https_rrecord() earlier when there's no record but DNS binaries are "HTTPS record aware"
- while loop was redundant
- better comments
Elsewhere:
make sure get_https_rrecord is called with a trailing dot for the NODE
as there is an inexplicable difference between a real Mac
which passes the run and the one in github
-"DNS_HTTPS_rrecord","testssl.sh/81.169.235.32","443","OK","81.169.235.32","",""
+"DNS_HTTPS_rrecord","testssl.sh/81.169.235.32","443","OK","1 . alpn='h2'","",""
The first line comes from the runner
This is a fresh start for #2484 as the PR wasn't ready yet for 3.2 by the time it was released. And it continues #2866
which was kind of messed up by accident.
The info for the HTTPS RR shows up in the very beginning, i.e. in `service_detection()`. All keys are listed now in bold, values in a regular font.
`get_https_rrecord()` was introduced by copying and modifying `get_caa_rr_record()`.
There's a similar obstacle as with CAA RRs: older binaries show the resource records binary encoded. Thus a new set of global vars is introduced HAS_*_HTTPS which check whether the binaries support decoding the RR directly. As of now raw decoding doesn't work completely.
Todo:
- Add logic in QUIC
- if RR is detected and not QUIC is possible
- add time for QUIC detection when RR is retrieved
- show full HTTPS RR record, at least when having a new DNS client
- coninue with raw decoding, if possible (otherwise problematic for MacOS)
- shorten the comments in `get_https_rrecord()`
- man page
- when ASSUME_HTTP is set and no services was detected: this needs to be handled
- The placement of the output should be reconsidered and/or cached when multiple IPs belong to a FQDN
set_scanning_defaults() reset do_ipv4_only/do_ipv6_only to false. When
no individual test is requested on the command line, count_do_variables
returns 0 and parse_cmd_line() calls set_scanning_defaults() to enable
the standard test suite, which silently wiped out the user's -4/-6
selection. As a result determine_ip_addresses() fell through to its
"scan everything" branch and scanned both the A and AAAA records,
producing one scanResult per IP.
The IP-version flags are connection options, not test-selection state,
and are already initialized in initialize_globals(), so remove the
stray resets from set_scanning_defaults().
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dirk Wetter