Merge pull request #1807 from tosticated/custom_http_headers

Custom HTTP request headers support added. Addresses #1770

CHANGELOG.md

@@ -28,6 +28,7 @@
 * Added --overwrite argument to support overwriting output files without warning
 * Headerflag X-XSS-Protection is now labeled as INFO
 * Client simulation runs in wide mode which is even better readable
+* Added --reqheader to support custom headers in HTTP requests
 
 ### Features implemented / improvements in 3.0
 

CREDITS.md

@@ -42,6 +42,7 @@ Full contribution, see git log.
 * Jim Blankendaal
   - maximum certificate lifespan of 398 days
   - ssl renegotiation amount variable
+  - custom http request headers
 
 * Frank Breedijk
   - Detection of insecure redirects
@@ -181,4 +182,3 @@ Probably more I forgot to mention which did give me feedback, bug reports and he
 * Ivan Ristic/Qualys for the liberal license which made it possible to make partly use of the client data
 
 * My family for supporting me doing this work
-

doc/testssl.1

@@ -136,6 +136,9 @@ Please note that \fBfname\fR has to be in Unix format\. DOS carriage returns won
 .P
 \fB\-\-basicauth <user:pass>\fR This can be set to provide HTTP basic auth credentials which are used during checks for security headers\. BASICAUTH is the ENV variable you can use instead\.
 .
+.P
+\fB\-\-reqheader <header>\fR This can be used to add additional HTTP request headers in the correct format \fBHeadername: headercontent\fR\. This parameter can be called multiple times if required\. For example: \fB\-\-reqheader \'Proxy\-Authorization: Basic dGVzdHNzbDpydWxlcw==\' \-\-reqheader \'ClientID: 0xDEADBEAF\'\fR\. REQHEADER is the corresponding environment variable\.
+.
 .SS "SPECIAL INVOCATIONS"
 \fB\-t <protocol>, \-\-starttls <protocol>\fR does a default run against a STARTTLS enabled \fBprotocol\fR\. \fBprotocol\fR must be one of \fBftp\fR, \fBsmtp\fR, \fBpop3\fR, \fBimap\fR, \fBxmpp\fR, \fBxmpp-server\fR, \fBtelnet\fR, \fBldap\fR, \fBirc\fR, \fBlmtp\fR, \fBnntp\fR, \fBpostgres\fR, \fBmysql\fR\. For the latter four you need e\.g\. the supplied OpenSSL or OpenSSL version 1\.1\.1\. Please note: MongoDB doesn\'t offer a STARTTLS connection, LDAP currently only works with \fB\-\-ssl\-native\fR\. \fBtelnet\fR and \fBirc\fR is WIP\.
 .

doc/testssl.1.html

@@ -187,6 +187,8 @@ The same can be achieved by setting the environment variable <code>WARNINGS</cod
 
 <p><code>--basicauth &lt;user:pass&gt;</code> This can be set to provide HTTP basic auth credentials which are used during checks for security headers. BASICAUTH is the ENV variable you can use instead.</p>
 
+<p><code>--reqheader &lt;header&gt;</code> This can be used to add additional HTTP request headers in the correct format <code>Headername: headercontent</code>. This parameter can be called multiple times if required. For example: <code>--reqheader &#39;Proxy-Authorization: Basic dGVzdHNzbDpydWxlcw==&#39; --reqheader &#39;ClientID: 0xDEADBEAF&#39;</code>. REQHEADER is the corresponding environment variable.</p>
+
 <h3 id="SPECIAL-INVOCATIONS">SPECIAL INVOCATIONS</h3>
 
 <p><code>-t &lt;protocol>, --starttls &lt;protocol></code>    does a default run against a STARTTLS enabled <code>protocol</code>. <code>protocol</code> must be one of <code>ftp</code>, <code>smtp</code>,  <code>pop3</code>, <code>imap</code>, <code>xmpp</code>, <code>xmpp-server</code>, <code>telnet</code>, <code>ldap</code>, <code>irc</code>, <code>lmtp</code>, <code>nntp</code>, <code>postgres</code>, <code>mysql</code>. For the latter four you need e.g. the supplied OpenSSL or OpenSSL version 1.1.1. Please note: MongoDB doesn't offer a STARTTLS connection, LDAP currently only works with <code>--ssl-native</code>. <code>telnet</code> and <code>irc</code> is WIP.</p>

doc/testssl.1.md

@@ -110,6 +110,8 @@ The same can be achieved by setting the environment variable `WARNINGS`.
 
 `--basicauth <user:pass>` This can be set to provide HTTP basic auth credentials which are used during checks for security headers. BASICAUTH is the ENV variable you can use instead.
 
+`--reqheader <header>` This can be used to add additional HTTP request headers in the correct format `Headername: headercontent`. This parameter can be called multiple times if required. For example: `--reqheader 'Proxy-Authorization: Basic dGVzdHNzbDpydWxlcw==' --reqheader 'ClientID: 0xDEADBEAF'`. REQHEADER is the corresponding environment variable.
+
 
 ### SPECIAL INVOCATIONS
 

testssl.sh

@@ -162,6 +162,7 @@ QUIET=${QUIET:-false}                   # don't output the banner. By doing this
 SSL_NATIVE=${SSL_NATIVE:-false}         # we do per default bash sockets where possible "true": switch back to "openssl native"
 ASSUME_HTTP=${ASSUME_HTTP:-false}       # in seldom cases (WAF, old servers, grumpy SSL) service detection fails. "True" enforces HTTP checks
 BASICAUTH=${BASICAUTH:-""}              # HTTP basic auth credentials can be set here like user:pass
+REQHEADER=${REQHEADER:-""}              # HTTP custom request header can be set here like Header: content. Can be used multiple times.
 BUGS=${BUGS:-""}                        # -bugs option from openssl, needed for some BIG IP F5
 WARNINGS=${WARNINGS:-""}                # can be either off or batch
 DEBUG=${DEBUG:-0}                       # 1: normal output the files in /tmp/ are kept for further debugging purposes
@@ -373,6 +374,7 @@ TLS_NOW=""                              # Similar
 TLS_DIFFTIME_SET=false                  # Tells TLS functions to measure the TLS difftime or not
 NOW_TIME=""
 HTTP_TIME=""
+REQHEADERS=()
 GET_REQ11=""
 START_TIME=0                            # time in epoch when the action started
 END_TIME=0                              # .. ended
@@ -886,6 +888,15 @@ is_ipv6addr() {
           return 1
 }
 
+join_by() {
+     # joins an array using a custom delimiter https://web.archive.org/web/20201222183540/https://stackoverflow.com/questions/1527049/how-can-i-join-elements-of-an-array-in-bash/17841619#17841619
+     local d=$1
+     shift
+     local f=$1
+     shift
+     printf %s "$f" "${@/#/$d}";
+}
+
 ###### END universal helper function definitions ######
 
 ###### START ServerHello/OpenSSL/F5 function definitions ######
@@ -19239,6 +19250,7 @@ tuning / connect options (most also can be preset via environment variables):
      --phone-out                   allow to contact external servers for CRL download and querying OCSP responder
      --add-ca <CA files|CA dir>    path to <CAdir> with *.pem or a comma separated list of CA files to include in trust check
      --basicauth <user:pass>       provide HTTP basic auth information.
+     --reqheader <header>          add custom http request headers
 
 output options (can also be preset via environment variables):
      --quiet                       don't output the banner. By doing this you acknowledge usage terms normally appearing in the banner
@@ -19391,6 +19403,7 @@ SHOW_EACH_C: $SHOW_EACH_C
 SSL_NATIVE: $SSL_NATIVE
 ASSUME_HTTP $ASSUME_HTTP
 BASICAUTH: $BASICAUTH
+REQHEADER: $REQHEADER
 SNEAKY: $SNEAKY
 OFFENSIVE: $OFFENSIVE
 PHONE_OUT: $PHONE_OUT
@@ -20514,6 +20527,7 @@ determine_service() {
      local ua
      local protocol
      local basicauth_header=""
+     local reqheader=""
 
      # Check if we can connect to $NODEIP:$PORT. Attention: This ALWAYS uses sockets. Thus timeouts for --ssl-=native do not apply
      if ! fd_socket 5; then
@@ -20541,7 +20555,10 @@ determine_service() {
           if [[ -n "$BASICAUTH" ]]; then
                basicauth_header="Authorization: Basic $(safe_echo "$BASICAUTH" | $OPENSSL base64 2>/dev/null)\r\n"
           fi
-          GET_REQ11="GET $URL_PATH HTTP/1.1\r\nHost: $NODE\r\nUser-Agent: $ua\r\n${basicauth_header}Accept-Encoding: identity\r\nAccept: text/*\r\nConnection: Close\r\n\r\n"
+          if [[ -n "$REQHEADERS" ]]; then
+               reqheader="$(join_by "\r\n" "${REQHEADERS[@]}")\r\n" #Add all required custom http headers to one string with newlines
+          fi
+          GET_REQ11="GET $URL_PATH HTTP/1.1\r\nHost: $NODE\r\nUser-Agent: $ua\r\n${basicauth_header}${reqheader}Accept-Encoding: identity\r\nAccept: text/*\r\nConnection: Close\r\n\r\n"
           # returns always 0:
           service_detection $OPTIMAL_PROTO
      else # STARTTLS
@@ -22193,6 +22210,11 @@ parse_cmd_line() {
                     BASICAUTH="$(parse_opt_equal_sign "$1" "$2")"
                     [[ $? -eq 0 ]] && shift
                     ;;
+               --reqheader|--reqheader=*)
+                    REQHEADER="$(parse_opt_equal_sign "$1" "$2")"
+                    [[ $? -eq 0 ]] && shift
+                    REQHEADERS+=("$REQHEADER")
+                    ;;
                (--) shift
                     break
                     ;;