Commit Graph

1974 Commits

Author SHA1 Message Date
Dirk 63cb4ffc5e improved high level sections of DNS in determine_ip_addresses()
FIX #668

Polishing ``get_*_record()``

Simplfied ``main`` a bit
2017-06-01 18:08:13 +02:00
Dirk a90eb8c9be FIX #744 2017-06-01 16:24:45 +02:00
Dirk Wetter f3f29cd85c Merge pull request #760 from dcooper16/fix_757
Fix #757
2017-06-01 15:51:21 +02:00
Dirk e4f64463a4 FIX #758 2017-06-01 15:47:38 +02:00
David Cooper 5807b5e993 Fix #757
I believe I discovered the reason for issue #757: f2303a0d79.

This commit removed attempted to replace `$cbc_cipher_list_hex` (which was computed on the fly) with `$cbc_ciphers_hex` (which is static). However, the function was still using `$cbc_cipher_list_hex`, and since it wasn't being initialized to "" at the beginning of the function, the second call to `run_beast()` (to handle the second IP address) just appended to the value created by the first. Then, when the first two bytes were removed from the resulting string the result was a malformed cipher suite list, which caused `tls_sockets()` to fail.
2017-06-01 09:36:03 -04:00
Dirk 5890677d85 chmodded 2017-06-01 11:14:52 +02:00
Dirk de5b2aa042 readded basic check from seccubus whether check via starttls works 2017-06-01 11:14:04 +02:00
Dirk Wetter bd015b9129 Merge pull request #753 from dcooper16/mass_testing_command_line_error
Massing testing with command line error
2017-05-31 21:37:54 +02:00
Dirk 91b9236055 PoC for unit test in bash 2017-05-31 10:30:02 +02:00
David Cooper 28dadbfb19 Merge branch '2.9dev' into mass_testing_command_line_error 2017-05-30 16:52:55 -04:00
Dirk Wetter 55b89ee131 Rename 02_http.t to 32_http.t 2017-05-30 22:15:13 +02:00
Dirk Wetter 4afedb45b1 Rename 01_badssl.com.t to 31_badssl.com.t 2017-05-30 22:14:19 +02:00
Dirk Wetter 8e1ace839f Merge pull request #755 from dcooper16/stop_parent
Stop parent if child encounters parsing error
2017-05-30 21:05:05 +02:00
David Cooper 0f09af8566 Stop parent if child encounters parsing error
This PR implements the suggestion from #753 for a child process in mass testing to send a signal to the parent to exit if the child encounters an error parsing its command line. At the moment, the child only sends the signal if it encounters an error that results in the `help()` function being called, but that could easily be changed (e.g., to also send a signal if `fatal()` is called in the child process).

In the case of parallel mass testing, the cleanup function needs to call `get_next_message_testing_parallel_result()` for the child that sent the signal, since otherwise the child's error message would not be displayed. Since I cannot tell which child sent the signal, I just call `cleanup()`, which displays the output of all completed child processes. Since the child process will send the signal almost immediately after starting, it can be assumed the that process that send the signal will be the last one that completed, and so its output will be displayed last (so it isn't hidden from the user).

Note that PR #753 is still needed, since there are still scenarios in which a child would not produce any JSON output, but the parent testssl.sh would not exit (e.g., the child process cannot open a socket to the server it is supposed to test). In additional, PR #754 would still be useful, since it would be more user friendly to catch the error in the mass testing file immediately (when possible) rather that partway through a potentially time-consuming testing process.
2017-05-24 17:12:18 -04:00
David Cooper c831dd0fd3 Handle all empty JSON file cases 2017-05-23 14:52:25 -04:00
David Cooper 01fb0ba9a2 Merge branch '2.9dev' into mass_testing_command_line_error 2017-05-23 08:57:49 -04:00
Dirk de177a774c fix formatting problem in run_client_simulation() wide mode when CHACHA20/POLY1305-OLD ciphers
are encountered and remove 4 columns before protocols.

mimor readability improvements in prepare_array() and run_client_simulation()
2017-05-22 23:04:58 +02:00
David Cooper 1311fe595b Massing testing with command line error
There is a bug in testssl.sh that occurs if mass testing is being performed, there is an error in the command line for one of the child tests, and either a single HTML file or a single JSON file is being created.

If mass testing is being performed and `parse_cmd_line()` detects an error in the command line for one of the child tests, then it will call `help()`, which will exit the program, resulting in `cleanup ()` being called. `cleanup ()` will call `html_footer()` and `fileout_footer()`. Since `html_header()` and `json_header()` have not yet been called, `$HTMLHEADER` and `$JSONHEADER` will both be `true, and so `html_footer()` and `fileout_footer()` will output HTML and JSON footers, even though no headers have been output.

This PR fixes the problem by having `help()` set `$HTMLHEADER` and `$JSONHEADER` to `false` so that no HTML or JSON footers are created.

A related problem is that if a single JSON file is being created, the parent process will insert a separator (a comma) into the JSON file between the outputs of each child process. However, if there is an error in one of the child process's command lines, then this child process will not produce any JSON output and so the JSON file will have two consecutive separators (commas), which is invalid according to http://jsonlint.com.

This PR provides a partial fix for the problem for parallel mass testing by checking whether a child process has created a non-empty JSON output before adding a separator to the JSON file. It leaves two unresolved problems:

* It does not fix the problem at all for `run_mass_testing()`, where the separator is added before the test with the command line error is run.

* It does not fix the problem for parallel mass testing for the case in which the first child test has a command line error.
2017-05-22 16:57:15 -04:00
Dirk Wetter 26bf3300e8 Delete mapping.txt
has been replaced in 2.9dev by cipher-mapping.txt
2017-05-22 11:38:23 +02:00
Dirk d64fabafd5 completed the commit 601c810240
started to use the ip parameter to decide where the dns resoultion
takes place (see #739 and #748). --ip=proxy or DNS_VIA_PROXY=true
will mean DNS resolutioni is done by proxy,

furthermore: swapped a few pr_magenta by pr_warning. Generally
testssl.sh should use for warnings a warning function and not
directly a color (we want to be flexible). There are still a few
remainders for fatal() which use bold magenta and thus I haven't
changed yet.

Also html_reserved() is being called also if no html output
is being requested. This could be fixed better probably.
2017-05-19 20:28:18 +02:00
Dirk 601c810240 add stderr to line showing "Waiting for test" FIX #750 2017-05-19 17:09:47 +02:00
Dirk 041abd57ce FIX #749 and #751
Additionally fix a probable cmd line parsing problem where an argument '--file' was supplied with '='
2017-05-19 17:00:30 +02:00
Dirk 83b3be5636 update to also reflect dcooper16's work 2017-05-17 18:56:07 +02:00
Dirk Wetter a3c318655d Merge pull request #745 from dcooper16/run_mass_testing_parallel2
More improvements to mass testing in parallel
2017-05-17 18:34:13 +02:00
Dirk Wetter a5e224b082 Update Readme.md 2017-05-17 09:03:15 +02:00
David Cooper 9d9a658dca More improvements to mass testing in parallel
This PR introduces the following changes/improvements to running mass testing in parallel:

* Continuous feedback is provided on the progress of testing, so that testssl.sh doesn't appeared to be frozen, even if it is waiting a long time for a child test to complete. [The feedback text is sent to `stderr` so that it doesn't appear in the log file if the `--logging` option is used.]

* Text sent to `stderr` by a child test is captured and then displayed in context with the appropriate test rather than being displayed at the time the error occurs. [Note that this means that if the `--logfile <logfile>` option is used (with `logfile` being the name of a file), then `logfile` will include both `stdout` and `stderr`.]

* If a test is started but it does not complete (because testssl.sh was stopped or because the test timed out), then a message is displayed indicating that the test didn't finish.

I believe that `run_mass_testing_parallel()` is either ready for use or nearly ready for use. So, I changed the program to use `run_mass_testing_parallel()` rather than `run_mass_testing()` if the `$EXPERIMENTAL` flag is set.
2017-05-16 14:16:35 -04:00
Dirk dfda82aa7d Merge branch '2.9dev' of github.com:drwetter/testssl.sh into 2.9dev 2017-05-16 09:46:55 +02:00
Dirk 3b1638f603 small performance improvement for prepare_arrays() by replacing grep + awk by awk only 2017-05-16 09:45:16 +02:00
Dirk 59a175cba3 changed to Linux 2017-05-15 20:53:09 +02:00
Dirk Wetter 5bb1a67dde Update Readme.md 2017-05-15 20:14:11 +02:00
Dirk e8b5a82c7e Considerable reliability improvements for ticketbleed: if a handshake server hello
was received, testssl.sh tries 3 times to get memory from the server, If
the server returns different memory it's highly likely vulnerable.
(some more vulnerable devices to test against would be appreciated).

This is the default now -- all other hosts are mostly labled as OK.

Parsing SERVICE should be a little faster and more reliable

Increased needed debug level for output @ parse_tls_serverhello
2017-05-15 19:47:13 +02:00
Dirk b694930fc2 RFC 5077 states that the server MAY give a hint of the lifetime of the ticket.
Sometimes it just does not. In those case also sometimes session resumption
via tickets is supported.

This fixes the output for Session Ticket RFC 5077 and doesn't draw the wrong
conclusion from a missing lifetime hint.

It also tests for ticketbleed first whether there's a session ticket TLS
extension.
2017-05-15 13:18:20 +02:00
Dirk 49b1be9f34 fix json/csv output for ticketbleed 2017-05-15 10:08:34 +02:00
Dirk b4889a731b Upgraded proxy sockets to use CONNECT 1.0 to avoid proxy problems -- for the time being (see also #741/ #739) 2017-05-12 18:31:59 +02:00
Dirk 2aa68827b9 don't do double work, reordering stuff 2017-05-12 17:58:20 +02:00
Dirk f70bc4e08f better platform support, revert to pure /bin/sh, better verbosity... 2017-05-12 17:21:45 +02:00
Dirk Wetter bed1f8cc0d Merge pull request #743 from dcooper16/broken_pipe
Prevent broken pipe error
2017-05-12 16:18:18 +02:00
David Cooper 9897debd4d Prevent broken pipe error
When the `--log`, `--logging`, or `--logfile <logfile>` option is being used and testssl.sh is stopped, a "printf: write error: Broken pipe" message tends to appear. From what I can tell, this is a result of the `tee` process being killed before the `cleanup ()` function completes. At the moment, `cleanup ()` doesn't write very much to `stdout`, but if parallel mass testing is performed, then `cleanup ()` may try to write the results of several previously completed tests.

This PR fixes the problem by adding the `-i` option ("ignore interrupt signals") to `tee`.

Note that I have tested this on a Linux desktop and an OS X laptop, but don't have a MS Windows computer on which to test this change.
2017-05-12 09:51:19 -04:00
Dirk Wetter 2750febb2f Merge pull request #740 from dcooper16/run_mass_testing_parallel
Improvements to mass testing in parallel
2017-05-10 18:49:55 +02:00
David Cooper 55df788937 Improvements to mass testing in parallel
This PR provides improvements to `run_mass_testing_parallel()`. Currently, `run_mass_testing_parallel()` treats `$MAX_PARALLEL` as the maximum difference between the number of the test whose results were last processed and the number of the most recently started test. This means that test #40 will not be started until the results of test #20 have been processed. I've encountered situations in which tests 21 though 39 have completed, but test #20 is still running, and so no new tests are started.

This PR fixes the problem by checking the status of all running child tests to see if any are complete, rather than just looking at `$NEXT_PARALLEL_TEST_TO_FINISH`. This prevents one slow child test (or a few slow child tests) from slowing up the entire mass testing process.

This PR also changes the basis for determining whether a slow child process should be killed. Rather than waiting `$MAX_WAIT_TEST` seconds from the time that the parent started waiting (which is rather arbitrary), it kills the process if `$MAX_WAIT_TEST` seconds have passed since the child test was started. Given this, and that the above change makes it less likely that a slow child test will slow up the overall testing, I increased `$MAX_WAIT_TEST` from 600 seconds to 1200 seconds.

I added some `debugme` statements that provide feedback on the status of testing, but in non-debug mode there may be a perception issue. If one test (e.g., test #20) is very slow, testssl.sh will not display any results from later tests until the slow test finishes, even though testssl.sh will continue running new tests in the background. The user, seeing no output from testssl.sh for an extended period of time, may think that testssl.sh has frozen, even though it is really just holding back on displaying the later results so that the results will be displayed in the order in which the tests were started.
2017-05-10 12:18:59 -04:00
Dirk 2936a42bc7 address #626 2017-05-09 21:58:03 +02:00
Dirk 9ed47eaa19 FIX #718 (added TLS padding to ticketbleed handshake). Also added TLS extension Signature Algorithms 2017-05-09 17:29:57 +02:00
Dirk 23e6209beb Merge branch '2.9dev' of github.com:drwetter/testssl.sh into 2.9dev 2017-05-08 23:55:19 +02:00
Dirk ebd9e6ae65 manually merged #728 (see #423), credits also to @seccubus. Unfortunately the unit tests don't make so much sense atm 2017-05-08 23:51:37 +02:00
Dirk Wetter 53da6da77b Merge pull request #730 from typingArtist/729_catch_read_timeout
correctly capture return code in starttls_full_read
2017-05-04 22:32:23 +02:00
Dirk 699b48c8b8 lowering severity and taking other clients as browsers into account #735 2017-05-04 12:34:20 +02:00
Dirk Wetter 19052da1a5 Merge pull request #735 from tkaehn/alert_on_missing_sans
Alert on missing SANs
2017-05-04 11:52:47 +02:00
Thomas Kähn 5293c51bc4 Alert on missing SANs 2017-05-04 11:09:27 +02:00
Dirk Wetter c9b6ee25b1 Delete 11_hpkp.t 2017-05-04 10:29:06 +02:00
Dirk 7d8479f55e temporary disabled until either an replacement has been coded or host is up again 2017-05-04 10:14:42 +02:00