1SSL tests 2========= 3 4SSL testcases are configured in the `ssl-tests` directory. 5 6Each `ssl_*.cnf.in` file contains a number of test configurations. These files 7are used to generate testcases in the OpenSSL CONF format. 8 9The precise test output can be dependent on the library configuration. The test 10harness generates the output files on the fly. 11 12However, for verification, we also include checked-in configuration outputs 13corresponding to the default configuration. These testcases live in 14`test/ssl-tests/*.cnf` files. 15 16For more details, see `ssl-tests/01-simple.cnf.in` for an example. 17 18Configuring the test 19-------------------- 20 21First, give your test a name. The names do not have to be unique. 22 23An example test input looks like this: 24 25 { 26 name => "test-default", 27 server => { "CipherString" => "DEFAULT" }, 28 client => { "CipherString" => "DEFAULT" }, 29 test => { "ExpectedResult" => "Success" }, 30 } 31 32The test section supports the following options 33 34### Test mode 35 36* Method - the method to test. One of DTLS or TLS. 37 38* HandshakeMode - which handshake flavour to test: 39 - Simple - plain handshake (default) 40 - Resume - test resumption 41 - RenegotiateServer - test server initiated renegotiation 42 - RenegotiateClient - test client initiated renegotiation 43 44When HandshakeMode is Resume or Renegotiate, the original handshake is expected 45to succeed. All configured test expectations are verified against the second 46handshake. 47 48* ApplicationData - amount of application data bytes to send (integer, defaults 49 to 256 bytes). Applies to both client and server. Application data is sent in 50 64kB chunks (but limited by MaxFragmentSize and available parallelization, see 51 below). 52 53* MaxFragmentSize - maximum send fragment size (integer, defaults to 512 in 54 tests - see `SSL_CTX_set_max_send_fragment` for documentation). Applies to 55 both client and server. Lowering the fragment size will split handshake and 56 application data up between more `SSL_write` calls, thus allowing to exercise 57 different code paths. In particular, if the buffer size (64kB) is at least 58 four times as large as the maximum fragment, interleaved multi-buffer crypto 59 implementations may be used on some platforms. 60 61### Test expectations 62 63* ExpectedResult - expected handshake outcome. One of 64 - Success - handshake success 65 - ServerFail - serverside handshake failure 66 - ClientFail - clientside handshake failure 67 - InternalError - some other error 68 69* ExpectedClientAlert, ExpectedServerAlert - expected alert. See 70 `test/helpers/ssl_test_ctx.c` for known values. Note: the expected alert is currently 71 matched against the _last_ received alert (i.e., a fatal alert or a 72 `close_notify`). Warning alert expectations are not yet supported. (A warning 73 alert will not be correctly matched, if followed by a `close_notify` or 74 another alert.) 75 76* ExpectedProtocol - expected negotiated protocol. One of 77 SSLv3, TLSv1, TLSv1.1, TLSv1.2. 78 79* SessionTicketExpected - whether or not a session ticket is expected 80 - Ignore - do not check for a session ticket (default) 81 - Yes - a session ticket is expected 82 - No - a session ticket is not expected 83 84* SessionIdExpected - whether or not a session id is expected 85 - Ignore - do not check for a session id (default) 86 - Yes - a session id is expected 87 - No - a session id is not expected 88 89* ResumptionExpected - whether or not resumption is expected (Resume mode only) 90 - Yes - resumed handshake 91 - No - full handshake (default) 92 93* ExpectedNPNProtocol, ExpectedALPNProtocol - NPN and ALPN expectations. 94 95* ExpectedTmpKeyType - the expected algorithm or curve of server temp key 96 97* ExpectedServerCertType, ExpectedClientCertType - the expected algorithm or 98 curve of server or client certificate 99 100* ExpectedServerSignHash, ExpectedClientSignHash - the expected 101 signing hash used by server or client certificate 102 103* ExpectedServerSignType, ExpectedClientSignType - the expected 104 signature type used by server or client when signing messages 105 106* ExpectedClientCANames - for client auth list of CA names the server must 107 send. If this is "empty" the list is expected to be empty otherwise it 108 is a file of certificates whose subject names form the list. 109 110* ExpectedServerCANames - list of CA names the client must send, TLS 1.3 only. 111 If this is "empty" the list is expected to be empty otherwise it is a file 112 of certificates whose subject names form the list. 113 114Configuring the client and server 115--------------------------------- 116 117The client and server configurations can be any valid `SSL_CTX` 118configurations. For details, see the manpages for `SSL_CONF_cmd`. 119 120Give your configurations as a dictionary of CONF commands, e.g. 121 122 server => { 123 "CipherString" => "DEFAULT", 124 "MinProtocol" => "TLSv1", 125 } 126 127The following sections may optionally be defined: 128 129* server2 - this section configures a secondary context that is selected via the 130 ServerName test option. This context is used whenever a ServerNameCallback is 131 specified. If the server2 section is not present, then the configuration 132 matches server. 133* resume_server - this section configures the client to resume its session 134 against a different server. This context is used whenever HandshakeMode is 135 Resume. If the resume_server section is not present, then the configuration 136 matches server. 137* resume_client - this section configures the client to resume its session with 138 a different configuration. In practice this may occur when, for example, 139 upgraded clients reuse sessions persisted on disk. This context is used 140 whenever HandshakeMode is Resume. If the resume_client section is not present, 141 then the configuration matches client. 142 143### Configuring callbacks and additional options 144 145Additional handshake settings can be configured in the `extra` section of each 146client and server: 147 148 client => { 149 "CipherString" => "DEFAULT", 150 extra => { 151 "ServerName" => "server2", 152 } 153 } 154 155#### Supported client-side options 156 157* ClientVerifyCallback - the client's custom certificate verify callback. 158 Used to test callback behaviour. One of 159 - None - no custom callback (default) 160 - AcceptAll - accepts all certificates. 161 - RejectAll - rejects all certificates. 162 163* ServerName - the server the client should attempt to connect to. One of 164 - None - do not use SNI (default) 165 - server1 - the initial context 166 - server2 - the secondary context 167 - invalid - an unknown context 168 169* CTValidation - Certificate Transparency validation strategy. One of 170 - None - no validation (default) 171 - Permissive - SSL_CT_VALIDATION_PERMISSIVE 172 - Strict - SSL_CT_VALIDATION_STRICT 173 174#### Supported server-side options 175 176* ServerNameCallback - the SNI switching callback to use 177 - None - no callback (default) 178 - IgnoreMismatch - continue the handshake on SNI mismatch 179 - RejectMismatch - abort the handshake on SNI mismatch 180 181* BrokenSessionTicket - a special test case where the session ticket callback 182 does not initialize crypto. 183 - No (default) 184 - Yes 185 186#### Mutually supported options 187 188* NPNProtocols, ALPNProtocols - NPN and ALPN settings. Server and client 189 protocols can be specified as a comma-separated list, and a callback with the 190 recommended behaviour will be installed automatically. 191 192* SRPUser, SRPPassword - SRP settings. For client, this is the SRP user to 193 connect as; for server, this is a known SRP user. 194 195### Default server and client configurations 196 197The default server certificate and CA files are added to the configurations 198automatically. Server certificate verification is requested by default. 199 200You can override these options by redefining them: 201 202 client => { 203 "VerifyCAFile" => "/path/to/custom/file" 204 } 205 206or by deleting them 207 208 client => { 209 "VerifyCAFile" => undef 210 } 211 212Adding a test to the test harness 213--------------------------------- 214 2151. Add a new test configuration to `test/ssl-tests`, following the examples of 216 existing `*.cnf.in` files (for example, `01-simple.cnf.in`). 217 2182. Generate the generated `*.cnf` test input file. You can do so by running 219 `generate_ssl_tests.pl`: 220 221 $ ./config 222 $ cd test 223 $ TOP=.. perl -I ../util/perl/ generate_ssl_tests.pl \ 224 ssl-tests/my.cnf.in default > ssl-tests/my.cnf 225 226where `my.cnf.in` is your test input file and `default` is the provider to use. 227For all the pre-generated test files you should use the default provider. 228 229For example, to generate the test cases in `ssl-tests/01-simple.cnf.in`, do 230 231 $ TOP=.. perl -I ../util/perl/ generate_ssl_tests.pl \ 232 ssl-tests/01-simple.cnf.in default > ssl-tests/01-simple.cnf 233 234Alternatively (hackish but simple), you can comment out 235 236 unlink glob $tmp_file; 237 238in `test/recipes/80-test_ssl_new.t` and run 239 240 $ make TESTS=test_ssl_new test 241 242This will save the generated output in a `*.tmp` file in the build directory. 243 2443. Update the number of tests planned in `test/recipes/80-test_ssl_new.t`. If 245 the test suite has any skip conditions, update those too (see 246 `test/recipes/80-test_ssl_new.t` for details). 247 248Running the tests with the test harness 249--------------------------------------- 250 251 HARNESS_VERBOSE=yes make TESTS=test_ssl_new test 252 253Running a test manually 254----------------------- 255 256These steps are only needed during development. End users should run `make test` 257or follow the instructions above to run the SSL test suite. 258 259To run an SSL test manually from the command line, the `TEST_CERTS_DIR` 260environment variable to point to the location of the certs. E.g., from the root 261OpenSSL directory, do 262 263 $ CTLOG_FILE=test/ct/log_list.cnf TEST_CERTS_DIR=test/certs test/ssl_test \ 264 test/ssl-tests/01-simple.cnf default 265 266or for shared builds 267 268 $ CTLOG_FILE=test/ct/log_list.cnf TEST_CERTS_DIR=test/certs \ 269 util/wrap.pl test/ssl_test test/ssl-tests/01-simple.cnf default 270 271In the above examples, `default` is the provider to use. 272 273Note that the test expectations sometimes depend on the Configure settings. For 274example, the negotiated protocol depends on the set of available (enabled) 275protocols: a build with `enable-ssl3` has different test expectations than a 276build with `no-ssl3`. 277 278The Perl test harness automatically generates expected outputs, so users who 279just run `make test` do not need any extra steps. 280 281However, when running a test manually, keep in mind that the repository version 282of the generated `test/ssl-tests/*.cnf` correspond to expected outputs in with 283the default Configure options. To run `ssl_test` manually from the command line 284in a build with a different configuration, you may need to generate the right 285`*.cnf` file from the `*.cnf.in` input first. 286 287Running a test manually via make 288-------------------------------- 289 290Individual tests may be run by adding the SSL_TESTS variable to the `make` 291command line. The SSL_TESTS variable is set to the list of input (or ".in") 292files. The values in SSL_TESTS are globbed. 293 294 $ make test TESTS=test_ssl_new SSL_TESTS="0*.cnf.in" 295 296 $ make test TESTS=test_ssl_new SSL_TESTS="01-simple.cnf.in 05-sni.cnf.in" 297
