1Running external test suites with OpenSSL 2========================================= 3 4It is possible to integrate external test suites into OpenSSL's `make test`. 5This capability is considered a developer option and does not work on all 6platforms. 7 8Python PYCA/Cryptography test suite 9=================================== 10 11This python test suite runs cryptographic tests with a local OpenSSL build as 12the implementation. 13 14First checkout the `PYCA/Cryptography` module into `./pyca-cryptography` using: 15 16 $ git submodule update --init 17 18Then configure/build OpenSSL compatible with the python module: 19 20 $ ./config enable-external-tests 21 $ make 22 23The tests will run in a python virtual environment which requires virtualenv 24to be installed. 25 26 $ make test VERBOSE=1 TESTS=test_external_pyca 27 28Test failures and suppressions 29------------------------------ 30 31Some tests target older (<=1.0.2) versions so will not run. Other tests target 32other crypto implementations so are not relevant. Currently no tests fail. 33 34krb5 test suite 35=============== 36 37Much like the PYCA/Cryptography test suite, this builds and runs the krb5 38tests against the local OpenSSL build. 39 40You will need a git checkout of krb5 at the top level: 41 42 $ git clone https://github.com/krb5/krb5 43 44krb5's master has to pass this same CI, but a known-good version is 45krb5-1.15.1-final if you want to be sure. 46 47 $ cd krb5 48 $ git checkout krb5-1.15.1-final 49 $ cd .. 50 51OpenSSL must be built with external tests enabled: 52 53 $ ./config enable-external-tests 54 $ make 55 56krb5's tests will then be run as part of the rest of the suite, or can be 57explicitly run (with more debugging): 58 59 $ VERBOSE=1 make TESTS=test_external_krb5 test 60 61Test-failures suppressions 62-------------------------- 63 64krb5 will automatically adapt its test suite to account for the configuration 65of your system. Certain tests may require more installed packages to run. No 66tests are expected to fail. 67 68GOST engine test suite 69====================== 70 71Much like the PYCA/Cryptography test suite, this builds and runs the GOST engine 72tests against the local OpenSSL build. 73 74You will need a git checkout of gost-engine at the top level: 75 76 $ git submodule update --init 77 78Then configure/build OpenSSL enabling external tests: 79 80 $ ./config enable-external-tests 81 $ make 82 83GOST engine requires CMake for the build process. 84 85GOST engine tests will then be run as part of the rest of the suite, or can be 86explicitly run (with more debugging): 87 88 $ make test VERBOSE=1 TESTS=test_external_gost_engine 89 90OQSprovider test suite 91====================== 92 93Much like the PYCA/Cryptography test suite, this builds and runs the OQS 94(OpenQuantumSafe -- www.openquantumsafe.org) provider tests against the 95local OpenSSL build. 96 97You will need a git checkout of oqsprovider at the top level: 98 99 $ git submodule update --init 100 101Then configure/build OpenSSL enabling external tests: 102 103 $ ./config enable-external-tests 104 $ make 105 106oqsprovider requires CMake for the build process. 107 108OQSprovider tests will then be run as part of the rest of the suite, or can be 109explicitly run (with more debugging): 110 111 $ make test VERBOSE=1 TESTS=test_external_oqsprovider 112 113The environment variable `OQS_SKIP_TESTS` can be set to select tests and 114algorithms to be skipped, e.g. as follows: 115 116 OQS_SKIP_TESTS=kyber make test TESTS=test_external_oqsprovider 117 118The names of all supported quantum-safe algorithms are available at 119<https://github.com/open-quantum-safe/oqs-provider#algorithms> 120 121pkcs11-provider test suite 122========================== 123 124This builds and runs pkcs11-provider tests against the local OpenSSL build. 125 126You will need a git checkout of pkcs11-provider at the top level: 127 128 $ git submodule update --init 129 130Then configure/build OpenSSL enabling external tests: 131 132 $ ./config enable-external-tests 133 $ make 134 135pkcs11-provider requires meson for the build process. Moreover, it requires 136softhsm and nss softokn tokens and certtool, certutil, pkcs11-tool and expect 137to run the tests. 138 139Tests will then be run as part of the rest of the suite, or can be 140explicitly run (with more debugging): 141 142 $ make test VERBOSE=1 TESTS=test_external_pkcs11_provider 143 144Test failures and suppressions 145------------------------------ 146 147There are tests for different software tokens - softhsm, nss-softokn and kryoptic. 148Kryoptic tests will not run at this point. Currently no test fails. 149 150Updating test suites 151==================== 152 153To update the commit for any of the above test suites: 154 155- Make sure the submodules are cloned locally: 156 157 $ git submodule update --init --recursive 158 159- Enter subdirectory and pull from the repository (use a specific branch/tag if required): 160 161 $ cd `<submodule-dir>` 162 $ git pull origin master 163 164- Go to root directory, there should be a new git status: 165 166 $ cd ../ 167 $ git status 168 ... 169 # modified: `<submodule-dir>` (new commits) 170 ... 171 172- Add/commit/push the update 173 174 $ git add `<submodule-dir>` 175 $ git commit -m `"Updated <submodule> to latest commit"` 176 $ git push 177
