#
d82f9f96 |
| 23-Sep-2024 |
Viktor Szakats |
build: add pytest targets It enables running pytests in cmake jobs, regardless of underlying build tool choice (= makes it work with ninja.) Also: - drop pytest logic l
build: add pytest targets It enables running pytests in cmake jobs, regardless of underlying build tool choice (= makes it work with ninja.) Also: - drop pytest logic launching `make` and exiting in case of failure. Maybe there is a better way and keep this functionality somehow, bind it to a command-line option? make it fail softly? - GHA/linux: invoke pytest via the build, not directly. - autotools: add missing dummy runtests targets when cross-compiling. Closes #15034
show more ...
|
#
44d1b6c2 |
| 29-Aug-2024 |
Stefan Eissing |
pytest: add ftp upload ascii test Add a test the uploads a text file in ascii mode and checks that lengths match expectations. Closes #14721
|
#
ea6f5c9f |
| 27-Aug-2024 |
Stefan Eissing |
connect: limit update IP info Update IP related information at the connection and the transfer in two places only: once the filter chain connects and when a transfer is added to a co
connect: limit update IP info Update IP related information at the connection and the transfer in two places only: once the filter chain connects and when a transfer is added to a connection. The latter only updates on reuse when the filters already are connected. The only user of that information before a full connect is the HAProxy filter. Add cfilter CF_QUERY_IP_INFO query to let it find the information from the filters "below". This solves two issues with the previous version: - updates where often done twice with the same info - happy eyeballing filter "forks" could overwrite each others updates before the full winner was determined. Closes #14699
show more ...
|
#
c9b95c0b |
| 19-Jun-2024 |
Stefan Eissing |
lib: graceful connection shutdown When libcurl discards a connection there are two phases this may go through: "shutdown" and "closing". If a connection is aborted, the shutdown phas
lib: graceful connection shutdown When libcurl discards a connection there are two phases this may go through: "shutdown" and "closing". If a connection is aborted, the shutdown phase is skipped and it is closed right away. The connection filters attached to the connection implement the phases in their `do_shutdown()` and `do_close()` callbacks. Filters carry now a `shutdown` flags next to `connected` to keep track of the shutdown operation. Filters are shut down from top to bottom. If a filter is not connected, its shutdown is skipped. Notable filters that *do* something during shutdown are HTTP/2 and TLS. HTTP/2 sends the GOAWAY frame. TLS sends its close notify and expects to receive a close notify from the server. As sends and receives may EAGAIN on the network, a shutdown is often not successful right away and needs to poll the connection's socket(s). To facilitate this, such connections are placed on a new shutdown list inside the connection cache. Since managing this list requires the cooperation of a multi handle, only the connection cache belonging to a multi handle is used. If a connection was in another cache when being discarded, it is removed there and added to the multi's cache. If no multi handle is available at that time, the connection is shutdown and closed in a one-time, best-effort attempt. When a multi handle is destroyed, all connection still on the shutdown list are discarded with a final shutdown attempt and close. In curl debug builds, the environment variable `CURL_GRACEFUL_SHUTDOWN` can be set to make this graceful with a timeout in milliseconds given by the variable. The shutdown list is limited to the max number of connections configured for a multi cache. Set via CURLMOPT_MAX_TOTAL_CONNECTIONS. When the limit is reached, the oldest connection on the shutdown list is discarded. - In multi_wait() and multi_waitfds(), collect all connection caches involved (each transfer might carry its own) into a temporary list. Let each connection cache on the list contribute sockets and POLLIN/OUT events it's connections are waiting for. - in multi_perform() collect the connection caches the same way and let them peform their maintenance. This will make another non-blocking attempt to shutdown all connections on its shutdown list. - for event based multis (multi->socket_cb set), add the sockets and their poll events via the callback. When `multi_socket()` is invoked for a socket not known by an active transfer, forward this to the multi's cache for processing. On closing a connection, remove its socket(s) via the callback. TLS connection filters MUST NOT send close nofity messages in their `do_close()` implementation. The reason is that a TLS close notify signals a success. When a connection is aborted and skips its shutdown phase, the server needs to see a missing close notify to detect something has gone wrong. A graceful shutdown of FTP's data connection is performed implicitly before regarding the upload/download as complete and continuing on the control connection. For FTP without TLS, there is just the socket close happening. But with TLS, the sent/received close notify signals that the transfer is complete and healthy. Servers like `vsftpd` verify that and reject uploads without a TLS close notify. - added test_19_* for shutdown related tests - test_19_01 and test_19_02 test for TCP RST packets which happen without a graceful shutdown and should no longer appear otherwise. - add test_19_03 for handling shutdowns by the server - add test_19_04 for handling shutdowns by curl - add test_19_05 for event based shutdowny by server - add test_30_06/07 and test_31_06/07 for shutdown checks on FTP up- and downloads. Closes #13976
show more ...
|
#
34555724 |
| 15-May-2024 |
Stefan Eissing |
pytest: fixes for recent python, add FTP tests Fixes: - in uds tests, abort also silently on os errors - be conservative on the h3 goaway duration - detect curl debug build and u
pytest: fixes for recent python, add FTP tests Fixes: - in uds tests, abort also silently on os errors - be conservative on the h3 goaway duration - detect curl debug build and use in checks - fix caddy version check for slight difference under linux - set caddy default path fitting for linux - fix deprecation warnings in valid time checks FTP tests: - add '--with-test-vsftpd=path' to configure - use vsftpd default path suitable for linux - add test_30 with plain FTP tests - add test_31 with --ssl-reqd FTP tests - add vsftpd to linux GHA for pytest workflows Closes #13661
show more ...
|
#
b06619d0 |
| 26-Apr-2024 |
Stefan Eissing |
tests: add SNI and peer name checks - connect to DNS names with trailing dot - connect to DNS names with double trailing dot - rustls, always give `peer->hostname` and let it f
tests: add SNI and peer name checks - connect to DNS names with trailing dot - connect to DNS names with double trailing dot - rustls, always give `peer->hostname` and let it figure out SNI itself - add SNI tests for ip address and localhost - document in code and TODO that QUIC with ngtcp2+wolfssl does not do proper peer verification of the certificate - mbedtls, skip tests with ip address verification as not supported by the library Closes #13486
show more ...
|
#
3e16c97f |
| 18-Apr-2024 |
Stefan Eissing |
tests: check caddy server version to match test expectations - new caddy servers no longer return 200 on POSTs, but 405 as they should Closes #13405
|
#
08d10d2a |
| 15-Apr-2024 |
Stefan Eissing |
http3: extend download abort tests, fixes in ngtcp2 - fix flow handling in ngtcp2 to ACK data on streams we abort ourself. - extend test_02_23* cases to also run for h3 - skip
http3: extend download abort tests, fixes in ngtcp2 - fix flow handling in ngtcp2 to ACK data on streams we abort ourself. - extend test_02_23* cases to also run for h3 - skip test_02_23* for OpenSSL QUIC as it gets stalled on progressing the connection Closes #13374
show more ...
|
#
b30d694a |
| 28-Mar-2024 |
Stefan Eissing |
content_encoding: brotli and others, pass through 0-length writes - curl's transfer handling may write 0-length chunks at the end of the download with an EOS flag. (HTTP/2 does this co
content_encoding: brotli and others, pass through 0-length writes - curl's transfer handling may write 0-length chunks at the end of the download with an EOS flag. (HTTP/2 does this commonly) - content encoders need to pass-through such a write and not count this as error in case they are finished decoding Fixes #13209 Fixes #13212 Closes #13219
show more ...
|
#
3d0fd382 |
| 15-Mar-2024 |
Stefan Eissing |
mbedtls: fix pytest for newer versions Fix the expectations in pytest for newer versions of mbedtls Closes #13132
|
#
c177e194 |
| 01-Feb-2024 |
Stefan Eissing |
https-proxy: use IP address and cert with ip in alt names - improve info logging when peer verification fails to indicate if DNS name or ip address has been tried to match - add te
https-proxy: use IP address and cert with ip in alt names - improve info logging when peer verification fails to indicate if DNS name or ip address has been tried to match - add test case for contacting https proxy with ip address - add pytest env check on loaded credentials and re-issue when they are no longer valid - disable proxy ip address test for bearssl, since not supported there Ref: #12831 Closes #12838
show more ...
|
#
3b30cc1a |
| 06-Sep-2023 |
Stefan Eissing |
pytest: improvements - set CURL_CI for pytest runs in CI environments - exclude timing sensitive tests from CI runs - for failed results, list only the log and stat of the fail
pytest: improvements - set CURL_CI for pytest runs in CI environments - exclude timing sensitive tests from CI runs - for failed results, list only the log and stat of the failed transfer - fix type in http.c comment Closes #11812
show more ...
|
#
331b89a3 |
| 29-Aug-2023 |
Stefan Eissing |
http2: polish things around POST - added test cases for various code paths - fixed handling of blocked write when stream had been closed inbetween attempts - re-enabled DEBUGAS
http2: polish things around POST - added test cases for various code paths - fixed handling of blocked write when stream had been closed inbetween attempts - re-enabled DEBUGASSERT on send with smaller data size - in debug builds, environment variables can be set to simulate a slow network when sending data. cf-socket.c and vquic.c support * CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be answered with a EAGAIN. TCP/UNIX sockets. This is chosen randomly. * CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written to the network. TCP/UNIX sockets. Example: 80 means a send with 1000 bytes would only send 800 This is applied to every send. * CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be answered with EAGAIN. QUIC only. This is chosen randomly. Closes #11756
show more ...
|
#
ce3dce90 |
| 31-Aug-2023 |
Viktor Szakats |
tidy-up: mostly whitespace nits - delete completed TODO from `./CMakeLists.txt`. - convert a C++ comment to C89 in `./CMake/CurlTests.c`. - delete duplicate EOLs from EOF. - add
tidy-up: mostly whitespace nits - delete completed TODO from `./CMakeLists.txt`. - convert a C++ comment to C89 in `./CMake/CurlTests.c`. - delete duplicate EOLs from EOF. - add missing EOL at EOF. - delete whitespace at EOL (except from expected test results). - convert tabs to spaces. - convert CRLF EOLs to LF in GHA yaml. - text casing fixes in `./CMakeLists.txt`. - fix a codespell typo in `packages/OS400/initscript.sh`. Closes #11772
show more ...
|
#
acd82c8b |
| 26-Apr-2023 |
Stefan Eissing |
tests/http: more tests with specific clients - Makefile support for building test specific clients in tests/http/clients - auto-make of clients when invoking pytest - added test_09_0
tests/http: more tests with specific clients - Makefile support for building test specific clients in tests/http/clients - auto-make of clients when invoking pytest - added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush - added test_02_21 for lib based downloads and pausing/unpausing transfers curl url parser: - added internal method `curl_url_set_authority()` for setting the authority part of a url (used for PUSH_PROMISE) http2: - made logging of PUSH_PROMISE handling nicer Placing python test requirements in requirements.txt files - separate files to base test suite and http tests since use and module lists differ - using the files in the gh workflows websocket test cases, fixes for we and bufq - bufq: account for spare chunks in space calculation - bufq: reset chunks that are skipped empty - ws: correctly encode frames with 126 bytes payload - ws: update frame meta information on first call of collect callback that fills user buffer - test client ws-data: some test/reporting improvements Closes #11006
show more ...
|
#
4ae2d9f2 |
| 06-Apr-2023 |
Stefan Eissing |
proxy: http2 proxy tunnel implementation - currently only on debug build and when env variable CURL_PROXY_TUNNEL_H2 is present. - will ALPN negotiate with the proxy server and swit
proxy: http2 proxy tunnel implementation - currently only on debug build and when env variable CURL_PROXY_TUNNEL_H2 is present. - will ALPN negotiate with the proxy server and switch tunnel filter based on the protocol negotiated. - http/1.1 tunnel code moved into cf-h1-proxy.[ch] - http/2 tunnel code implemented in cf-h2-proxy.[ch] - tunnel start and ALPN set remains in http_proxy.c - moving all haproxy related code into cf-haproxy.[ch] VTLS changes - SSL filters rely solely on the "alpn" specification they are created with and no longer check conn->bits.tls_enable_alpn. - checks on which ALPN specification to use (or none at all) are done in vtls.c when creating the filter. Testing - added a nghttpx forward proxy to the pytest setup that speaks HTTP/2 and forwards all requests to the Apache httpd forward proxy server. - extending test coverage in test_10 cases - adding proxy tests for direct/tunnel h1/h2 use of basic auth. - adding test for http/1.1 and h2 proxy tunneling to pytest Closes #10780
show more ...
|
#
8f50e393 |
| 05-Apr-2023 |
Stefan Eissing |
vtls and h2 improvements - eliminate receive loop in vtls to fill buffer. This may lead to partial reads of data which is counter productive - let http2 instead loop smarter to pro
vtls and h2 improvements - eliminate receive loop in vtls to fill buffer. This may lead to partial reads of data which is counter productive - let http2 instead loop smarter to process pending network data without transfer switches scorecard improvements - do not start caddy when only httpd is requested - allow curl -v to stderr file on --curl-verbose Closes #10891
show more ...
|
#
e0c3424f |
| 03-Apr-2023 |
Stefan Eissing |
tests/http: improved httpd detection - better error messages when not found/complete - handling of `--without-test-httpd` Reported-by: kwind on github Fixes #10879 Close
tests/http: improved httpd detection - better error messages when not found/complete - handling of `--without-test-httpd` Reported-by: kwind on github Fixes #10879 Closes #10883
show more ...
|
#
3797f1a4 |
| 31-Mar-2023 |
Stefan Eissing |
rustls: fix error in recv handling - when rustls is told to recieve more TLS data and its internal plaintext buffers are full, it returns an IOERROR - avoid receiving TLS data whil
rustls: fix error in recv handling - when rustls is told to recieve more TLS data and its internal plaintext buffers are full, it returns an IOERROR - avoid receiving TLS data while plaintext is not read empty pytest: - increase curl run timeout when invoking pytest with higher verbosity Closes #10876
show more ...
|
#
744dcf22 |
| 30-Mar-2023 |
Stefan Eissing |
http2: flow control and buffer improvements - use bufq for send/receive of network data - usd bufq for send/receive of stream data - use HTTP/2 flow control with no-auto updates to c
http2: flow control and buffer improvements - use bufq for send/receive of network data - usd bufq for send/receive of stream data - use HTTP/2 flow control with no-auto updates to control the amount of data we are buffering for a stream HTTP/2 stream window set to 128K after local tests, defined code constant for now - elminiating PAUSEing nghttp2 processing when receiving data since a stream can now take in all DATA nghttp2 forwards Improved scorecard and adjuste http2 stream window sizes - scorecard improved output formatting and options default - scorecard now also benchmarks small requests / second Closes #10771
show more ...
|
#
8cabef6f |
| 24-Mar-2023 |
Stefan Eissing |
pytest: improvements for suitable curl and error output - will check built curl for http and https support and skip all tests if not there - will dump stdout/stderr/trace output on
pytest: improvements for suitable curl and error output - will check built curl for http and https support and skip all tests if not there - will dump stdout/stderr/trace output on errored responses Closes #10829
show more ...
|
#
9d107b69 |
| 17-Mar-2023 |
Stefan Eissing |
tests/http: add timeout to running curl in test cases - we had a CI case once where `curl` seemingly did not return and it was hard to guess what happened. - make curl execution in
tests/http: add timeout to running curl in test cases - we had a CI case once where `curl` seemingly did not return and it was hard to guess what happened. - make curl execution in test cases time out after 60 seconds Closes #10783
show more ...
|
#
c46761bd |
| 14-Mar-2023 |
Daniel Stenberg |
tests/http: remove year ranges from copyrights Closes #10763
|
#
7fa6e365 |
| 09-Mar-2023 |
Stefan Eissing |
tests/http: add pytest to GHA and improve tests - added to: ngtcp2-quictls, ngtcp2-gnutls and the linux varians quiche, bearssl, libressl, mbedtls, openssl3, rustls - added disable
tests/http: add pytest to GHA and improve tests - added to: ngtcp2-quictls, ngtcp2-gnutls and the linux varians quiche, bearssl, libressl, mbedtls, openssl3, rustls - added disabled in ngtcp2-wolfssl due to weird SSL_connect() errors not reproducable locally Improvements on pytest: - handling of systems with nghttpx in $PATH - configure will seach $PATH got nghttpx used in pytest - pytest fixes for managing nghttpx without h3 support - ngtcp2-wolfssl: use a fully enabled wolfssl build - lower parallel count for http/1.1 tests, since we do not want to test excessive connections. - check built curl for HTTPS-proxy support in proxy tests - bearssl does not like one of our critical cert extensions, making it non-critical now - bearssl is too slow for test_12, skipping - making sure we do h3 tests only when curl and server support is there Closes #10699
show more ...
|
#
b0564c1d |
| 06-Mar-2023 |
Stefan Eissing |
tests: use dynamic ports numbers in pytest suite - necessary ports are bound at start of test suite and then given to server fixtures for use. - this make parallel use of pytest (i
tests: use dynamic ports numbers in pytest suite - necessary ports are bound at start of test suite and then given to server fixtures for use. - this make parallel use of pytest (in separate directories), practically safe for use as OS tend to not reuse such port numbers for a while Closes #10692
show more ...
|