#
d76b6485 |
| 31-Aug-2024 |
Daniel Stenberg |
rand: only provide weak random when needed builds without TLS and builds using rustls Closes #14749
|
#
c5cb8e7c |
| 19-Aug-2024 |
Viktor Szakats |
tidy-up: spelling quiche and Rustls Closes #14605
|
#
0a5ea09a |
| 29-Feb-2024 |
Max Faxälv |
spnego_gssapi: implement TLS channel bindings for openssl Channel Bindings are used to tie the session context to a specific TLS channel. This is to provide additional proof of valid ide
spnego_gssapi: implement TLS channel bindings for openssl Channel Bindings are used to tie the session context to a specific TLS channel. This is to provide additional proof of valid identity, mitigating authentication relay attacks. Major web servers have the ability to require (None/Accept/Require) GSSAPI channel binding, rendering Curl unable to connect to such websites unless support for channel bindings is implemented. IIS calls this feature Extended Protection (EPA), which is used in Enterprise environments using Kerberos for authentication. This change require krb5 >= 1.19, otherwise channel bindings won't be forwarded through SPNEGO. Co-Authored-By: Steffen Kieß <947515+steffen-kiess@users.noreply.github.com> Closes #13098
show more ...
|
#
46a26f12 |
| 08-Jul-2024 |
Stefan Eissing |
vtls: replace addsessionid with set_sessionid - deduplicate the code in many tls backends that check for an existing id and delete it before adding the new one - rename ssl_primary
vtls: replace addsessionid with set_sessionid - deduplicate the code in many tls backends that check for an existing id and delete it before adding the new one - rename ssl_primary_config's `sessionid` bool to `cache_session` Closes #14121
show more ...
|
#
c9b95c0b |
| 19-Jun-2024 |
Stefan Eissing |
lib: graceful connection shutdown When libcurl discards a connection there are two phases this may go through: "shutdown" and "closing". If a connection is aborted, the shutdown phas
lib: graceful connection shutdown When libcurl discards a connection there are two phases this may go through: "shutdown" and "closing". If a connection is aborted, the shutdown phase is skipped and it is closed right away. The connection filters attached to the connection implement the phases in their `do_shutdown()` and `do_close()` callbacks. Filters carry now a `shutdown` flags next to `connected` to keep track of the shutdown operation. Filters are shut down from top to bottom. If a filter is not connected, its shutdown is skipped. Notable filters that *do* something during shutdown are HTTP/2 and TLS. HTTP/2 sends the GOAWAY frame. TLS sends its close notify and expects to receive a close notify from the server. As sends and receives may EAGAIN on the network, a shutdown is often not successful right away and needs to poll the connection's socket(s). To facilitate this, such connections are placed on a new shutdown list inside the connection cache. Since managing this list requires the cooperation of a multi handle, only the connection cache belonging to a multi handle is used. If a connection was in another cache when being discarded, it is removed there and added to the multi's cache. If no multi handle is available at that time, the connection is shutdown and closed in a one-time, best-effort attempt. When a multi handle is destroyed, all connection still on the shutdown list are discarded with a final shutdown attempt and close. In curl debug builds, the environment variable `CURL_GRACEFUL_SHUTDOWN` can be set to make this graceful with a timeout in milliseconds given by the variable. The shutdown list is limited to the max number of connections configured for a multi cache. Set via CURLMOPT_MAX_TOTAL_CONNECTIONS. When the limit is reached, the oldest connection on the shutdown list is discarded. - In multi_wait() and multi_waitfds(), collect all connection caches involved (each transfer might carry its own) into a temporary list. Let each connection cache on the list contribute sockets and POLLIN/OUT events it's connections are waiting for. - in multi_perform() collect the connection caches the same way and let them peform their maintenance. This will make another non-blocking attempt to shutdown all connections on its shutdown list. - for event based multis (multi->socket_cb set), add the sockets and their poll events via the callback. When `multi_socket()` is invoked for a socket not known by an active transfer, forward this to the multi's cache for processing. On closing a connection, remove its socket(s) via the callback. TLS connection filters MUST NOT send close nofity messages in their `do_close()` implementation. The reason is that a TLS close notify signals a success. When a connection is aborted and skips its shutdown phase, the server needs to see a missing close notify to detect something has gone wrong. A graceful shutdown of FTP's data connection is performed implicitly before regarding the upload/download as complete and continuing on the control connection. For FTP without TLS, there is just the socket close happening. But with TLS, the sent/received close notify signals that the transfer is complete and healthy. Servers like `vsftpd` verify that and reject uploads without a TLS close notify. - added test_19_* for shutdown related tests - test_19_01 and test_19_02 test for TCP RST packets which happen without a graceful shutdown and should no longer appear otherwise. - add test_19_03 for handling shutdowns by the server - add test_19_04 for handling shutdowns by curl - add test_19_05 for event based shutdowny by server - add test_30_06/07 and test_31_06/07 for shutdown checks on FTP up- and downloads. Closes #13976
show more ...
|
#
c31041b1 |
| 07-Jun-2024 |
Stefan Eissing |
connection: shutdown TLS (for FTP) better This adds connection shutdown infrastructure and first use for FTP. FTP data connections, when not encountering an error, are now shut down in a
connection: shutdown TLS (for FTP) better This adds connection shutdown infrastructure and first use for FTP. FTP data connections, when not encountering an error, are now shut down in a blocking way with a 2sec timeout. - add cfilter `Curl_cft_shutdown` callback - keep a shutdown start timestamp and timeout at connectdata - provide shutdown timeout default and member in `data->set.shutdowntimeout`. - provide methods for starting, interrogating and clearing shutdown timers - provide `Curl_conn_shutdown_blocking()` to shutdown the `sockindex` filter chain in a blocking way. Use that in FTP. - add `Curl_conn_cf_poll()` to wait for socket events during shutdown of a connection filter chain. This gets the monitoring sockets and events via the filters "adjust_pollset()" methods. This gives correct behaviour when shutting down a TLS connection through a HTTP/2 proxy. - Implement shutdown for all socket filters - for HTTP/2 and h2 proxying to send GOAWAY - for TLS backends to the best of their capabilities - for tcp socket filter to make a final, nonblocking receive to avoid unwanted RST states - add shutdown forwarding to happy eyeballers and https connect ballers when applicable. Closes #13904
show more ...
|
#
937ba94e |
| 04-Jun-2024 |
Stefan Eissing |
vtls: new io_need flags for poll handling - decouple need to recv/send from negotiation state, we need this later in shutdown handling as well - move ssl enums from urldata.h to vt
vtls: new io_need flags for poll handling - decouple need to recv/send from negotiation state, we need this later in shutdown handling as well - move ssl enums from urldata.h to vtls_int.h - implement use of `connssl->io_need` in vtls.c. and all backends Closes #13879
show more ...
|
#
e101a7a8 |
| 11-Apr-2024 |
Stefan Eissing |
multi: add multi->proto_hash, a key-value store for protocol data - add `Curl_hash_add2()` that passes a destructor function for the element added. Call element destructor instead of h
multi: add multi->proto_hash, a key-value store for protocol data - add `Curl_hash_add2()` that passes a destructor function for the element added. Call element destructor instead of hash destructor if present. - multi: add `proto_hash` for protocol related information, remove `struct multi_ssl_backend_data`. - openssl: use multi->proto_hash to keep x509 shared store - schannel: use multi->proto_hash to keep x509 shared store - vtls: remove Curl_free_multi_ssl_backend_data() and its equivalents in the TLS backends Closes #13345
show more ...
|
#
fb22459d |
| 26-Apr-2024 |
Stefan Eissing |
vtls: TLS session storage overhaul - add session with destructor callback - remove vtls `session_free` method - let `Curl_ssl_addsessionid()` take ownership of session object,
vtls: TLS session storage overhaul - add session with destructor callback - remove vtls `session_free` method - let `Curl_ssl_addsessionid()` take ownership of session object, freeing it also on failures - change tls backend use - test_17, add tests for SSL session resumption Closes #13386
show more ...
|
#
32101010 |
| 22-Mar-2024 |
Stefan Eissing |
tls: use shared init code for TCP+QUIC Closes #13172
|
#
c765b04d |
| 08-Mar-2024 |
Stefan Eissing |
TLS: start shutdown only when peer did not already close - When curl sees a TCP close from the peer, do not start a TLS shutdown. TLS shutdown is a handshake and if the peer already cl
TLS: start shutdown only when peer did not already close - When curl sees a TCP close from the peer, do not start a TLS shutdown. TLS shutdown is a handshake and if the peer already closed the connection, it is not interested in participating. Reported-by: dfdity on github Assisted-by: Jiří Bok Assisted-by: Pēteris Caune Fixes #10290 Closes #13087
show more ...
|
#
395365ad |
| 27-Nov-2023 |
Daniel Stenberg |
openssl: when a session-ID is reused, skip OCSP stapling Fixes #12399 Reported-by: Alexey Larikov Closes #12418
|
#
fa714830 |
| 17-Nov-2023 |
Stefan Eissing |
vtls/vquic, keep peer name information together - add `struct ssl_peer` to keep hostname, dispname and sni for a filter - allocate `sni` for use in VTLS backend - eliminate `Cu
vtls/vquic, keep peer name information together - add `struct ssl_peer` to keep hostname, dispname and sni for a filter - allocate `sni` for use in VTLS backend - eliminate `Curl_ssl_snihost()` and its use of the download buffer - use ssl_peer in SSL and QUIC filters Closes #12349
show more ...
|
#
36662c38 |
| 06-Nov-2023 |
Michael Kaufmann |
vtls: use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0 Some servers don't support the ALPN protocol "http/1.0" (e.g. IIS 10), avoid it and use "http/1.1" instead. This rever
vtls: use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0 Some servers don't support the ALPN protocol "http/1.0" (e.g. IIS 10), avoid it and use "http/1.1" instead. This reverts commit df856cb5c9 (#10183). Fixes #12259 Closes #12285
show more ...
|
#
bf0e278a |
| 26-Oct-2023 |
Stefan Eissing |
vtls: cleanup SSL config management - remove `Curl_ssl_get_config()`, no longer needed Closes #12204
|
#
47f5b1a3 |
| 04-Sep-2023 |
Stefan Eissing |
lib: introduce struct easy_poll_set for poll information Connection filter had a `get_select_socks()` method, inspired by the various `getsocks` functions involved during the lifetime of
lib: introduce struct easy_poll_set for poll information Connection filter had a `get_select_socks()` method, inspired by the various `getsocks` functions involved during the lifetime of a transfer. These, depending on transfer state (CONNECT/DO/DONE/ etc.), return sockets to monitor and flag if this shall be done for POLLIN and/or POLLOUT. Due to this design, sockets and flags could only be added, not removed. This led to problems in filters like HTTP/2 where flow control prohibits the sending of data until the peer increases the flow window. The general transfer loop wants to write, adds POLLOUT, the socket is writeable but no data can be written. This leads to cpu busy loops. To prevent that, HTTP/2 did set the `SEND_HOLD` flag of such a blocked transfer, so the transfer loop cedes further attempts. This works if only one such filter is involved. If a HTTP/2 transfer goes through a HTTP/2 proxy, two filters are setting/clearing this flag and may step on each other's toes. Connection filters `get_select_socks()` is replaced by `adjust_pollset()`. They get passed a `struct easy_pollset` that keeps up to `MAX_SOCKSPEREASYHANDLE` sockets and their `POLLIN|POLLOUT` flags. This struct is initialized in `multi_getsock()` by calling the various `getsocks()` implementations based on transfer state, as before. After protocol handlers/transfer loop have set the sockets and flags they want, the `easy_pollset` is *always* passed to the filters. Filters "higher" in the chain are called first, starting at the first not-yet-connection one. Each filter may add sockets and/or change flags. When all flags are removed, the socket itself is removed from the pollset. Example: * transfer wants to send, adds POLLOUT * http/2 filter has a flow control block, removes POLLOUT and adds POLLIN (it is waiting on a WINDOW_UPDATE from the server) * TLS filter is connected and changes nothing * h2-proxy filter also has a flow control block on its tunnel stream, removes POLLOUT and adds POLLIN also. * socket filter is connected and changes nothing * The resulting pollset is then mixed together with all other transfers and their pollsets, just as before. Use of `SEND_HOLD` is no longer necessary in the filters. All filters are adapted for the changed method. The handling in `multi.c` has been adjusted, but its state handling the the protocol handlers' `getsocks` method are untouched. The most affected filters are http/2, ngtcp2, quiche and h2-proxy. TLS filters needed to be adjusted for the connecting handshake read/write handling. No noticeable difference in performance was detected in local scorecard runs. Closes #11833
show more ...
|
#
78d6232f |
| 07-Aug-2023 |
Daniel Stenberg |
gskit: remove We remove support for building curl with gskit. - This is a niche TLS library, only running on some IBM systems - no regular curl contributors use this backend
gskit: remove We remove support for building curl with gskit. - This is a niche TLS library, only running on some IBM systems - no regular curl contributors use this backend - no CI builds use or verify this backend - gskit, or the curl adaption for it, lacks many modern TLS features making it an inferior solution - build breakages in this code take weeks or more to get detected - fixing gskit code is mostly done "flying blind" This removal has been advertized in DEPRECATED in Jan 2, 2023 and it has been mentioned on the curl-library mailing list. It could be brought back, this is not a ban. Given proper effort and will, gskit support is welcome back into the curl TLS backend family. Closes #11460
show more ...
|
#
7c8bae0d |
| 29-Jul-2023 |
Daniel Stenberg |
nss: remove support for this TLS library Closes #11459
|
#
3f8fc257 |
| 09-May-2023 |
Viktor Szakats |
cmake: add support for "unity" builds Aka "jumbo" or "amalgamation" builds. It means to compile all sources per target as a single C source. This is experimental. You can enable
cmake: add support for "unity" builds Aka "jumbo" or "amalgamation" builds. It means to compile all sources per target as a single C source. This is experimental. You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake. It requires CMake 3.16 or newer. It makes builds (much) faster, allows for better optimizations and tends to promote less ambiguous code. Also add a new AppVeyor CI job and convert an existing one to use "unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job. Fix related issues: - add missing include guard to `easy_lock.h`. - rename static variables and functions (and a macro) with names reused across sources, or shadowed by local variables. - add an `#undef` after use. - add a missing `#undef` before use. - move internal definitions from `ftp.h` to `ftp.c`. - `curl_memory.h` fixes to make it work when included repeatedly. - stop building/linking curlx bits twice for a static-mode curl tool. These caused doubly defined symbols in unity builds. - silence missing extern declarations compiler warning for ` _CRT_glob`. - fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`. - fix colliding static symbols in debug mode: `debugtime()` and `statename`. - rename `ssl_backend_data` structure to unique names for each TLS-backend, along with the `ssl_connect_data` struct member referencing them. This required adding casts for each access. - add workaround for missing `[P]UNICODE_STRING` types in certain Windows builds when compiling `lib/ldap.c`. To support "unity" builds, we had to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows `schannel.h` option) _globally_. This caused an indirect inclusion of Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled as well. This requires `[P]UNICODE_STRING` types, which is apperantly not defined automatically (as seen with both MSVS and mingw-w64). This patch includes `<subauth.h>` to fix it. Ref: https://github.com/curl/curl/runs/13987772013 Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c - tweak unity builds to compile `lib/memdebug.c` separately in memory trace builds to avoid PP confusion. - force-disable unity for test programs. - do not compile and link libcurl sources to libtests _twice_ when libcurl is built in static mode. KNOWN ISSUES: - running tests with unity builds may fail in cases. - some build configurations/env may not compile in unity mode. E.g.: https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250 Ref: https://github.com/libssh2/libssh2/issues/1034 Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html Ref: https://en.wikipedia.org/wiki/Unity_build Closes #11095
show more ...
|
#
4ae2d9f2 |
| 06-Apr-2023 |
Stefan Eissing |
proxy: http2 proxy tunnel implementation - currently only on debug build and when env variable CURL_PROXY_TUNNEL_H2 is present. - will ALPN negotiate with the proxy server and swit
proxy: http2 proxy tunnel implementation - currently only on debug build and when env variable CURL_PROXY_TUNNEL_H2 is present. - will ALPN negotiate with the proxy server and switch tunnel filter based on the protocol negotiated. - http/1.1 tunnel code moved into cf-h1-proxy.[ch] - http/2 tunnel code implemented in cf-h2-proxy.[ch] - tunnel start and ALPN set remains in http_proxy.c - moving all haproxy related code into cf-haproxy.[ch] VTLS changes - SSL filters rely solely on the "alpn" specification they are created with and no longer check conn->bits.tls_enable_alpn. - checks on which ALPN specification to use (or none at all) are done in vtls.c when creating the filter. Testing - added a nghttpx forward proxy to the pytest setup that speaks HTTP/2 and forwards all requests to the Apache httpd forward proxy server. - extending test coverage in test_10 cases - adding proxy tests for direct/tunnel h1/h2 use of basic auth. - adding test for http/1.1 and h2 proxy tunneling to pytest Closes #10780
show more ...
|
#
67115824 |
| 01-Feb-2023 |
Stefan Eissing |
connections: introduce http/3 happy eyeballs New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing. - filter is installed when `--http3` in the tool is used (or the equivalent CURL
connections: introduce http/3 happy eyeballs New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing. - filter is installed when `--http3` in the tool is used (or the equivalent CURLOPT_ done in the library) - starts a QUIC/HTTP/3 connect right away. Should that not succeed after 100ms (subject to change), a parallel attempt is started for HTTP/2 and HTTP/1.1 via TCP - both attempts are subject to IPv6/IPv4 eyeballing, same as happens for other connections - tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT - use a `soft` timeout at half the value. When the soft timeout expires, the HTTPS-CONNECT filter checks if the QUIC filter has received any data from the server. If not, it will start the HTTP/2 attempt. HTTP/3(ngtcp2) improvements. - setting call_data in all cfilter calls similar to http/2 and vtls filters for use in callback where no stream data is available. - returning CURLE_PARTIAL_FILE for prematurely terminated transfers - enabling pytest test_05 for h3 - shifting functionality to "connect" UDP sockets from ngtcp2 implementation into the udp socket cfilter. Because unconnected UDP sockets are weird. For example they error when adding to a pollset. HTTP/3(quiche) improvements. - fixed upload bug in quiche implementation, now passes 251 and pytest - error codes on stream RESET - improved debug logs - handling of DRAIN during connect - limiting pending event queue HTTP/2 cfilter improvements. - use LOG_CF macros for dynamic logging in debug build - fix CURLcode on RST streams to be CURLE_PARTIAL_FILE - enable pytest test_05 for h2 - fix upload pytests and improve parallel transfer performance. GOAWAY handling for ngtcp2/quiche - during connect, when the remote server refuses to accept new connections and closes immediately (so the local conn goes into DRAIN phase), the connection is torn down and a another attempt is made after a short grace period. This is the behaviour observed with nghttpx when we tell it to shut down gracefully. Tested in pytest test_03_02. TLS improvements - ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces copy of logic in all tls backends. - standardized the infof logging of offered ALPNs - ALPN negotiated: have common function for all backends that sets alpn proprty and connection related things based on the negotiated protocol (or lack thereof). - new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation. Invoke: python3 tests/tests-httpd/scorecard.py --help for usage. Improvements on gathering connect statistics and socket access. - new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters report connection statistics. This is triggered when the connection has completely connected. - new void Curl_pgrsTimeWas(..) method to report a timer update with a timestamp of when it happend. This allows for updating timers "later", e.g. a connect statistic after full connectivity has been reached. - in case of HTTP eyeballing, the previous changes will update statistics only from the filter chain that "won" the eyeballing. - new cfilter query CF_QUERY_SOCKET for retrieving the socket used by a filter chain. Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket() for convenient use of this query. - Change VTLS backend to query their sub-filters for the socket when checks during the handshake are made. HTTP/3 documentation on how https eyeballing works. TLS improvements - ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces copy of logic in all tls backends. - standardized the infof logging of offered ALPNs - ALPN negotiated: have common function for all backends that sets alpn proprty and connection related things based on the negotiated protocol (or lack thereof). Scorecard with Caddy. - configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing - tests/tests-httpd/scorecard.py now measures download speeds with caddy pytest improvements - adding Makfile to clean gen dir - adding nghttpx rundir creation on start - checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old. - catch exception when checking for caddy existance on system. Closes #10349
show more ...
|
#
9e93bd47 |
| 23-Jan-2023 |
Stefan Eissing |
vtls: Manage current easy handle in nested cfilter calls The previous implementation cleared `data` so the outer invocation lost its data, which could lead to a crash. Bug: http
vtls: Manage current easy handle in nested cfilter calls The previous implementation cleared `data` so the outer invocation lost its data, which could lead to a crash. Bug: https://github.com/curl/curl/issues/10336 Reported-by: Fujii Hironori Closes https://github.com/curl/curl/pull/10340
show more ...
|
#
f8da4f2f |
| 17-Jan-2023 |
Stefan Eissing |
vtls: fix hostname handling in filters - Copy the hostname and dispname to ssl_connect_data. Use a copy instead of referencing the `connectdata` instance since this may get free
vtls: fix hostname handling in filters - Copy the hostname and dispname to ssl_connect_data. Use a copy instead of referencing the `connectdata` instance since this may get free'ed on connection reuse. Reported-by: Stefan Talpalaru Reported-by: sergio-nsk@users.noreply.github.com Fixes https://github.com/curl/curl/issues/10273 Fixes https://github.com/curl/curl/issues/10309 Closes https://github.com/curl/curl/pull/10310
show more ...
|
#
2bc1d775 |
| 02-Jan-2023 |
Daniel Stenberg |
copyright: update all copyright lines and remove year ranges - they are mostly pointless in all major jurisdictions - many big corporations and projects already don't use them - save
copyright: update all copyright lines and remove year ranges - they are mostly pointless in all major jurisdictions - many big corporations and projects already don't use them - saves us from pointless churn - git keeps history for us - the year range is kept in COPYING checksrc is updated to allow non-year using copyright statements Closes #10205
show more ...
|
#
ae99ac6f |
| 28-Nov-2022 |
Ikko Ashimine |
vtls: fix typo in vtls_int.h paramter -> parameter Closes: #9996 Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
|