#
d78e129d |
| 27-Sep-2024 |
Daniel Stenberg |
WebSockets: make support official (non-experimental) Inverts the configure/cmake options to instead provide options that disable WebSockets and have them (ws + wss) enabled by default.
WebSockets: make support official (non-experimental) Inverts the configure/cmake options to instead provide options that disable WebSockets and have them (ws + wss) enabled by default. Closes #14936
show more ...
|
#
d83b528a |
| 10-Sep-2024 |
Viktor Szakats |
tidy-up: spelling C89, Schannel, Secure Transport, contractions. Cherry-picked from #14692 Closes #14996
|
#
47d6ec98 |
| 19-Sep-2024 |
Stefan Eissing |
alt-svc: honor data->state.httpwant When a transfer is set for a speficif HTTP version, only lookup that protocol in the alt-svc mappings. When no speicific version is requested, sca
alt-svc: honor data->state.httpwant When a transfer is set for a speficif HTTP version, only lookup that protocol in the alt-svc mappings. When no speicific version is requested, scan all entries as before. Closes #14966
show more ...
|
#
433d7303 |
| 19-Sep-2024 |
Stefan Eissing |
url: connection reuse on h3 connections - When searching for existing connections, interpret the default CURL_HTTP_VERSION_2TLS as "anything goes". This will allow us to reuse HT
url: connection reuse on h3 connections - When searching for existing connections, interpret the default CURL_HTTP_VERSION_2TLS as "anything goes". This will allow us to reuse HTTP/3 connections better - add 'http/1.1' as allowed protocol identifier in Alt-Svc files - add test_02_0[345] for testing protocol selection on provided alt-svc files Fixes #14890 Reported-by: MacKenzie Closes #14966
show more ...
|
#
fbf5d507 |
| 18-Sep-2024 |
Daniel Stenberg |
lib/src: white space edits to comply better with code style ... as checksrc now finds and complains about these. Closes #14921
|
#
50166c0d |
| 13-Sep-2024 |
Stefan Eissing |
connect: store connection info when really done Output the 'Connected to...' info message when the connection has been fully established and all information is available. Due to
connect: store connection info when really done Output the 'Connected to...' info message when the connection has been fully established and all information is available. Due to our happy eyeballing, we should not emit info messages in filters, because they may be part of an eyeballing attempt and may be discarded later for another chain. Closes #14897
show more ...
|
#
4ff04615 |
| 02-Sep-2024 |
Daniel Stenberg |
lib: use FMT_ as prefix instead of CURL_FORMAT_ For printf format defines used internally. Makes the code slighly easier to read. Closes #14764
|
#
00ef6073 |
| 30-Aug-2024 |
Stefan Eissing |
url: fix connection reuse for HTTP/2 upgrades Normally, when a connection's filters have all connected, the multiplex status is determined. However, HTTP/2 Upgrade: requests will onl
url: fix connection reuse for HTTP/2 upgrades Normally, when a connection's filters have all connected, the multiplex status is determined. However, HTTP/2 Upgrade: requests will only do this when the first server response has been received. The current connection reuse mechanism does not accomodate that and when the time between connect and response is large enough, connection reuse may not happen as desired. See test case 2405 failures, such as in https://github.com/curl/curl/actions/runs/10629497461/job/29467166451 Add 'conn->bits.asks_multiplex' as indicator that a connection is still being evaluated for mulitplexing, so that new transfers may wait on this to be cleared. Closes #14739
show more ...
|
#
64ab0ace |
| 28-Aug-2024 |
Stefan Eissing |
urldata: remove crlf_conversions counter Since the introduction of client writers, we check the body length in the PROTOCOL phase and do FTP lineend conversions laster in the CONTENT
urldata: remove crlf_conversions counter Since the introduction of client writers, we check the body length in the PROTOCOL phase and do FTP lineend conversions laster in the CONTENT_DECODING phase. This means we no longer need to count the conversions for length checks. Closes #14709
show more ...
|
#
9280bbea |
| 28-Aug-2024 |
Stefan Eissing |
urldata: remove proxy_connect_closed bit The connections 'proxy_connect_closed' bit was not used any more. Remove it. Closes #14708
|
#
ea6f5c9f |
| 27-Aug-2024 |
Stefan Eissing |
connect: limit update IP info Update IP related information at the connection and the transfer in two places only: once the filter chain connects and when a transfer is added to a co
connect: limit update IP info Update IP related information at the connection and the transfer in two places only: once the filter chain connects and when a transfer is added to a connection. The latter only updates on reuse when the filters already are connected. The only user of that information before a full connect is the HAProxy filter. Add cfilter CF_QUERY_IP_INFO query to let it find the information from the filters "below". This solves two issues with the previous version: - updates where often done twice with the same info - happy eyeballing filter "forks" could overwrite each others updates before the full winner was determined. Closes #14699
show more ...
|
#
1be704e1 |
| 23-Aug-2024 |
Stefan Eissing |
cpool: rename "connection cache/conncache" to "Connection Pools/cpool" This is a better match for what they do and the general "cpool" var/function prefix works well. The pool n
cpool: rename "connection cache/conncache" to "Connection Pools/cpool" This is a better match for what they do and the general "cpool" var/function prefix works well. The pool now handles very long hostnames correctly. The following changes have been made: * 'struct connectdata', e.g. connections, keep new members named `destination` and ' destination_len' that fully specifies interface+port+hostname of where the connection is going to. This is used in the pool for "bundling" of connections with the same destination. There is no limit on the length any more. * Locking: all locks are done inside conncache.c when calling into the pool and released on return. This eliminates hazards of the callers keeping track. * 'struct connectbundle' is now internal to the pool. It is no longer referenced by a connection. * 'bundle->multiuse' no longer exists. HTTP/2 and 3 and TLS filters no longer need to set it. Instead, the multi checks on leaving MSTATE_CONNECT or MSTATE_CONNECTING if the connection is now multiplexed and new, e.g. not conn->bits.reuse. In that case the processing of pending handles is triggered. * The pool's init is provided with a callback to invoke on all connections being discarded. This allows the cleanups in `Curl_disconnect` to run, wherever it is decided to retire a connection. * Several pool operations can now be fully done with one call. Pruning dead connections, upkeep and checks on pool limits can now directly discard connections and need no longer return those to the caller for doing that (as we have now the callback described above). * Finding a connection for reuse is now done via `Curl_cpool_find()` and the caller provides callbacks to evaluate the connection candidates. * The 'Curl_cpool_check_limits()' now directly uses the max values that may be set in the transfer's multi. No need to pass them around. Curl_multi_max_host_connections() and Curl_multi_max_total_connections() are gone. * Add method 'Curl_node_llist()' to get the llist a node is in. Used in cpool to verify connection are indeed in the list (or not in any list) as they need to. I left the conncache.[ch] as is for now and also did not touch the documentation. If we update that outside the feature window, we can do this in a separate PR. Multi-thread safety is not achieved by this PR, but since more details on how pools operate are now "internal" it is a better starting point to go for this in the future. Closes #14662
show more ...
|
#
3b057d4b |
| 21-Aug-2024 |
Daniel Stenberg |
test1521: verify setting options to NULL better Previously this test allowed several error values when setting options. This made this test miss #14629. Now, errors are generall
test1521: verify setting options to NULL better Previously this test allowed several error values when setting options. This made this test miss #14629. Now, errors are generally not accepted for setopts: - numerical setopts accept CURLE_BAD_FUNCTION_ARGUMENT for funny input - the first setopt to an option accepts CURLE_NOT_BUILT_IN or CURLE_UNKNOWN_OPTION for when they are disabled/not built-in - there is an allowlist concept for some return code for some variables, managed at the top of the mk-lib1521.pl script In curl.h: remove the OBSOLETE named values from the setopt list. Closes #14634
show more ...
|
#
a58b50fc |
| 15-Aug-2024 |
Stefan Eissing |
transfer: Curl_sendrecv() and event related improvements - Renames Curl_readwrite() to Curl_sendrecv() to reflect that it is mainly about talking to the server, not reads or writes to
transfer: Curl_sendrecv() and event related improvements - Renames Curl_readwrite() to Curl_sendrecv() to reflect that it is mainly about talking to the server, not reads or writes to the client. Add a `nowp` parameter since the single caller already has this. - Curl_sendrecv() now runs all possible operations whenever it is called and either it had been polling sockets or the 'select_bits' are set. POLL_IN/POLL_OUT are not always directly related to send/recv operations. Filters like HTTP/2, QUIC or TLS may monitor reverse directions. If a transfer does not want to send (KEEP_SEND), it will not do so, as before. Same for receives. - Curl_update_timer() now checks the absolute timestamp of an expiry and the last/new timeout to determine if the application needs to stop/start/restart its timer. This fixes edge cases where updates did not happen as they should have. - improved --test-event curl_easy_perform() simulation to handle situations where no sockets are registered but a timeout is in place. - fixed bug in events_socket() that complained about removing a socket that was unknown, when indeed it had removed the socket just before, only it was the last in the list - fixed conncache's internal handle to carry the multi instance (where the cache has one) so that operations on the closure handle trigger event callbacks correctly. - fixed conncache to not POLL_REMOVE a socket twice when a conneciton was closed. Closes #14561
show more ...
|
#
22d292b3 |
| 12-Aug-2024 |
Stefan Eissing |
urldata: introduce `data->mid`, a unique identifier inside a multi `data->id` is unique in *most* situations, but not in all. If a libcurl application uses more than one connection cache
urldata: introduce `data->mid`, a unique identifier inside a multi `data->id` is unique in *most* situations, but not in all. If a libcurl application uses more than one connection cache, they will overlap. This is a rare situations, but libcurl apps do crazy things. However, for informative things, like tracing, `data->id` is superior, since it assigns new ids in curl's serial curl_easy_perform() use. Introduce `data->mid` which is a unique identifer inside one multi instance, assigned on multi_add_handle() and cleared on multi_remove_handle(). Use the `mid` in DoH operations and also in h2/h3 stream hashes. Reported-by: 罗朝辉 Fixes #14414 Closes #14499
show more ...
|
#
ba235ab2 |
| 10-Aug-2024 |
Daniel Stenberg |
llist: remove direct struct accesses, use only functions - Turned them all into functions to also do asserts etc. - The llist related structs got all their fields renamed in order t
llist: remove direct struct accesses, use only functions - Turned them all into functions to also do asserts etc. - The llist related structs got all their fields renamed in order to make sure no existing code remains using direct access. - Each list node struct now points back to the list it "lives in", so Curl_node_remove() no longer needs the list pointer. - Rename the node struct and some of the access functions. - Added lots of ASSERTs to verify API being used correctly - Fix some cases of API misuse Add docs/LLIST.md documenting the internal linked list API. Closes #14485
show more ...
|
#
f0a55181 |
| 12-Aug-2024 |
Daniel Stenberg |
urldata: remove 'scratch' from the UrlState struct It is not used anywhere anymore Follow-up to e3905de8196d67b89df160 Closes #14500
|
#
f81f351b |
| 02-Aug-2024 |
Viktor Szakats |
tidy-up: OS names Use these words and casing more consistently across text, comments and one curl tool output: AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux,
tidy-up: OS names Use these words and casing more consistently across text, comments and one curl tool output: AIX, ALPN, ANSI, BSD, Cygwin, Darwin, FreeBSD, GitHub, HP-UX, Linux, macOS, MS-DOS, MSYS, MinGW, NTLM, POSIX, Solaris, UNIX, Unix, Unicode, WINE, WebDAV, Win32, winbind, WinIDN, Windows, Windows CE, Winsock. Mostly OS names and a few more. Also a couple of other minor text fixups. Closes #14360
show more ...
|
#
5a9262a3 |
| 12-Jul-2024 |
Stefan Eissing |
url: dns_entry related improvements Replace Curl_resolv_unlock() with Curl_resolv_unlink(): -replace inuse member with refcount in Curl_dns_entry - pass Curl_dns_entry ** t
url: dns_entry related improvements Replace Curl_resolv_unlock() with Curl_resolv_unlink(): -replace inuse member with refcount in Curl_dns_entry - pass Curl_dns_entry ** to unlink, so it gets always cleared - solve potential (but unlikley) UAF in FTP's handling of looked up Curl_dns_entry. Esp. do not use addr information after unlinking an entry. In reality, the unlink will not free memory, as the dns entry is still referenced by the hostcache. But this is not safe and relying on no other code pruning the cache in the meantime. - pass permanent flag when adding a dns entry instead of fixing timestamp afterwards. url.c: fold several static *resolve_* functions into one. Closes #14195
show more ...
|
#
c074ba64 |
| 01-Jul-2024 |
Daniel Stenberg |
code: language cleanup in comments Based on the standards and guidelines we use for our documentation. - expand contractions (they're => they are etc) - host name = > hostname
code: language cleanup in comments Based on the standards and guidelines we use for our documentation. - expand contractions (they're => they are etc) - host name = > hostname - file name => filename - user name = username - man page => manpage - run-time => runtime - set-up => setup - back-end => backend - a HTTP => an HTTP - Two spaces after a period => one space after period Closes #14073
show more ...
|
#
c9b95c0b |
| 19-Jun-2024 |
Stefan Eissing |
lib: graceful connection shutdown When libcurl discards a connection there are two phases this may go through: "shutdown" and "closing". If a connection is aborted, the shutdown phas
lib: graceful connection shutdown When libcurl discards a connection there are two phases this may go through: "shutdown" and "closing". If a connection is aborted, the shutdown phase is skipped and it is closed right away. The connection filters attached to the connection implement the phases in their `do_shutdown()` and `do_close()` callbacks. Filters carry now a `shutdown` flags next to `connected` to keep track of the shutdown operation. Filters are shut down from top to bottom. If a filter is not connected, its shutdown is skipped. Notable filters that *do* something during shutdown are HTTP/2 and TLS. HTTP/2 sends the GOAWAY frame. TLS sends its close notify and expects to receive a close notify from the server. As sends and receives may EAGAIN on the network, a shutdown is often not successful right away and needs to poll the connection's socket(s). To facilitate this, such connections are placed on a new shutdown list inside the connection cache. Since managing this list requires the cooperation of a multi handle, only the connection cache belonging to a multi handle is used. If a connection was in another cache when being discarded, it is removed there and added to the multi's cache. If no multi handle is available at that time, the connection is shutdown and closed in a one-time, best-effort attempt. When a multi handle is destroyed, all connection still on the shutdown list are discarded with a final shutdown attempt and close. In curl debug builds, the environment variable `CURL_GRACEFUL_SHUTDOWN` can be set to make this graceful with a timeout in milliseconds given by the variable. The shutdown list is limited to the max number of connections configured for a multi cache. Set via CURLMOPT_MAX_TOTAL_CONNECTIONS. When the limit is reached, the oldest connection on the shutdown list is discarded. - In multi_wait() and multi_waitfds(), collect all connection caches involved (each transfer might carry its own) into a temporary list. Let each connection cache on the list contribute sockets and POLLIN/OUT events it's connections are waiting for. - in multi_perform() collect the connection caches the same way and let them peform their maintenance. This will make another non-blocking attempt to shutdown all connections on its shutdown list. - for event based multis (multi->socket_cb set), add the sockets and their poll events via the callback. When `multi_socket()` is invoked for a socket not known by an active transfer, forward this to the multi's cache for processing. On closing a connection, remove its socket(s) via the callback. TLS connection filters MUST NOT send close nofity messages in their `do_close()` implementation. The reason is that a TLS close notify signals a success. When a connection is aborted and skips its shutdown phase, the server needs to see a missing close notify to detect something has gone wrong. A graceful shutdown of FTP's data connection is performed implicitly before regarding the upload/download as complete and continuing on the control connection. For FTP without TLS, there is just the socket close happening. But with TLS, the sent/received close notify signals that the transfer is complete and healthy. Servers like `vsftpd` verify that and reject uploads without a TLS close notify. - added test_19_* for shutdown related tests - test_19_01 and test_19_02 test for TCP RST packets which happen without a graceful shutdown and should no longer appear otherwise. - add test_19_03 for handling shutdowns by the server - add test_19_04 for handling shutdowns by curl - add test_19_05 for event based shutdowny by server - add test_30_06/07 and test_31_06/07 for shutdown checks on FTP up- and downloads. Closes #13976
show more ...
|
#
b77d627d |
| 05-Jun-2024 |
Andy Pan |
tcpkeepalive: add CURLOPT_TCP_KEEPCNT and --keepalive-cnt Closes #13885
|
#
385c62aa |
| 10-Jun-2024 |
Stefan Eissing |
lib: xfer_setup and non-blocking shutdown - clarify Curl_xfer_setup() with RECV/SEND flags and different calls for which socket they operate on. Add a shutdown flag for secondary
lib: xfer_setup and non-blocking shutdown - clarify Curl_xfer_setup() with RECV/SEND flags and different calls for which socket they operate on. Add a shutdown flag for secondary sockets - change Curl_xfer_setup() calls to new functions - implement non-blocking connection shutdown at the end of receiving or sending a transfer Closes #13913
show more ...
|
#
72abf7c1 |
| 02-Jun-2024 |
Viktor Szakats |
lib: tidy up types and casts Cherry-picked from #13489 Closes #13862
|
#
b049388d |
| 04-Jun-2024 |
Daniel Stenberg |
url: allow DoH transfers to override max connection limit When reaching the set maximum limit of allowed connections, allow a new connection anyway if the transfer is created for the (in
url: allow DoH transfers to override max connection limit When reaching the set maximum limit of allowed connections, allow a new connection anyway if the transfer is created for the (internal) purpose of doing a DoH name resolve. Otherwise, unrelated "normal" transfers can starve out new DoH requests making it impossible to name resolve for new transfers. Bug: https://curl.se/mail/lib-2024-06/0001.html Reported-by: kartatz Closes #13880
show more ...
|