#
1be704e1 |
| 23-Aug-2024 |
Stefan Eissing |
cpool: rename "connection cache/conncache" to "Connection Pools/cpool" This is a better match for what they do and the general "cpool" var/function prefix works well. The pool n
cpool: rename "connection cache/conncache" to "Connection Pools/cpool" This is a better match for what they do and the general "cpool" var/function prefix works well. The pool now handles very long hostnames correctly. The following changes have been made: * 'struct connectdata', e.g. connections, keep new members named `destination` and ' destination_len' that fully specifies interface+port+hostname of where the connection is going to. This is used in the pool for "bundling" of connections with the same destination. There is no limit on the length any more. * Locking: all locks are done inside conncache.c when calling into the pool and released on return. This eliminates hazards of the callers keeping track. * 'struct connectbundle' is now internal to the pool. It is no longer referenced by a connection. * 'bundle->multiuse' no longer exists. HTTP/2 and 3 and TLS filters no longer need to set it. Instead, the multi checks on leaving MSTATE_CONNECT or MSTATE_CONNECTING if the connection is now multiplexed and new, e.g. not conn->bits.reuse. In that case the processing of pending handles is triggered. * The pool's init is provided with a callback to invoke on all connections being discarded. This allows the cleanups in `Curl_disconnect` to run, wherever it is decided to retire a connection. * Several pool operations can now be fully done with one call. Pruning dead connections, upkeep and checks on pool limits can now directly discard connections and need no longer return those to the caller for doing that (as we have now the callback described above). * Finding a connection for reuse is now done via `Curl_cpool_find()` and the caller provides callbacks to evaluate the connection candidates. * The 'Curl_cpool_check_limits()' now directly uses the max values that may be set in the transfer's multi. No need to pass them around. Curl_multi_max_host_connections() and Curl_multi_max_total_connections() are gone. * Add method 'Curl_node_llist()' to get the llist a node is in. Used in cpool to verify connection are indeed in the list (or not in any list) as they need to. I left the conncache.[ch] as is for now and also did not touch the documentation. If we update that outside the feature window, we can do this in a separate PR. Multi-thread safety is not achieved by this PR, but since more details on how pools operate are now "internal" it is a better starting point to go for this in the future. Closes #14662
show more ...
|
#
a58b50fc |
| 15-Aug-2024 |
Stefan Eissing |
transfer: Curl_sendrecv() and event related improvements - Renames Curl_readwrite() to Curl_sendrecv() to reflect that it is mainly about talking to the server, not reads or writes to
transfer: Curl_sendrecv() and event related improvements - Renames Curl_readwrite() to Curl_sendrecv() to reflect that it is mainly about talking to the server, not reads or writes to the client. Add a `nowp` parameter since the single caller already has this. - Curl_sendrecv() now runs all possible operations whenever it is called and either it had been polling sockets or the 'select_bits' are set. POLL_IN/POLL_OUT are not always directly related to send/recv operations. Filters like HTTP/2, QUIC or TLS may monitor reverse directions. If a transfer does not want to send (KEEP_SEND), it will not do so, as before. Same for receives. - Curl_update_timer() now checks the absolute timestamp of an expiry and the last/new timeout to determine if the application needs to stop/start/restart its timer. This fixes edge cases where updates did not happen as they should have. - improved --test-event curl_easy_perform() simulation to handle situations where no sockets are registered but a timeout is in place. - fixed bug in events_socket() that complained about removing a socket that was unknown, when indeed it had removed the socket just before, only it was the last in the list - fixed conncache's internal handle to carry the multi instance (where the cache has one) so that operations on the closure handle trigger event callbacks correctly. - fixed conncache to not POLL_REMOVE a socket twice when a conneciton was closed. Closes #14561
show more ...
|
#
22d292b3 |
| 12-Aug-2024 |
Stefan Eissing |
urldata: introduce `data->mid`, a unique identifier inside a multi `data->id` is unique in *most* situations, but not in all. If a libcurl application uses more than one connection cache
urldata: introduce `data->mid`, a unique identifier inside a multi `data->id` is unique in *most* situations, but not in all. If a libcurl application uses more than one connection cache, they will overlap. This is a rare situations, but libcurl apps do crazy things. However, for informative things, like tracing, `data->id` is superior, since it assigns new ids in curl's serial curl_easy_perform() use. Introduce `data->mid` which is a unique identifer inside one multi instance, assigned on multi_add_handle() and cleared on multi_remove_handle(). Use the `mid` in DoH operations and also in h2/h3 stream hashes. Reported-by: 罗朝辉 Fixes #14414 Closes #14499
show more ...
|
#
ae620a70 |
| 22-Jul-2024 |
Stefan Eissing |
conncache: connection shutdown, multi_socket handling - implement the socket hash user/reader/writer processing also for connections that are being shut down by the connection cache.
conncache: connection shutdown, multi_socket handling - implement the socket hash user/reader/writer processing also for connections that are being shut down by the connection cache. - split out handling of current vs. last pollset socket event handling into a function available in other code parts - add `shutdown_poll` pollset to `connectdata` struct so that changes in the pollset can be recorded during shutdown. (The internal handle cannot keep it since it might be used for many connections) Reported-by: calvin2021y on github Fixes #14252 Closes #14257
show more ...
|
#
c074ba64 |
| 01-Jul-2024 |
Daniel Stenberg |
code: language cleanup in comments Based on the standards and guidelines we use for our documentation. - expand contractions (they're => they are etc) - host name = > hostname
code: language cleanup in comments Based on the standards and guidelines we use for our documentation. - expand contractions (they're => they are etc) - host name = > hostname - file name => filename - user name = username - man page => manpage - run-time => runtime - set-up => setup - back-end => backend - a HTTP => an HTTP - Two spaces after a period => one space after period Closes #14073
show more ...
|
#
a35bbe89 |
| 05-May-2024 |
Viktor Szakats |
lib: bump hash sizes to `size_t` Follow-up to cc907e80a2498c0599253271a6f657f614b52a4e #13502 Cherry-picked from #13489 Closes #13601
|
#
bf567dd9 |
| 09-Jan-2024 |
Jay Satiro |
lib: use multi instead of multi_easy for the active multi - Use data->multi and not data->multi_easy to refer to the active multi. The easy handle's active multi is always data->mul
lib: use multi instead of multi_easy for the active multi - Use data->multi and not data->multi_easy to refer to the active multi. The easy handle's active multi is always data->multi. This is a follow up to 757dfdf which changed curl so that an easy handle used with the easy interface and then multi interface cannot have two different multi handles associated with it at the same time (data->multi_easy from the easy interface and data->multi from the multi interface). Closes https://github.com/curl/curl/pull/12665
show more ...
|
#
e3905de8 |
| 28-Feb-2024 |
Stefan Eissing |
lib: further send/upload handling polish - Move all the "upload_done" handling to request.c - add possibility to abort sending of a request - add `Curl_req_done_sending()` f
lib: further send/upload handling polish - Move all the "upload_done" handling to request.c - add possibility to abort sending of a request - add `Curl_req_done_sending()` for checks - transfer.c: readwrite_upload() now clean - removing data->state.ulbuf and data->req.upload_fromhere - as well as data->req.upload_present - set data->req.upload_done on having read all from the client and completely flushed the send buffer - tftp, remove setting of data->req.upload_fromhere - serves no purpose as `upload_present` is not set and the data itself is directly `sendto()` anyway - smtp, make upload EOB conversion a client reader - xfer_ulbuf addition - add xfer_ulbuf for borrowing, similar to xfer_buf - use in file upload - use in c-hyper body sending - h1-proxy, remove init of data->state.uilbuf that is never used - smb, add own send_buf instead of using data->state.ulbuf Closes #13010
show more ...
|
#
476adfea |
| 26-Jan-2024 |
Stefan Eissing |
multi: add xfer_buf to multi handle - can be borrowed by transfer during recv-write operation - needs to be released before borrowing again - adjustis size to `data->set.buffer_size`
multi: add xfer_buf to multi handle - can be borrowed by transfer during recv-write operation - needs to be released before borrowing again - adjustis size to `data->set.buffer_size` - used in transfer.c readwrite_data() Closes #12805
show more ...
|
#
d15ffe18 |
| 26-Sep-2023 |
Stefan Eissing |
multiif.h: remove Curl_multi_dump declaration Follow-up to d850eea2 which removed the Curl_multi_dump definition. Closes https://github.com/curl/curl/pull/11946
|
#
519b1cf9 |
| 04-Sep-2023 |
Stefan Eissing |
multi: more efficient pollfd count for poll - do not use separate pollfds for sockets that have POLLIN+POLLOUT Closes #11792
|
#
d4618a3f |
| 31-Jul-2023 |
Daniel Stenberg |
http2: avoid too early connection re-use/multiplexing HTTP/1 connections that are upgraded to HTTP/2 should not be picked up for reuse and multiplexing by other handles until the 101 swi
http2: avoid too early connection re-use/multiplexing HTTP/1 connections that are upgraded to HTTP/2 should not be picked up for reuse and multiplexing by other handles until the 101 switching process is completed. Lots-of-debgging-by: Stefan Eissing Reported-by: Richard W.M. Jones Bug: https://curl.se/mail/lib-2023-07/0045.html Closes #11557
show more ...
|
#
2bc1d775 |
| 02-Jan-2023 |
Daniel Stenberg |
copyright: update all copyright lines and remove year ranges - they are mostly pointless in all major jurisdictions - many big corporations and projects already don't use them - save
copyright: update all copyright lines and remove year ranges - they are mostly pointless in all major jurisdictions - many big corporations and projects already don't use them - saves us from pointless churn - git keeps history for us - the year range is kept in COPYING checksrc is updated to allow non-year using copyright statements Closes #10205
show more ...
|
#
7632c0d2 |
| 27-Aug-2022 |
Daniel Stenberg |
multi: use larger dns hash table for multi interface Have curl_multi_init() use a much larger DNS hash table than used for the easy interface to scale and perform better when used with _
multi: use larger dns hash table for multi interface Have curl_multi_init() use a much larger DNS hash table than used for the easy interface to scale and perform better when used with _many_ host names. curl_share_init() sets an in-between size. Inspired-by: Ivan Tsybulin See #9340 Closes #9376
show more ...
|
#
ad9bc597 |
| 17-May-2022 |
max.mehl |
copyright: make repository REUSE compliant Add licensing and copyright information for all files in this repository. This either happens in the file itself as a comment header or in the
copyright: make repository REUSE compliant Add licensing and copyright information for all files in this repository. This either happens in the file itself as a comment header or in the file `.reuse/dep5`. This commit also adds a Github workflow to check pull requests and adapts copyright.pl to the changes. Closes #8869
show more ...
|
#
e07a9b66 |
| 27-Apr-2022 |
Adam Rosenfield |
conn: fix typo 'connnection' -> 'connection' in two function names Closes #8759
|
#
2b3dd01b |
| 02-Dec-2021 |
Daniel Stenberg |
multi: handle errors returned from socket/timer callbacks The callbacks were partially documented to support this. Now the behavior is documented and returning error from either of these
multi: handle errors returned from socket/timer callbacks The callbacks were partially documented to support this. Now the behavior is documented and returning error from either of these callbacks will effectively kill all currently ongoing transfers. Added test 530 to verify Reported-by: Marcelo Juchem Fixes #8083 Closes #8089
show more ...
|
Revision tags: curl-7_76_1, curl-7_76_0, curl-7_75_0 |
|
#
215db086 |
| 08-Jan-2021 |
Daniel Stenberg |
lib: pass in 'struct Curl_easy *' to most functions ... in most cases instead of 'struct connectdata *' but in some cases in addition to. - We mostly operate on transfers and no
lib: pass in 'struct Curl_easy *' to most functions ... in most cases instead of 'struct connectdata *' but in some cases in addition to. - We mostly operate on transfers and not connections. - We need the transfer handle to log, store data and more. Everything in libcurl is driven by a transfer (the CURL * in the public API). - This work clarifies and separates the transfers from the connections better. - We should avoid "conn->data". Since individual connections can be used by many transfers when multiplexing, making sure that conn->data points to the current and correct transfer at all times is difficult and has been notoriously error-prone over the years. The goal is to ultimately remove the conn->data pointer for this reason. Closes #6425
show more ...
|
Revision tags: curl-7_74_0 |
|
#
4d2f8006 |
| 04-Nov-2020 |
Daniel Stenberg |
curl.se: new home Closes #6172
|
Revision tags: curl-7_73_0, tiny-curl-7_72_0, curl-7_72_0, curl-7_71_1 |
|
#
ff1c37cb |
| 23-Jun-2020 |
Daniel Stenberg |
url: make sure pushed streams get an allocated download buffer Follow-up to c4e6968127e876b0 When a new transfer is created, as a resuly of an acknowledged push, that transfer n
url: make sure pushed streams get an allocated download buffer Follow-up to c4e6968127e876b0 When a new transfer is created, as a resuly of an acknowledged push, that transfer needs a download buffer allocated. Closes #5590
show more ...
|
Revision tags: curl-7_71_0, curl-7_70_0 |
|
#
c0690271 |
| 26-Apr-2020 |
Daniel Stenberg |
conncache: various concept cleanups More connection cache accesses are protected by locks. CONNCACHE_* is a beter prefix for the connection cache lock macros. Curl_attach_c
conncache: various concept cleanups More connection cache accesses are protected by locks. CONNCACHE_* is a beter prefix for the connection cache lock macros. Curl_attach_connnection: now called as soon as there's a connection struct available and before the connection is added to the connection cache. Curl_disconnect: now assumes that the connection is already removed from the connection cache. Ref: #4915 Closes #5009
show more ...
|
Revision tags: curl-7_69_1, curl-7_69_0 |
|
#
96075328 |
| 05-Jan-2020 |
Daniel Stenberg |
ConnectionExists: respect the max_concurrent_streams limits A regression made the code use 'multiplexed' as a boolean instead of the counter it is intended to be. This made curl try to "
ConnectionExists: respect the max_concurrent_streams limits A regression made the code use 'multiplexed' as a boolean instead of the counter it is intended to be. This made curl try to "over-populate" connections with new streams. This regression came with 41fcdf71a1, shipped in curl 7.65.0. Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same check. Reported-by: Kunal Ekawde Fixes #4779 Closes #4784
show more ...
|
Revision tags: curl-7_68_0, curl-7_67_0 |
|
#
c124e6b3 |
| 24-Sep-2019 |
Kunal Ekawde |
CURLMOPT_MAX_CONCURRENT_STREAMS: new setopt Closes #4410
|
Revision tags: curl-7_66_0 |
|
#
02346abc |
| 29-Jul-2019 |
Daniel Stenberg |
curl_multi_poll: a sister to curl_multi_wait() that waits more Repeatedly we see problems where using curl_multi_wait() is difficult or just awkward because if it has no file descriptor
curl_multi_poll: a sister to curl_multi_wait() that waits more Repeatedly we see problems where using curl_multi_wait() is difficult or just awkward because if it has no file descriptor to wait for internally, it returns immediately and leaves it to the caller to wait for a small amount of time in order to avoid occasional busy-looping. This is often missed or misunderstood, leading to underperforming applications. This change introduces curl_multi_poll() as a replacement drop-in function that accepts the exact same set of arguments. This function works identically to curl_multi_wait() - EXCEPT - for the case when there's nothing to wait for internally, as then this function will by itself wait for a "suitable" short time before it returns. This effectiely avoids all risks of busy-looping and should also make it less likely that apps "over-wait". This also changes the curl tool to use this funtion internally when doing parallel transfers and changes curl_easy_perform() to use it internally. Closes #4163
show more ...
|
#
b1616dad |
| 31-Jul-2019 |
Daniel Stenberg |
timediff: make it 64 bit (if possible) even with 32 bit time_t ... to make it hold microseconds too. Fixes #4165 Closes #4168
|