#
8dd81bd5 |
| 21-Mar-2024 |
Stefan Eissing |
lib: add Curl_xfer_write_resp_hd Add method in protocol handlers to allow writing of a single, 0-terminated header line. Avoids parsing and copying these lines. Closes #13165
|
#
4e4e8af1 |
| 11-Mar-2024 |
Stefan Eissing |
lib: move 'done' parameter to SingleRequests A transfer may do several `SingleRequest`s for its success. This happens regularly for authentication, follows and retries on failed connecti
lib: move 'done' parameter to SingleRequests A transfer may do several `SingleRequest`s for its success. This happens regularly for authentication, follows and retries on failed connections. The "readwrite()" calls and functions connected to those carried a `bool *done` parameter to indicate that the current `SingleRequest` is over. This may happen before `upload_done` or `download_done` bits of `SingleRequest` are set. The problem with that is now `write_resp()` protocol handlers are invoked in places where the `bool *done` cannot be passed up to the caller. Instead of being a bool in the call chain, it needs to become a member of `SingleRequest`, reflecting its state. This removes the `bool *done` parameter and adds the `done` bit to `SingleRequest` instead. It adds `Curl_req_soft_reset()` for using a `SingleRequest` in a follow up, clearing `done` and other flags/counters. Closes #13096
show more ...
|
#
e3905de8 |
| 28-Feb-2024 |
Stefan Eissing |
lib: further send/upload handling polish - Move all the "upload_done" handling to request.c - add possibility to abort sending of a request - add `Curl_req_done_sending()` f
lib: further send/upload handling polish - Move all the "upload_done" handling to request.c - add possibility to abort sending of a request - add `Curl_req_done_sending()` for checks - transfer.c: readwrite_upload() now clean - removing data->state.ulbuf and data->req.upload_fromhere - as well as data->req.upload_present - set data->req.upload_done on having read all from the client and completely flushed the send buffer - tftp, remove setting of data->req.upload_fromhere - serves no purpose as `upload_present` is not set and the data itself is directly `sendto()` anyway - smtp, make upload EOB conversion a client reader - xfer_ulbuf addition - add xfer_ulbuf for borrowing, similar to xfer_buf - use in file upload - use in c-hyper body sending - h1-proxy, remove init of data->state.uilbuf that is never used - smb, add own send_buf instead of using data->state.ulbuf Closes #13010
show more ...
|
#
9369c30c |
| 15-Feb-2024 |
Stefan Eissing |
lib: Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to clarify when and at what level they operate - send/recv of transfer related
lib: Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to clarify when and at what level they operate - send/recv of transfer related data is now done via `Curl_xfer_send()/Curl_xfer_recv()` which no longer has socket/socketindex as parameter. It decides on the transfer setup of `conn->sockfd` and `conn->writesockfd` on which connection filter chain to operate. - send/recv on a specific connection filter chain is done via `Curl_conn_send()/Curl_conn_recv()` which get the socket index as parameter. - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming consistency - clarify that the special CURLE_AGAIN hangling to return `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is returned by all other send() variants. - fix a bug in websocket `curl_ws_recv()` that mixed up data when it arrived in more than a single chunk (to be made into a sperate PR, also) Added as documented [in CLIENT-READER.md](https://github.com/curl/curl/blob/5b1f31dfbab8aef467c419c68aa06dc738cb75d4/docs/CLIENT-READERS.md). - old `Curl_buffer_send()` completely replaced by new `Curl_req_send()` - old `Curl_fillreadbuffer()` replaced with `Curl_client_read()` - HTTP chunked uploads are now formatted in a client reader added when needed. - FTP line-end conversions are done in a client reader added when needed. - when sending requests headers, remaining buffer space is filled with body data for sending in "one go". This is independent of the request body size. Resolves #12938 as now small and large requests have the same code path. Changes done to test cases: - test513: now fails before sending request headers as this initial "client read" triggers the setup fault. Behaves now the same as in hyper build - test547, test555, test1620: fix the length check in the lib code to only fail for reads *smaller* than expected. This was a bug in the test code that never triggered in the old implementation. Closes #12969
show more ...
|
#
37551535 |
| 15-Feb-2024 |
Stefan Eissing |
lib: Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to clarify when and at what level they operate - send/recv of transfer related
lib: Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to clarify when and at what level they operate - send/recv of transfer related data is now done via `Curl_xfer_send()/Curl_xfer_recv()` which no longer has socket/socketindex as parameter. It decides on the transfer setup of `conn->sockfd` and `conn->writesockfd` on which connection filter chain to operate. - send/recv on a specific connection filter chain is done via `Curl_conn_send()/Curl_conn_recv()` which get the socket index as parameter. - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming consistency - clarify that the special CURLE_AGAIN hangling to return `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is returned by all other send() variants. - fix a bug in websocket `curl_ws_recv()` that mixed up data when it arrived in more than a single chunk The method for sending not just raw bytes, but bytes that are either "headers" or "body". The send abstraction stack, to to bottom, now is: * `Curl_req_send()`: has parameter to indicate amount of header bytes, buffers all data. * `Curl_xfer_send()`: knows on which socket index to send, returns amount of bytes sent. * `Curl_conn_send()`: called with socket index, returns amount of bytes sent. In addition there is `Curl_req_flush()` for writing out all buffered bytes. `Curl_req_send()` is active for requests without body, `Curl_buffer_send()` still being used for others. This is because the special quirks need to be addressed in future parts: * `expect-100` handling * `Curl_fillreadbuffer()` needs to add directly to the new `data->req.sendbuf` * special body handlings, like `chunked` encodings and line end conversions will be moved into something like a Client Reader. In functions of the pattern `CURLcode xxx_send(..., ssize_t *written)`, replace the `ssize_t` with a `size_t`. It makes no sense to allow for negative values as the returned `CURLcode` already specifies error conditions. This allows easier handling of lengths without casting. Closes #12964
show more ...
|
#
59298221 |
| 14-Feb-2024 |
Stefan Eissing |
lib: send rework Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify when and at what level they operate - sen
lib: send rework Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify when and at what level they operate - send/recv of transfer related data is now done via `Curl_xfer_send()/Curl_xfer_recv()` which no longer has socket/socketindex as parameter. It decides on the transfer setup of `conn->sockfd` and `conn->writesockfd` on which connection filter chain to operate. - send/recv on a specific connection filter chain is done via `Curl_conn_send()/Curl_conn_recv()` which get the socket index as parameter. - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming consistency - clarify that the special CURLE_AGAIN handling to return `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is returned by all other send() variants. SingleRequest reshuffling - move functions into request.[ch] - differentiate between reset and free - add Curl_req_done() to perform last actions - add a send `bufq` to SingleRequest for future use in keeping upload data Closes #12963
show more ...
|
#
463472a2 |
| 07-Feb-2024 |
Stefan Eissing |
lib: move client writer into own source Refactoring of the client writer that passes the data to the client/application's callback functions. - split out into own source cw-out.
lib: move client writer into own source Refactoring of the client writer that passes the data to the client/application's callback functions. - split out into own source cw-out.[ch] from sendf.c - move tempwrite and tempcount from data->state into the context of the client writer - redesign the 3 tempwrite dynbufs as a linked list of dynbufs. On paused transfers, this allows to "record" interleaved HEADER/BODY chunks to be "played back" in the same order on unpausing. - keep the overall size limit of all buffered data to DYN_PAUSE_BUFFER. On exceeding that, return CURLE_TOO_LARGE instead of CURLE_OUT_OF_MEMORY as before. - add method to be called when a transfer is DONE to allow writing of any data still buffered - when paused, record HEADER writes exactly as they come for later playback. HEADERs are documented to be written one-by-one. Closes #12898
show more ...
|
#
d7b6ce64 |
| 01-Dec-2023 |
Stefan Eissing |
lib: replace readwrite with write_resp This clarifies the handling of server responses by folding the code for the complicated protocols into their protocol handlers. This concerns m
lib: replace readwrite with write_resp This clarifies the handling of server responses by folding the code for the complicated protocols into their protocol handlers. This concerns mainly HTTP and its bastard sibling RTSP. The terms "read" and "write" are often used without clear context if they refer to the connect or the client/application side of a transfer. This PR uses "read/write" for operations on the client side and "send/receive" for the connection, e.g. server side. If this is considered useful, we can revisit renaming of further methods in another PR. Curl's protocol handler `readwrite()` method been changed: ```diff - CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn, - const char *buf, size_t blen, - size_t *pconsumed, bool *readmore); + CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t blen, + bool is_eos, bool *done); ``` The name was changed to clarify that this writes reponse data to the client side. The parameter changes are: * `conn` removed as it always operates on `data->conn` * `pconsumed` removed as the method needs to handle all data on success * `readmore` removed as no longer necessary * `is_eos` as indicator that this is the last call for the transfer response (end-of-stream). * `done` TRUE on return iff the transfer response is to be treated as finished This change affects many files only because of updated comments in handlers that provide no implementation. The real change is that the HTTP protocol handlers now provide an implementation. The HTTP protocol handlers `write_resp()` implementation will get passed **all** raw data of a server response for the transfer. The HTTP/1.x formatted status and headers, as well as the undecoded response body. `Curl_http_write_resp_hds()` is used internally to parse the response headers and pass them on. This method is public as the RTSP protocol handler also uses it. HTTP/1.1 "chunked" transport encoding is now part of the general *content encoding* writer stack, just like other encodings. A new flag `CLIENTWRITE_EOS` was added for the last client write. This allows writers to verify that they are in a valid end state. The chunked decoder will check if it indeed has seen the last chunk. The general response handling in `transfer.c:466` happens in function `readwrite_data()`. This mainly operates now like: ``` static CURLcode readwrite_data(data, ...) { do { Curl_xfer_recv_resp(data, buf) ... Curl_xfer_write_resp(data, buf) ... } while(interested); ... } ``` All the response data handling is implemented in `Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()` implementation if available, or does the default behaviour. All raw response data needs to pass through this function. Which also means that anyone in possession of such data may call `Curl_xfer_write_resp()`. Closes #12480
show more ...
|
#
a0f94800 |
| 05-Jan-2024 |
Stefan Eissing |
transfer: adjust_pollset improvements - let `multi_getsock()` initialize the pollset in what the transfer state requires in regards to SEND/RECV - change connection filters `adjust
transfer: adjust_pollset improvements - let `multi_getsock()` initialize the pollset in what the transfer state requires in regards to SEND/RECV - change connection filters `adjust_pollset()` implementation to react on the presence of POLLIN/-OUT in the pollset and no longer check CURL_WANT_SEND/CURL_WANT_RECV - cf-socket will no longer add POLLIN on its own - http2 and http/3 filters will only do adjustments if the passed pollset wants to POLLIN/OUT for the transfer on the socket. This is similar to the HTTP/2 proxy filter and works in stacked filters. Closes #12640
show more ...
|
#
8706b680 |
| 13-Dec-2023 |
Stefan Eissing |
lib: eliminate `conn->cselect_bits` - use `data->state.dselect_bits` everywhere instead - remove `bool *comeback` parameter as non-zero `data->state.dselect_bits` will indicate tha
lib: eliminate `conn->cselect_bits` - use `data->state.dselect_bits` everywhere instead - remove `bool *comeback` parameter as non-zero `data->state.dselect_bits` will indicate that IO is incomplete. Closes #12512
show more ...
|
#
2bc1d775 |
| 02-Jan-2023 |
Daniel Stenberg |
copyright: update all copyright lines and remove year ranges - they are mostly pointless in all major jurisdictions - many big corporations and projects already don't use them - save
copyright: update all copyright lines and remove year ranges - they are mostly pointless in all major jurisdictions - many big corporations and projects already don't use them - saves us from pointless churn - git keeps history for us - the year range is kept in COPYING checksrc is updated to allow non-year using copyright statements Closes #10205
show more ...
|
#
1b397318 |
| 22-Nov-2022 |
Daniel Stenberg |
lib: rewind BEFORE request instead of AFTER previous This makes a big difference for cases when the rewind is not actually necessary to perofm (for example HTTP response code 301 convert
lib: rewind BEFORE request instead of AFTER previous This makes a big difference for cases when the rewind is not actually necessary to perofm (for example HTTP response code 301 converts to GET) and therefore the rewind can be avoided. In particular for situations when that rewind fails, for example when reading from a pipe or similar. Reported-by: Ali Utku Selen Fixes #9735 Closes #9958
show more ...
|
#
ad9bc597 |
| 17-May-2022 |
max.mehl |
copyright: make repository REUSE compliant Add licensing and copyright information for all files in this repository. This either happens in the file itself as a comment header or in the
copyright: make repository REUSE compliant Add licensing and copyright information for all files in this repository. This either happens in the file itself as a comment header or in the file `.reuse/dep5`. This commit also adds a Github workflow to check pull requests and adapts copyright.pl to the changes. Closes #8869
show more ...
|
#
9bc3cebc |
| 08-Feb-2022 |
HenrikHolst |
misc: remove strlen for Curl_checkheaders + Curl_checkProxyheaders Closes #8409
|
Revision tags: curl-7_76_1, curl-7_76_0, curl-7_75_0 |
|
#
219d9f86 |
| 19-Jan-2021 |
Daniel Stenberg |
transfer: remove conn->data use Closes #6486
|
#
215db086 |
| 08-Jan-2021 |
Daniel Stenberg |
lib: pass in 'struct Curl_easy *' to most functions ... in most cases instead of 'struct connectdata *' but in some cases in addition to. - We mostly operate on transfers and no
lib: pass in 'struct Curl_easy *' to most functions ... in most cases instead of 'struct connectdata *' but in some cases in addition to. - We mostly operate on transfers and not connections. - We need the transfer handle to log, store data and more. Everything in libcurl is driven by a transfer (the CURL * in the public API). - This work clarifies and separates the transfers from the connections better. - We should avoid "conn->data". Since individual connections can be used by many transfers when multiplexing, making sure that conn->data points to the current and correct transfer at all times is difficult and has been notoriously error-prone over the years. The goal is to ultimately remove the conn->data pointer for this reason. Closes #6425
show more ...
|
#
648712ee |
| 01-Jan-2021 |
Daniel Stenberg |
httpauth: make multi-request auth work with custom port When doing HTTP authentication and a port number set with CURLOPT_PORT, the code would previously have the URL's port number overr
httpauth: make multi-request auth work with custom port When doing HTTP authentication and a port number set with CURLOPT_PORT, the code would previously have the URL's port number override as if it had been a redirect to an absolute URL. Added test 1568 to verify. Reported-by: UrsusArctos on github Fixes #6397 Closes #6400
show more ...
|
Revision tags: curl-7_74_0 |
|
#
ac0a88fd |
| 05-Nov-2020 |
Daniel Stenberg |
copyright: fix year ranges Follow-up from 4d2f8006777
|
#
4d2f8006 |
| 04-Nov-2020 |
Daniel Stenberg |
curl.se: new home Closes #6172
|
Revision tags: curl-7_73_0, tiny-curl-7_72_0, curl-7_72_0, curl-7_71_1, curl-7_71_0, curl-7_70_0, curl-7_69_1, curl-7_69_0, curl-7_68_0, curl-7_67_0, curl-7_66_0 |
|
#
a55faf33 |
| 30-Jul-2019 |
Daniel Stenberg |
cleanup: remove the 'numsocks' argument used in many places It was used (intended) to pass in the size of the 'socks' array that is also passed to these functions, but was rarely actuall
cleanup: remove the 'numsocks' argument used in many places It was used (intended) to pass in the size of the 'socks' array that is also passed to these functions, but was rarely actually checked/used and the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries that should be used instead. Closes #4169
show more ...
|
Revision tags: curl-7_65_3, curl-7_65_2 |
|
#
11d7fe11 |
| 10-Jul-2019 |
Daniel Gustafsson |
headers: Remove no longer exported functions There were a leftover few prototypes of Curl_ functions that we used to export but no longer do, this removes those prototypes and cleans up
headers: Remove no longer exported functions There were a leftover few prototypes of Curl_ functions that we used to export but no longer do, this removes those prototypes and cleans up any comments still referring to them. Curl_write32_le(), Curl_strcpy_url(), Curl_strlen_url(), Curl_up_free() Curl_concat_url(), Curl_detach_connnection(), Curl_http_setup_conn() were made static in 05b100aee247bb9bec8e9a1b0166496aa4248d1c. Curl_http_perhapsrewind() made static in 574aecee208f79d391f10d57520b3. For the remainder, I didn't trawl the Git logs hard enough to capture their exact time of deletion, but they were all gone: Curl_splayprint(), Curl_http2_send_request(), Curl_global_host_cache_dtor(), Curl_scan_cache_used(), Curl_hostcache_destroy(), Curl_second_connect(), Curl_http_auth_stage() and Curl_close_connections(). Closes #4096 Reviewed-by: Daniel Stenberg <daniel@haxx.se>
show more ...
|
#
265f7f42 |
| 24-Jun-2019 |
Daniel Stenberg |
http2: call done_sending on end of upload To make sure a HTTP/2 stream registers the end of stream. Bug #4043 made me find this problem but this fix doesn't correct the reported
http2: call done_sending on end of upload To make sure a HTTP/2 stream registers the end of stream. Bug #4043 made me find this problem but this fix doesn't correct the reported issue. Closes #4068
show more ...
|
Revision tags: curl-7_65_1, curl-7_65_0, curl-7_64_1 |
|
#
65eb65fd |
| 28-Feb-2019 |
Daniel Stenberg |
urldata: simplify bytecounters - no need to have them protocol specific - no need to set pointers to them with the Curl_setup_transfer() call - make Curl_setup_transfer() o
urldata: simplify bytecounters - no need to have them protocol specific - no need to set pointers to them with the Curl_setup_transfer() call - make Curl_setup_transfer() operate on a transfer pointer, not connection - switch some counters from long to the more proper curl_off_t type Closes #3627
show more ...
|
Revision tags: curl-7_64_0, curl-7_63_0, curl-7_62_0 |
|
#
b801b453 |
| 23-Sep-2018 |
Viktor Szakats |
whitespace fixes - replace tabs with spaces where possible - remove line ending spaces - remove double/triple newlines at EOF - fix a non-UTF-8 character - cleanup a few inde
whitespace fixes - replace tabs with spaces where possible - remove line ending spaces - remove double/triple newlines at EOF - fix a non-UTF-8 character - cleanup a few indentations/line continuations in manual examples Closes https://github.com/curl/curl/pull/3037
show more ...
|
Revision tags: curl-7_61_1 |
|
#
1a890997 |
| 31-Aug-2018 |
Daniel Stenberg |
all: s/int/size_t cleanup Assisted-by: Rikard Falkeborn Closes #2922
|