#
10879379 |
| 10-Apr-2024 |
RainRat |
misc: fix typos Closes #13344
|
#
8dd81bd5 |
| 21-Mar-2024 |
Stefan Eissing |
lib: add Curl_xfer_write_resp_hd Add method in protocol handlers to allow writing of a single, 0-terminated header line. Avoids parsing and copying these lines. Closes #13165
|
#
f46385d3 |
| 25-Mar-2024 |
Michał Antoniak <47522782+MAntoniak@users.noreply.github.com> |
urldata: remove fields not used depending on used features Reduced size of dynamically_allocated_data structure. Reduced number of stored values in enum dupstring and enum dupblob.
urldata: remove fields not used depending on used features Reduced size of dynamically_allocated_data structure. Reduced number of stored values in enum dupstring and enum dupblob. This affects the reduced array placed in the UserDefined structure. Closes #13188
show more ...
|
#
80a3b830 |
| 11-Mar-2024 |
Stefan Eissing |
http: expect 100 rework Move all handling of HTTP's `Expect: 100-continue` feature into a client reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer sending on general
http: expect 100 rework Move all handling of HTTP's `Expect: 100-continue` feature into a client reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer sending on general events like a timer. HTTP installs a `CURL_CR_PROTOCOL` reader when announcing `Expect: 100-continue`. That reader works as follows: - on first invocation, records time, starts the `EXPIRE_100_TIMEOUT` timer, disables `KEEP_SEND`, enables `KEEP_SEND_TIMER` and returns 0, eos=FALSE like a paused upload. - on subsequent invocation it checks if the timer has expired. If so, it enables `KEEP_SEND` and switches to passing through reads to the underlying readers. Transfer handling's `readwrite()` will be invoked when a timer expires (like `EXPIRE_100_TIMEOUT`) or when data from the server arrives. Seeing `KEEP_SEND_TIMER`, it will try to upload more data, which triggers reading from the client readers again. Which then may lead to a new pausing or cause the upload to start. Flags and timestamps connected to this have been moved from `SingleRequest` into the reader's context. Closes #13110
show more ...
|
#
4e4e8af1 |
| 11-Mar-2024 |
Stefan Eissing |
lib: move 'done' parameter to SingleRequests A transfer may do several `SingleRequest`s for its success. This happens regularly for authentication, follows and retries on failed connecti
lib: move 'done' parameter to SingleRequests A transfer may do several `SingleRequest`s for its success. This happens regularly for authentication, follows and retries on failed connections. The "readwrite()" calls and functions connected to those carried a `bool *done` parameter to indicate that the current `SingleRequest` is over. This may happen before `upload_done` or `download_done` bits of `SingleRequest` are set. The problem with that is now `write_resp()` protocol handlers are invoked in places where the `bool *done` cannot be passed up to the caller. Instead of being a bool in the call chain, it needs to become a member of `SingleRequest`, reflecting its state. This removes the `bool *done` parameter and adds the `done` bit to `SingleRequest` instead. It adds `Curl_req_soft_reset()` for using a `SingleRequest` in a follow up, clearing `done` and other flags/counters. Closes #13096
show more ...
|
#
0ba47146 |
| 29-Feb-2024 |
Stefan Eissing |
mime: add client reader Add `mime` client reader. Encapsulates reading from mime parts, getting their length, rewinding and unpausing. - remove special mime handling from sendf.
mime: add client reader Add `mime` client reader. Encapsulates reading from mime parts, getting their length, rewinding and unpausing. - remove special mime handling from sendf.c and easy.c - add general "unpause" method to client readers - use new reader in http/imap/smtp - make some mime functions static that are now only used internally In addition: - remove flag 'forbidchunk' as no longer needed Closes #13039
show more ...
|
#
14bcea07 |
| 29-Feb-2024 |
Stefan Eissing |
lib: enhance client reader resume + rewind - update client reader documentation - client reader, add rewind capabilities - tell creader to rewind on next start - Curl_cli
lib: enhance client reader resume + rewind - update client reader documentation - client reader, add rewind capabilities - tell creader to rewind on next start - Curl_client_reset() will keep reader for future rewind if requested - add Curl_client_cleanup() for freeing all resources independent of rewinds - add Curl_client_start() to trigger rewinds - move rewind code from multi.c to sendf.c and make part of "cr-in"'s implementation - http, move the "resume_from" handling into the client readers - the setup of a HTTP request is reshuffled to follow: * determine method, target, auth negotiation * install the client reader(s) for the request, including crlf conversions and "chunked" encoding * apply ranges to client reader * concat request headers, upgrades, cookies, etc. * complete request by determining Content-Length of installed readers in combination with method * send - add methods for client readers to * return the overall length they will generate (or -1 when unknown) * return the amount of data on the CLIENT level, so that expect-100 can decide if it want to apply itself * set a "resume_from" offset or fail if unsupported - struct HTTP has become largely empty now - rename `Client_reader_*` to `Curl_creader_*` Closes #13026
show more ...
|
#
e3905de8 |
| 28-Feb-2024 |
Stefan Eissing |
lib: further send/upload handling polish - Move all the "upload_done" handling to request.c - add possibility to abort sending of a request - add `Curl_req_done_sending()` f
lib: further send/upload handling polish - Move all the "upload_done" handling to request.c - add possibility to abort sending of a request - add `Curl_req_done_sending()` for checks - transfer.c: readwrite_upload() now clean - removing data->state.ulbuf and data->req.upload_fromhere - as well as data->req.upload_present - set data->req.upload_done on having read all from the client and completely flushed the send buffer - tftp, remove setting of data->req.upload_fromhere - serves no purpose as `upload_present` is not set and the data itself is directly `sendto()` anyway - smtp, make upload EOB conversion a client reader - xfer_ulbuf addition - add xfer_ulbuf for borrowing, similar to xfer_buf - use in file upload - use in c-hyper body sending - h1-proxy, remove init of data->state.uilbuf that is never used - smb, add own send_buf instead of using data->state.ulbuf Closes #13010
show more ...
|
#
9369c30c |
| 15-Feb-2024 |
Stefan Eissing |
lib: Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to clarify when and at what level they operate - send/recv of transfer related
lib: Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to clarify when and at what level they operate - send/recv of transfer related data is now done via `Curl_xfer_send()/Curl_xfer_recv()` which no longer has socket/socketindex as parameter. It decides on the transfer setup of `conn->sockfd` and `conn->writesockfd` on which connection filter chain to operate. - send/recv on a specific connection filter chain is done via `Curl_conn_send()/Curl_conn_recv()` which get the socket index as parameter. - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming consistency - clarify that the special CURLE_AGAIN hangling to return `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is returned by all other send() variants. - fix a bug in websocket `curl_ws_recv()` that mixed up data when it arrived in more than a single chunk (to be made into a sperate PR, also) Added as documented [in CLIENT-READER.md](https://github.com/curl/curl/blob/5b1f31dfbab8aef467c419c68aa06dc738cb75d4/docs/CLIENT-READERS.md). - old `Curl_buffer_send()` completely replaced by new `Curl_req_send()` - old `Curl_fillreadbuffer()` replaced with `Curl_client_read()` - HTTP chunked uploads are now formatted in a client reader added when needed. - FTP line-end conversions are done in a client reader added when needed. - when sending requests headers, remaining buffer space is filled with body data for sending in "one go". This is independent of the request body size. Resolves #12938 as now small and large requests have the same code path. Changes done to test cases: - test513: now fails before sending request headers as this initial "client read" triggers the setup fault. Behaves now the same as in hyper build - test547, test555, test1620: fix the length check in the lib code to only fail for reads *smaller* than expected. This was a bug in the test code that never triggered in the old implementation. Closes #12969
show more ...
|
#
37551535 |
| 15-Feb-2024 |
Stefan Eissing |
lib: Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to clarify when and at what level they operate - send/recv of transfer related
lib: Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to clarify when and at what level they operate - send/recv of transfer related data is now done via `Curl_xfer_send()/Curl_xfer_recv()` which no longer has socket/socketindex as parameter. It decides on the transfer setup of `conn->sockfd` and `conn->writesockfd` on which connection filter chain to operate. - send/recv on a specific connection filter chain is done via `Curl_conn_send()/Curl_conn_recv()` which get the socket index as parameter. - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming consistency - clarify that the special CURLE_AGAIN hangling to return `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is returned by all other send() variants. - fix a bug in websocket `curl_ws_recv()` that mixed up data when it arrived in more than a single chunk The method for sending not just raw bytes, but bytes that are either "headers" or "body". The send abstraction stack, to to bottom, now is: * `Curl_req_send()`: has parameter to indicate amount of header bytes, buffers all data. * `Curl_xfer_send()`: knows on which socket index to send, returns amount of bytes sent. * `Curl_conn_send()`: called with socket index, returns amount of bytes sent. In addition there is `Curl_req_flush()` for writing out all buffered bytes. `Curl_req_send()` is active for requests without body, `Curl_buffer_send()` still being used for others. This is because the special quirks need to be addressed in future parts: * `expect-100` handling * `Curl_fillreadbuffer()` needs to add directly to the new `data->req.sendbuf` * special body handlings, like `chunked` encodings and line end conversions will be moved into something like a Client Reader. In functions of the pattern `CURLcode xxx_send(..., ssize_t *written)`, replace the `ssize_t` with a `size_t`. It makes no sense to allow for negative values as the returned `CURLcode` already specifies error conditions. This allows easier handling of lengths without casting. Closes #12964
show more ...
|
#
59298221 |
| 14-Feb-2024 |
Stefan Eissing |
lib: send rework Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify when and at what level they operate - sen
lib: send rework Curl_read/Curl_write clarifications - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify when and at what level they operate - send/recv of transfer related data is now done via `Curl_xfer_send()/Curl_xfer_recv()` which no longer has socket/socketindex as parameter. It decides on the transfer setup of `conn->sockfd` and `conn->writesockfd` on which connection filter chain to operate. - send/recv on a specific connection filter chain is done via `Curl_conn_send()/Curl_conn_recv()` which get the socket index as parameter. - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming consistency - clarify that the special CURLE_AGAIN handling to return `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is returned by all other send() variants. SingleRequest reshuffling - move functions into request.[ch] - differentiate between reset and free - add Curl_req_done() to perform last actions - add a send `bufq` to SingleRequest for future use in keeping upload data Closes #12963
show more ...
|
#
524253dc |
| 19-Jan-2024 |
Stefan Eissing |
rtsp: Convert assertion into debug log Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934 - write excess bytes to the client where the standard excess bytes chec
rtsp: Convert assertion into debug log Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934 - write excess bytes to the client where the standard excess bytes checks will report any wrongness and fail the transfer Fixes #12738 Closes #12739
show more ...
|
#
036eb150 |
| 15-Jan-2024 |
Stefan Eissing |
rtsp: deal with borked server responses - enforce a response body length of 0, if the response has no Content-lenght. This is according to the RTSP spec. - excess bytes in a
rtsp: deal with borked server responses - enforce a response body length of 0, if the response has no Content-lenght. This is according to the RTSP spec. - excess bytes in a response body are forwarded to the client writers which will report and fail the transfer Follow-up to d7b6ce6 Fixes #12701 Closes #12706
show more ...
|
#
d7b6ce64 |
| 01-Dec-2023 |
Stefan Eissing |
lib: replace readwrite with write_resp This clarifies the handling of server responses by folding the code for the complicated protocols into their protocol handlers. This concerns m
lib: replace readwrite with write_resp This clarifies the handling of server responses by folding the code for the complicated protocols into their protocol handlers. This concerns mainly HTTP and its bastard sibling RTSP. The terms "read" and "write" are often used without clear context if they refer to the connect or the client/application side of a transfer. This PR uses "read/write" for operations on the client side and "send/receive" for the connection, e.g. server side. If this is considered useful, we can revisit renaming of further methods in another PR. Curl's protocol handler `readwrite()` method been changed: ```diff - CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn, - const char *buf, size_t blen, - size_t *pconsumed, bool *readmore); + CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t blen, + bool is_eos, bool *done); ``` The name was changed to clarify that this writes reponse data to the client side. The parameter changes are: * `conn` removed as it always operates on `data->conn` * `pconsumed` removed as the method needs to handle all data on success * `readmore` removed as no longer necessary * `is_eos` as indicator that this is the last call for the transfer response (end-of-stream). * `done` TRUE on return iff the transfer response is to be treated as finished This change affects many files only because of updated comments in handlers that provide no implementation. The real change is that the HTTP protocol handlers now provide an implementation. The HTTP protocol handlers `write_resp()` implementation will get passed **all** raw data of a server response for the transfer. The HTTP/1.x formatted status and headers, as well as the undecoded response body. `Curl_http_write_resp_hds()` is used internally to parse the response headers and pass them on. This method is public as the RTSP protocol handler also uses it. HTTP/1.1 "chunked" transport encoding is now part of the general *content encoding* writer stack, just like other encodings. A new flag `CLIENTWRITE_EOS` was added for the last client write. This allows writers to verify that they are in a valid end state. The chunked decoder will check if it indeed has seen the last chunk. The general response handling in `transfer.c:466` happens in function `readwrite_data()`. This mainly operates now like: ``` static CURLcode readwrite_data(data, ...) { do { Curl_xfer_recv_resp(data, buf) ... Curl_xfer_write_resp(data, buf) ... } while(interested); ... } ``` All the response data handling is implemented in `Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()` implementation if available, or does the default behaviour. All raw response data needs to pass through this function. Which also means that anyone in possession of such data may call `Curl_xfer_write_resp()`. Closes #12480
show more ...
|
#
7c992dd9 |
| 08-Dec-2023 |
Daniel Stenberg |
lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding Since the copy does not stop at a null byte, let's not call it anything that makes you think it works like the common s
lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding Since the copy does not stop at a null byte, let's not call it anything that makes you think it works like the common strndup() function. Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat Closes #12490
show more ...
|
#
7309b9cb |
| 05-Dec-2023 |
Daniel Stenberg |
lib: strndup/memdup instead of malloc, memcpy and null-terminate - bufref: use strndup - cookie: use strndup - formdata: use strndup - ftp: use strndup - gtls: use aprin
lib: strndup/memdup instead of malloc, memcpy and null-terminate - bufref: use strndup - cookie: use strndup - formdata: use strndup - ftp: use strndup - gtls: use aprintf instead of malloc + strcpy * 2 - http: use strndup - mbedtls: use strndup - md4: use memdup - ntlm: use memdup - ntlm_sspi: use strndup - pingpong: use memdup - rtsp: use strndup instead of malloc, memcpy and null-terminate - sectransp: use strndup - socks_gssapi.c: use memdup - vtls: use dynbuf instead of malloc, snprintf and memcpy - vtls: use strdup instead of malloc + memcpy - wolfssh: use strndup Closes #12453
show more ...
|
#
5b65e7d1 |
| 21-Nov-2023 |
Stefan Eissing |
transfer: cleanup done+excess handling - add `SingleRequest->download_done` as indicator that all download bytes have been received - remove `stop_reading` bool from readwrite func
transfer: cleanup done+excess handling - add `SingleRequest->download_done` as indicator that all download bytes have been received - remove `stop_reading` bool from readwrite functions - move excess body handling into client download writer Closes #12371
show more ...
|
#
0510e8b5 |
| 23-Nov-2023 |
Daniel Stenberg |
lib: fix comment typos Five separate ones, found by codespell Closes #12390
|
#
1cd2f007 |
| 06-Nov-2023 |
Stefan Eissing |
transfer: readwrite improvements - changed header/chunk/handler->readwrite prototypes to accept `buf`, `blen` and a `pconsumed` pointer. They now get the buffer to work on and re
transfer: readwrite improvements - changed header/chunk/handler->readwrite prototypes to accept `buf`, `blen` and a `pconsumed` pointer. They now get the buffer to work on and report back how many bytes they consumed - eliminated `k->str` in SingleRequest - improved excess data handling to properly calculate with any body data left in the headerb buffer - eliminated `k->badheader` enum to only be a bool Closes #12283
show more ...
|
#
ad051e1c |
| 23-Oct-2023 |
Stefan Eissing |
lib: client writer, part 2, accounting + logging This PR has these changes: Renaming of unencode_* to cwriter, e.g. client writers - documentation of sendf.h functions - mov
lib: client writer, part 2, accounting + logging This PR has these changes: Renaming of unencode_* to cwriter, e.g. client writers - documentation of sendf.h functions - move max decode stack checks back to content_encoding.c - define writer phase which was used as order before - introduce phases for monitoring inbetween decode phases - offering default implementations for init/write/close Add type paramter to client writer's do_write() - always pass all writes through the writer stack - writers who only care about BODY data will pass other writes unchanged add RAW and PROTOCOL client writers - RAW used for Curl_debug() logging of CURLINFO_DATA_IN - PROTOCOL used for updates to data->req.bytecount, max_filesize checks and Curl_pgrsSetDownloadCounter() - remove all updates of data->req.bytecount and calls to Curl_pgrsSetDownloadCounter() and Curl_debug() from other code - adjust test457 expected output to no longer see the excess write Closes #12184
show more ...
|
#
7eb31c85 |
| 07-Oct-2023 |
Stefan Eissing |
RTSP: improved RTP parser - fix HTTP header parsing to report incomplete lines it buffers as consumed! - re-implement the RTP parser for interleave RTP messages for robustnes
RTSP: improved RTP parser - fix HTTP header parsing to report incomplete lines it buffers as consumed! - re-implement the RTP parser for interleave RTP messages for robustness. It is now keeping its state at the connection - RTSP protocol handler "readwrite" implementation now tracks if the response is before/in/after header parsing or "in" a bod by calling "Curl_http_readwrite_headers()" itself. This allows it to know when non-RTP bytes are "junk" or HEADER or BODY. - tested with #12035 and various small receive sizes where current master fails Closes #12052
show more ...
|
#
7815647d |
| 25-Apr-2023 |
Daniel Stenberg |
lib: unify the upload/method handling By making sure we set state.upload based on the set.method value and not independently as set.upload, we reduce confusion and mixup risks, both
lib: unify the upload/method handling By making sure we set state.upload based on the set.method value and not independently as set.upload, we reduce confusion and mixup risks, both internally and externally. Closes #11017
show more ...
|
#
aa3ee0a1 |
| 13-Apr-2023 |
Daniel Stenberg |
rtsp: skip NULL assigns after Curl_safefree() ... since this is a macro that assigns NULL itself. Pointed out by PVS. Ref: #10929 Closes #10946
|
#
4399a532 |
| 04-Apr-2023 |
Daniel Stenberg |
rtsp: convert mallocs to dynbuf for RTP buffering Closes #10786
|
#
2d59eb00 |
| 30-Mar-2023 |
Daniel Stenberg |
rtsp: fix Value stored to 'skip_size' is never read Pointed out by scan-build Follow-up to 6c6306f3008f2c9b20a64 Closes #10872
|