lib: add curl_multi_waitfds

New function call, similar to curl_multi_fdset()

Closes #13135

hyper: implement unpausing via client reader

Just a tidy up to contain 'ifdef' pollution of common
code parts with implementation specifics.

- remove the ifdef hyper unpausing i

hyper: implement unpausing via client reader

Just a tidy up to contain 'ifdef' pollution of common
code parts with implementation specifics.

- remove the ifdef hyper unpausing in easy.c
- add hyper client reader for CURL_CR_PROTOCOL phase
that implements the unpause method for calling
the hyper waker if it is set

Closes #13075

show more ...

lib: enhance client reader resume + rewind

- update client reader documentation
- client reader, add rewind capabilities
- tell creader to rewind on next start
- Curl_cli

lib: enhance client reader resume + rewind

- update client reader documentation
- client reader, add rewind capabilities
- tell creader to rewind on next start
- Curl_client_reset() will keep reader for future rewind if requested
- add Curl_client_cleanup() for freeing all resources independent of
rewinds
- add Curl_client_start() to trigger rewinds
- move rewind code from multi.c to sendf.c and make part of
"cr-in"'s implementation
- http, move the "resume_from" handling into the client readers
- the setup of a HTTP request is reshuffled to follow:
* determine method, target, auth negotiation
* install the client reader(s) for the request, including crlf
conversions and "chunked" encoding
* apply ranges to client reader
* concat request headers, upgrades, cookies, etc.
* complete request by determining Content-Length of installed
readers in combination with method
* send
- add methods for client readers to
* return the overall length they will generate (or -1 when unknown)
* return the amount of data on the CLIENT level, so that
expect-100 can decide if it want to apply itself
* set a "resume_from" offset or fail if unsupported
- struct HTTP has become largely empty now
- rename `Client_reader_*` to `Curl_creader_*`

Closes #13026

show more ...

lib: Curl_read/Curl_write clarifications

- replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to
clarify when and at what level they operate
- send/recv of transfer related

lib: Curl_read/Curl_write clarifications

- replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to
clarify when and at what level they operate
- send/recv of transfer related data is now done via
`Curl_xfer_send()/Curl_xfer_recv()` which no longer has
socket/socketindex as parameter. It decides on the transfer
setup of `conn->sockfd` and `conn->writesockfd` on which
connection filter chain to operate.
- send/recv on a specific connection filter chain is done via
`Curl_conn_send()/Curl_conn_recv()` which get the socket index
as parameter.
- rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for
naming consistency
- clarify that the special CURLE_AGAIN hangling to return
`CURLE_OK` with length 0 only applies to `Curl_xfer_send()`
and CURLE_AGAIN is returned by all other send() variants.
- fix a bug in websocket `curl_ws_recv()` that mixed up data
when it arrived in more than a single chunk (to be made
into a sperate PR, also)

Added as documented [in
CLIENT-READER.md](https://github.com/curl/curl/blob/5b1f31dfbab8aef467c419c68aa06dc738cb75d4/docs/CLIENT-READERS.md).

- old `Curl_buffer_send()` completely replaced by new `Curl_req_send()`
- old `Curl_fillreadbuffer()` replaced with `Curl_client_read()`
- HTTP chunked uploads are now formatted in a client reader added when
needed.
- FTP line-end conversions are done in a client reader added when
needed.
- when sending requests headers, remaining buffer space is filled with
body data for sending in "one go". This is independent of the request
body size. Resolves #12938 as now small and large requests have the
same code path.

Changes done to test cases:

- test513: now fails before sending request headers as this initial
"client read" triggers the setup fault. Behaves now the same as in
hyper build
- test547, test555, test1620: fix the length check in the lib code to
only fail for reads *smaller* than expected. This was a bug in the
test code that never triggered in the old implementation.

Closes #12969

show more ...

lib: replace readwrite with write_resp

This clarifies the handling of server responses by folding the code for
the complicated protocols into their protocol handlers. This concerns
m

lib: replace readwrite with write_resp

This clarifies the handling of server responses by folding the code for
the complicated protocols into their protocol handlers. This concerns
mainly HTTP and its bastard sibling RTSP.

The terms "read" and "write" are often used without clear context if
they refer to the connect or the client/application side of a
transfer. This PR uses "read/write" for operations on the client side
and "send/receive" for the connection, e.g. server side. If this is
considered useful, we can revisit renaming of further methods in another
PR.

Curl's protocol handler `readwrite()` method been changed:

```diff
- CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn,
- const char *buf, size_t blen,
- size_t *pconsumed, bool *readmore);
+ CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t blen,
+ bool is_eos, bool *done);
```

The name was changed to clarify that this writes reponse data to the
client side. The parameter changes are:

* `conn` removed as it always operates on `data->conn`
* `pconsumed` removed as the method needs to handle all data on success
* `readmore` removed as no longer necessary
* `is_eos` as indicator that this is the last call for the transfer
response (end-of-stream).
* `done` TRUE on return iff the transfer response is to be treated as
finished

This change affects many files only because of updated comments in
handlers that provide no implementation. The real change is that the
HTTP protocol handlers now provide an implementation.

The HTTP protocol handlers `write_resp()` implementation will get passed
**all** raw data of a server response for the transfer. The HTTP/1.x
formatted status and headers, as well as the undecoded response
body. `Curl_http_write_resp_hds()` is used internally to parse the
response headers and pass them on. This method is public as the RTSP
protocol handler also uses it.

HTTP/1.1 "chunked" transport encoding is now part of the general
*content encoding* writer stack, just like other encodings. A new flag
`CLIENTWRITE_EOS` was added for the last client write. This allows
writers to verify that they are in a valid end state. The chunked
decoder will check if it indeed has seen the last chunk.

The general response handling in `transfer.c:466` happens in function
`readwrite_data()`. This mainly operates now like:

```
static CURLcode readwrite_data(data, ...)
{
do {
Curl_xfer_recv_resp(data, buf)
...
Curl_xfer_write_resp(data, buf)
...
} while(interested);
...
}
```

All the response data handling is implemented in
`Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()`
implementation if available, or does the default behaviour.

All raw response data needs to pass through this function. Which also
means that anyone in possession of such data may call
`Curl_xfer_write_resp()`.

Closes #12480

show more ...

curl_multi_get_handles: get easy handles from a multi handle

Closes #11750

singleuse: add scan for use in other source codes

This should reduce false-positive to almost zero. Checks for presence in
unit tests if --unit is specified, which is intended for debug

singleuse: add scan for use in other source codes

This should reduce false-positive to almost zero. Checks for presence in
unit tests if --unit is specified, which is intended for debug builds
where unit testing is enabled.

Closes #11932

show more ...

scripts/singleuse.pl: add curl_global_trace

scripts/singleuse.pl: add more API calls

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- save

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205

show more ...

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869

show more ...

urlapi: add curl_url_strerror()

Add curl_url_strerror() to convert CURLUcode into readable string and
facilitate easier troubleshooting in programs using URL API.
Extend CURLUcode wi

urlapi: add curl_url_strerror()

Add curl_url_strerror() to convert CURLUcode into readable string and
facilitate easier troubleshooting in programs using URL API.
Extend CURLUcode with CURLU_LAST for iteration in unit tests.
Update man pages with a mention of new function.
Update example code and tests with new functionality where it fits.

Closes #7605

show more ...

scripts: invoke interpreters through /usr/bin/env

Closes #7602

scripts/singleuse: add curl_easy_option*

curl.se: new home

Closes #6172

singleuse.pl: support new API functions, fix curl_dbg_ handling

scripts: fix typos

memdebug: make debug-specific functions use curl_dbg_ prefix

To not "collide" or use up the regular curl_ name space. Also makes them
easier to detect in helper scripts.

Closes

memdebug: make debug-specific functions use curl_dbg_ prefix

To not "collide" or use up the regular curl_ name space. Also makes them
easier to detect in helper scripts.

Closes #3656

show more ...

scripts/singleuse: script to use to track single-use functions

That is functions that are declared global but are not used from outside
of the file in which it is declared. Such function

scripts/singleuse: script to use to track single-use functions

That is functions that are declared global but are not used from outside
of the file in which it is declared. Such functions should be made
static or even at times be removed.

It also verifies that all used curl_ prefixed functions are "blessed"

Closes #3538

show more ...