lib: xfer_setup and non-blocking shutdown

- clarify Curl_xfer_setup() with RECV/SEND flags and different calls for
which socket they operate on. Add a shutdown flag for secondary

lib: xfer_setup and non-blocking shutdown

- clarify Curl_xfer_setup() with RECV/SEND flags and different calls for
which socket they operate on. Add a shutdown flag for secondary
sockets
- change Curl_xfer_setup() calls to new functions
- implement non-blocking connection shutdown at the end of receiving or
sending a transfer

Closes #13913

show more ...

lib: add Curl_xfer_write_resp_hd

Add method in protocol handlers to allow writing of a single,
0-terminated header line. Avoids parsing and copying these lines.

Closes #13165

lib: client reader polish

- seek_func/seek_client, use transfer values only
- remove copies held in `struct connectdata`, use only
ever `data->set.seek_func`
- reso

lib: client reader polish

- seek_func/seek_client, use transfer values only
- remove copies held in `struct connectdata`, use only
ever `data->set.seek_func`
- resolves possible issues in multiuse connections
- new mime post reader eliminates need to ever overwriting this

- websockets, remove empty Curl_ws_done() function

Closes #13079

show more ...

lib: send rework

Curl_read/Curl_write clarifications

- replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify
when and at what level they operate

- sen

lib: send rework

Curl_read/Curl_write clarifications

- replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify
when and at what level they operate

- send/recv of transfer related data is now done via
`Curl_xfer_send()/Curl_xfer_recv()` which no longer has
socket/socketindex as parameter. It decides on the transfer setup of
`conn->sockfd` and `conn->writesockfd` on which connection filter
chain to operate.

- send/recv on a specific connection filter chain is done via
`Curl_conn_send()/Curl_conn_recv()` which get the socket index as
parameter.

- rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming
consistency

- clarify that the special CURLE_AGAIN handling to return `CURLE_OK`
with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is
returned by all other send() variants.

SingleRequest reshuffling

- move functions into request.[ch]
- differentiate between reset and free
- add Curl_req_done() to perform last actions
- add a send `bufq` to SingleRequest for future use in keeping upload data

Closes #12963

show more ...

ssh: use stack scratch buffer for seeks

- instead of data->state.buffer

Closes #12794

lib: replace readwrite with write_resp

This clarifies the handling of server responses by folding the code for
the complicated protocols into their protocol handlers. This concerns
m

lib: replace readwrite with write_resp

This clarifies the handling of server responses by folding the code for
the complicated protocols into their protocol handlers. This concerns
mainly HTTP and its bastard sibling RTSP.

The terms "read" and "write" are often used without clear context if
they refer to the connect or the client/application side of a
transfer. This PR uses "read/write" for operations on the client side
and "send/receive" for the connection, e.g. server side. If this is
considered useful, we can revisit renaming of further methods in another
PR.

Curl's protocol handler `readwrite()` method been changed:

```diff
- CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn,
- const char *buf, size_t blen,
- size_t *pconsumed, bool *readmore);
+ CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t blen,
+ bool is_eos, bool *done);
```

The name was changed to clarify that this writes reponse data to the
client side. The parameter changes are:

* `conn` removed as it always operates on `data->conn`
* `pconsumed` removed as the method needs to handle all data on success
* `readmore` removed as no longer necessary
* `is_eos` as indicator that this is the last call for the transfer
response (end-of-stream).
* `done` TRUE on return iff the transfer response is to be treated as
finished

This change affects many files only because of updated comments in
handlers that provide no implementation. The real change is that the
HTTP protocol handlers now provide an implementation.

The HTTP protocol handlers `write_resp()` implementation will get passed
**all** raw data of a server response for the transfer. The HTTP/1.x
formatted status and headers, as well as the undecoded response
body. `Curl_http_write_resp_hds()` is used internally to parse the
response headers and pass them on. This method is public as the RTSP
protocol handler also uses it.

HTTP/1.1 "chunked" transport encoding is now part of the general
*content encoding* writer stack, just like other encodings. A new flag
`CLIENTWRITE_EOS` was added for the last client write. This allows
writers to verify that they are in a valid end state. The chunked
decoder will check if it indeed has seen the last chunk.

The general response handling in `transfer.c:466` happens in function
`readwrite_data()`. This mainly operates now like:

```
static CURLcode readwrite_data(data, ...)
{
do {
Curl_xfer_recv_resp(data, buf)
...
Curl_xfer_write_resp(data, buf)
...
} while(interested);
...
}
```

All the response data handling is implemented in
`Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()`
implementation if available, or does the default behaviour.

All raw response data needs to pass through this function. Which also
means that anyone in possession of such data may call
`Curl_xfer_write_resp()`.

Closes #12480

show more ...

lib: eliminate `conn->cselect_bits`

- use `data->state.dselect_bits` everywhere instead
- remove `bool *comeback` parameter as non-zero
`data->state.dselect_bits` will indicate tha

lib: eliminate `conn->cselect_bits`

- use `data->state.dselect_bits` everywhere instead
- remove `bool *comeback` parameter as non-zero
`data->state.dselect_bits` will indicate that IO is
incomplete.

Closes #12512

show more ...

lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding

Since the copy does not stop at a null byte, let's not call it anything
that makes you think it works like the common s

lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding

Since the copy does not stop at a null byte, let's not call it anything
that makes you think it works like the common strndup() function.

Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat

Closes #12490

show more ...

lib: strndup/memdup instead of malloc, memcpy and null-terminate

- bufref: use strndup
- cookie: use strndup
- formdata: use strndup
- ftp: use strndup
- gtls: use aprin

lib: strndup/memdup instead of malloc, memcpy and null-terminate

- bufref: use strndup
- cookie: use strndup
- formdata: use strndup
- ftp: use strndup
- gtls: use aprintf instead of malloc + strcpy * 2
- http: use strndup
- mbedtls: use strndup
- md4: use memdup
- ntlm: use memdup
- ntlm_sspi: use strndup
- pingpong: use memdup
- rtsp: use strndup instead of malloc, memcpy and null-terminate
- sectransp: use strndup
- socks_gssapi.c: use memdup
- vtls: use dynbuf instead of malloc, snprintf and memcpy
- vtls: use strdup instead of malloc + memcpy
- wolfssh: use strndup

Closes #12453

show more ...

wolfssh: remove redundant static prototypes

vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-Werror=redundant-decls]

Closes #12381

wolfssh: do cleanup in Curl_ssh_cleanup

Closes: #11921

spelling: use 'reuse' not 're-use' in code and elsewhere

Unify the spelling as both versions were previously used intermittently

Closes #11717

lib: fix some format specifiers

- Use CURL_FORMAT_CURL_OFF_T where %zd was erroneously used for some
curl_off_t variables.

- Use %zu where %zd was erroneously used for some si

lib: fix some format specifiers

- Use CURL_FORMAT_CURL_OFF_T where %zd was erroneously used for some
curl_off_t variables.

- Use %zu where %zd was erroneously used for some size_t variables.

Prior to this change some of the Windows CI tests were failing because
in Windows 32-bit targets have a 32-bit size_t and a 64-bit curl_off_t.
When %zd was used for some curl_off_t variables then only the lower
32-bits was read and the upper 32-bits would be read for part or all of
the next specifier.

Fixes https://github.com/curl/curl/issues/11327
Closes https://github.com/curl/curl/pull/11321

show more ...

checksrc: disallow spaces before labels

Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
th

checksrc: disallow spaces before labels

Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.

Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
line
* 1 was indented with the indentation level of the following line minus
three space (probably unintentional)

Co-Authored-By: Viktor Szakats

Closes #11134

show more ...

lib: unify the upload/method handling

By making sure we set state.upload based on the set.method value and not
independently as set.upload, we reduce confusion and mixup risks, both

lib: unify the upload/method handling

By making sure we set state.upload based on the set.method value and not
independently as set.upload, we reduce confusion and mixup risks, both
internally and externally.

Closes #11017

show more ...

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- save

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205

show more ...

vtls: localization of state data in filters

- almost all backend calls pass the Curl_cfilter intance instead of
connectdata+sockindex
- ssl_connect_data is remove from struct co

vtls: localization of state data in filters

- almost all backend calls pass the Curl_cfilter intance instead of
connectdata+sockindex
- ssl_connect_data is remove from struct connectdata and made internal
to vtls
- ssl_connect_data is allocated in the added filter, kept at cf->ctx

- added function to let a ssl filter access its ssl_primary_config and
ssl_config_data this selects the propert subfields in conn and data,
for filters added as plain or proxy
- adjusted all backends to use the changed api
- adjusted all backends to access config data via the exposed
functions, no longer using conn or data directly

cfilter renames for clear purpose:

- methods `Curl_conn_*(data, conn, sockindex)` work on the complete
filter chain at `sockindex` and connection `conn`.
- methods `Curl_cf_*(cf, ...)` work on a specific Curl_cfilter
instance.
- methods `Curl_conn_cf()` work on/with filter instances at a
connection.
- rebased and resolved some naming conflicts
- hostname validation (und session lookup) on SECONDARY use the same
name as on FIRST (again).

new debug macros and removing connectdata from function signatures where not
needed.

adapting schannel for new Curl_read_plain paramter.

Closes #9919

show more ...

lib: remove bad set.opt_no_body assignments

This struct field MUST remain what the application set it to, so that
handle reuse and handle duplication work.

Instead, the request

lib: remove bad set.opt_no_body assignments

This struct field MUST remain what the application set it to, so that
handle reuse and handle duplication work.

Instead, the request state bit 'no_body' is introduced for code flows
that need to change this in run-time.

Closes #9888

show more ...

lib: connection filters (cfilter) addition to curl:

- general construct/destroy in connectdata
- default implementations of callback functions
- connect: cfilters for connect and

lib: connection filters (cfilter) addition to curl:

- general construct/destroy in connectdata
- default implementations of callback functions
- connect: cfilters for connect and accept
- socks: cfilter for socks proxying
- http_proxy: cfilter for http proxy tunneling
- vtls: cfilters for primary and proxy ssl
- change in general handling of data/conn
- Curl_cfilter_setup() sets up filter chain based on data settings,
if none are installed by the protocol handler setup
- Curl_cfilter_connect() boot straps filters into `connected` status,
used by handlers and multi to reach further stages
- Curl_cfilter_is_connected() to check if a conn is connected,
e.g. all filters have done their work
- Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
indicators for multi select to work
- Curl_cfilter_data_pending() asks filters if the have incoming
data pending for recv
- Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
installed in conn->recv/conn->send for io handling
- Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
and addition/removal of a `data` from their connection
- adding vtl functions to prevent use of Curl_ssl globals directly
in other parts of the code.

Reviewed-by: Daniel Stenberg
Closes #9855

show more ...

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869

show more ...

lib: remove exclamation marks

... from infof() and failf() calls. Make them less attention seeking.

Closes #8713

lib: avoid fallthrough cases in switch statements

Commit b5a434f7f0ee4d64857f8592eced5b9007d83620 inhibits the warning
on implicit fallthrough cases, since the current coding of indicati

lib: avoid fallthrough cases in switch statements

Commit b5a434f7f0ee4d64857f8592eced5b9007d83620 inhibits the warning
on implicit fallthrough cases, since the current coding of indicating
fallthrough with comments is falling out of fashion with new compilers.
This attempts to make the issue smaller by rewriting fallthroughs to no
longer fallthrough, via either breaking the cases or turning switch
statements into if statements.

lib/content_encoding.c: the fallthrough codepath is simply copied
into the case as it's a single line.
lib/http_ntlm.c: the fallthrough case skips a state in the state-
machine and fast-forwards to NTLMSTATE_LAST. Do this before the
switch statement instead to set up the states that we actually
want.
lib/http_proxy.c: the fallthrough is just falling into exiting the
switch statement which can be done easily enough in the case.
lib/mime.c: switch statement rewritten as if statement.
lib/pop3.c: the fallthrough case skips to the next state in the
statemachine, do this explicitly instead.
lib/urlapi.c: switch statement rewritten as if statement.
lib/vssh/wolfssh.c: the fallthrough cases fast-forwards the state
machine, do this by running another iteration of the switch
statement instead.
lib/vtls/gtls.c: switch statement rewritten as if statement.
lib/vtls/nss.c: the fallthrough codepath is simply copied into the
case as it's a single line. Also twiddle a comment to not be
inside a non-brace if statement.

Closes: #7322
See-also: #7295
Reviewed-by: Daniel Stenberg <daniel@haxx.se>

show more ...

infof: remove newline from format strings, always append it

- the data needs to be "line-based" anyway since it's also passed to the
debug callback/application

- it makes info

infof: remove newline from format strings, always append it

- the data needs to be "line-based" anyway since it's also passed to the
debug callback/application

- it makes infof() work like failf() and consistency is good

- there's an assert that triggers on newlines in the format string

- Also removes a few instances of "..."

- Removes the code that would append "..." to the end of the data *iff*
it was truncated in infof()

Closes #7357

show more ...

version: turn version number functions into returning void

... as we never use the return codes from them.

Reviewed-by: Daniel Gustafsson
Closes #7319

conn: add 'attach' to protocol handler, make libssh2 use it

The libssh2 backend has SSH session associated with the connection but
the callback context is the easy handle, so when a conn

conn: add 'attach' to protocol handler, make libssh2 use it

The libssh2 backend has SSH session associated with the connection but
the callback context is the easy handle, so when a connection gets
attached to a transfer, the protocol handler now allows for a custom
function to get used to set things up correctly.

Reported-by: Michael O'Farrell
Fixes #6898
Closes #7078

show more ...