openssl quic: populate x509 store before handshake

Since OpenSSL does its own send/recv internally, we may miss the moment
to populate the x509 store right before the server response. Do

openssl quic: populate x509 store before handshake

Since OpenSSL does its own send/recv internally, we may miss the moment
to populate the x509 store right before the server response. Do it
instead before we start the handshake, at the loss of the time to set
this up.

Closes #15137

show more ...

lib: use bool/TRUE/FALSE properly

booleans should use the type 'bool' and set the value to TRUE/FALSE

non-booleans should not be 'bool' and should not set the value to
TRUE/FALS

lib: use bool/TRUE/FALSE properly

booleans should use the type 'bool' and set the value to TRUE/FALSE

non-booleans should not be 'bool' and should not set the value to
TRUE/FALSE

Closes #15123

show more ...

lib: fix unity builds with BearSSL, MSH3, Quiche, OmniOS

- fix MSH3 static symbol clash.
- fix Quiche static symbol clash.
- fix local macro clash with BearSSL header.
- fix loca

lib: fix unity builds with BearSSL, MSH3, Quiche, OmniOS

- fix MSH3 static symbol clash.
- fix Quiche static symbol clash.
- fix local macro clash with BearSSL header.
- fix local macro clash with OmniOS system header.
```
In file included from ../../lib/urldata.h:197,
from ../../lib/altsvc.c:32,
from libcurlall.c:2:
../../lib/cf-socket.h:55:25: error: expected ':', ',', ';', '}' or '__attribute__' before '.' token
55 | #define sa_addr _sa_ex_u.addr
| ^
In file included from ../../lib/urldata.h:197,
from ../../lib/altsvc.c:32,
from libcurlall.c:2:
../../lib/cf-socket.h:55:25: error: expected ':', ',', ';', '}' or '__attribute__' before '.' token
55 | #define sa_addr _sa_ex_u.addr
| ^
```
Ref: https://github.com/curl/curl/actions/runs/10738314933/job/29781644299?pr=14772#step:3:6115

Discovered while adding support for "unity" builds for autotools.

Required-by: #14922
Cherry-picked from #14815
Closes #14932

show more ...

lib/src: white space edits to comply better with code style

... as checksrc now finds and complains about these.

Closes #14921

lib, src, tests: added space around ternary expressions

Closes #14912

QUIC: on connect, keep on trying on draining server

Do not give up connect on servers that are in draining state. This might
indicate the QUIC server restarting and the UDP packet routin

QUIC: on connect, keep on trying on draining server

Do not give up connect on servers that are in draining state. This might
indicate the QUIC server restarting and the UDP packet routing still
hitting the instance shutting down.

Instead keep on connecting until the overall TIMEOUT fires.

Closes #14863

show more ...

lib: use FMT_ as prefix instead of CURL_FORMAT_

For printf format defines used internally. Makes the code slighly
easier to read.

Closes #14764

openssl quic: fix memory leak

When a OpenSSL quic connection filter is aborted early, as the
server was not responding, the ssl instances where not closed
as they should.

Fi

openssl quic: fix memory leak

When a OpenSSL quic connection filter is aborted early, as the
server was not responding, the ssl instances where not closed
as they should.

Fixes #14720
Reported-by: ralfjunker on github
Closes #14724

show more ...

cpool: rename "connection cache/conncache" to "Connection Pools/cpool"

This is a better match for what they do and the general "cpool"
var/function prefix works well.

The pool n

cpool: rename "connection cache/conncache" to "Connection Pools/cpool"

This is a better match for what they do and the general "cpool"
var/function prefix works well.

The pool now handles very long hostnames correctly.

The following changes have been made:

* 'struct connectdata', e.g. connections, keep new members
named `destination` and ' destination_len' that fully specifies
interface+port+hostname of where the connection is going to.
This is used in the pool for "bundling" of connections with
the same destination. There is no limit on the length any more.
* Locking: all locks are done inside conncache.c when calling
into the pool and released on return. This eliminates hazards
of the callers keeping track.
* 'struct connectbundle' is now internal to the pool. It is no
longer referenced by a connection.
* 'bundle->multiuse' no longer exists. HTTP/2 and 3 and TLS filters
no longer need to set it. Instead, the multi checks on leaving
MSTATE_CONNECT or MSTATE_CONNECTING if the connection is now
multiplexed and new, e.g. not conn->bits.reuse. In that case
the processing of pending handles is triggered.
* The pool's init is provided with a callback to invoke on all
connections being discarded. This allows the cleanups in
`Curl_disconnect` to run, wherever it is decided to retire
a connection.
* Several pool operations can now be fully done with one call.
Pruning dead connections, upkeep and checks on pool limits
can now directly discard connections and need no longer return
those to the caller for doing that (as we have now the callback
described above).
* Finding a connection for reuse is now done via `Curl_cpool_find()`
and the caller provides callbacks to evaluate the connection
candidates.
* The 'Curl_cpool_check_limits()' now directly uses the max values
that may be set in the transfer's multi. No need to pass them
around. Curl_multi_max_host_connections() and
Curl_multi_max_total_connections() are gone.
* Add method 'Curl_node_llist()' to get the llist a node is in.
Used in cpool to verify connection are indeed in the list (or
not in any list) as they need to.

I left the conncache.[ch] as is for now and also did not touch the
documentation. If we update that outside the feature window, we can
do this in a separate PR.

Multi-thread safety is not achieved by this PR, but since more details
on how pools operate are now "internal" it is a better starting
point to go for this in the future.

Closes #14662

show more ...

ngtcp2/osslq: remove NULL pointer dereferences

If data is NULL, then it does not make sense to pass it to failf.

Closes #14701

urldata: introduce `data->mid`, a unique identifier inside a multi

`data->id` is unique in *most* situations, but not in all. If a libcurl
application uses more than one connection cache

urldata: introduce `data->mid`, a unique identifier inside a multi

`data->id` is unique in *most* situations, but not in all. If a libcurl
application uses more than one connection cache, they will overlap. This
is a rare situations, but libcurl apps do crazy things. However, for
informative things, like tracing, `data->id` is superior, since it
assigns new ids in curl's serial curl_easy_perform() use.

Introduce `data->mid` which is a unique identifer inside one multi
instance, assigned on multi_add_handle() and cleared on
multi_remove_handle().

Use the `mid` in DoH operations and also in h2/h3 stream hashes.

Reported-by: 罗朝辉
Fixes #14414
Closes #14499

show more ...

http2+h3 filters: fix ctx init

Members of the filter context, like stream hash and buffers, need to be
initialized early and protected by a flag to also avoid double cleanup.

Th

http2+h3 filters: fix ctx init

Members of the filter context, like stream hash and buffers, need to be
initialized early and protected by a flag to also avoid double cleanup.

This allow the context to be used safely before a connect() is started
and the other parts of the context are set up.

Closes #14505

show more ...

llist: remove direct struct accesses, use only functions

- Turned them all into functions to also do asserts etc.

- The llist related structs got all their fields renamed in order t

llist: remove direct struct accesses, use only functions

- Turned them all into functions to also do asserts etc.

- The llist related structs got all their fields renamed in order to make
sure no existing code remains using direct access.

- Each list node struct now points back to the list it "lives in", so
Curl_node_remove() no longer needs the list pointer.

- Rename the node struct and some of the access functions.

- Added lots of ASSERTs to verify API being used correctly

- Fix some cases of API misuse

Add docs/LLIST.md documenting the internal linked list API.

Closes #14485

show more ...

multi: make the "general" list of easy handles a Curl_llist

Instead of having an especially "unique" linked list handler for the
main list of easy handles within the multi handle, this n

multi: make the "general" list of easy handles a Curl_llist

Instead of having an especially "unique" linked list handler for the
main list of easy handles within the multi handle, this now uses a
regular Curl_llist for this as well.

With this change, it is also clearer that every easy handle added to a
multi handle belongs to one and only one out of three different lists:

process - the general one for normal transfer processing

pending - queued up waiting to get a connection (MSTATE_PENDING)

msgsent - transfer completed (MSTATE_MSGSENT)

An easy handle must therefore be removed from the current list before it
gets added to another.

Closes #14474

show more ...

lib: add eos flag to send methods

Adds a `bool eos` flag to send methods to indicate that the data
is the last chunk the invovled transfer wants to send to the server.

This will

lib: add eos flag to send methods

Adds a `bool eos` flag to send methods to indicate that the data
is the last chunk the invovled transfer wants to send to the server.

This will help protocol filters like HTTP/2 and 3 to forward the
stream's EOF flag and also allow to EAGAIN such calls when buffers
are not yet fully flushed.

Closes #14220

show more ...

Revert "lib: send eos flag"

This reverts commit be93299f10ef0b2bf7fe5c82140120073831867a.

lib: send eos flag

Adds a `bool eos` flag to send methods to indicate that the data is the
last chunk the invovled transfer wants to send to the server.

This will help protocol

lib: send eos flag

Adds a `bool eos` flag to send methods to indicate that the data is the
last chunk the invovled transfer wants to send to the server.

This will help protocol filters like HTTP/2 and 3 to forward the
stream's EOF flag and also allow to EAGAIN such calls when buffers are
not yet fully flushed.

Closes #14220

show more ...

http/3: resume upload on ack if we have more data to send

Currently we're waiting for sendbuf_len_in_flight to hit zero before
resuming upload which means we're blocking and waiting for

http/3: resume upload on ack if we have more data to send

Currently we're waiting for sendbuf_len_in_flight to hit zero before
resuming upload which means we're blocking and waiting for _all_ acks to
arrive before sending more data. This causes significant delays especially
when ack delay is used on the server side.

The fix addresses several issues in h3 over ngtcp2:
- On ack we now call nghttp3_conn_resume_stream() when we have more
data to send.
- upload_left was incorrectly computed on CF_CTRL_DATA_DONE_SEND as
we need to subtract the ammount of data we have in flight.
- Remove upload_blocked_len as we Curl_bufq_write call will do the
right thing when called from cf_ngtcp2_send.

Fixes #14198
Closes #14209

show more ...

code: language cleanup in comments

Based on the standards and guidelines we use for our documentation.

- expand contractions (they're => they are etc)
- host name = > hostname

code: language cleanup in comments

Based on the standards and guidelines we use for our documentation.

- expand contractions (they're => they are etc)
- host name = > hostname
- file name => filename
- user name = username
- man page => manpage
- run-time => runtime
- set-up => setup
- back-end => backend
- a HTTP => an HTTP
- Two spaces after a period => one space after period

Closes #14073

show more ...

http/3: add shutdown support

- openssl-quic shutdown handling
- ngtcp2 shutdown handling
- quiche shutdown handling
- add test_19_06 for verfication

Reported-by: Dexter

http/3: add shutdown support

- openssl-quic shutdown handling
- ngtcp2 shutdown handling
- quiche shutdown handling
- add test_19_06 for verfication

Reported-by: Dexter Gerig
Closes #14027
Fixes #14022

show more ...

quic: require at least OpenSSL 3.3 for QUIC

- when checking for QUIC support in OpenSSL, also check
for it being at least 3.3.0
- remove workarounds for features buggy or missing i

quic: require at least OpenSSL 3.3 for QUIC

- when checking for QUIC support in OpenSSL, also check
for it being at least 3.3.0
- remove workarounds for features buggy or missing in 3.2

Closes #14026

show more ...

http: remove "struct HTTP"

It is not actually used anymore and only contained a dummy struct field.
Remove all traces and uses of it.

Closes #13927

connection: shutdown TLS (for FTP) better

This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a

connection: shutdown TLS (for FTP) better

This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a
blocking way with a 2sec timeout.

- add cfilter `Curl_cft_shutdown` callback
- keep a shutdown start timestamp and timeout at connectdata
- provide shutdown timeout default and member in
`data->set.shutdowntimeout`.
- provide methods for starting, interrogating and clearing
shutdown timers
- provide `Curl_conn_shutdown_blocking()` to shutdown the
`sockindex` filter chain in a blocking way. Use that in FTP.
- add `Curl_conn_cf_poll()` to wait for socket events during
shutdown of a connection filter chain.
This gets the monitoring sockets and events via the filters
"adjust_pollset()" methods. This gives correct behaviour when
shutting down a TLS connection through a HTTP/2 proxy.
- Implement shutdown for all socket filters
- for HTTP/2 and h2 proxying to send GOAWAY
- for TLS backends to the best of their capabilities
- for tcp socket filter to make a final, nonblocking
receive to avoid unwanted RST states
- add shutdown forwarding to happy eyeballers and
https connect ballers when applicable.

Closes #13904

show more ...

lib/v*: tidy up types and casts

Also add a couple of negative checks.

Cherry-picked from #13489
Closes #13622

http/2, http/3: decouple stream state from easy handle

- add `Curl_hash_offt` as hashmap between a `curl_off_t` and
an object. Use this in h2+h3 connection filters to associate
`

http/2, http/3: decouple stream state from easy handle

- add `Curl_hash_offt` as hashmap between a `curl_off_t` and
an object. Use this in h2+h3 connection filters to associate
`data->id` with the internal stream state.
- changed implementations of all affected connection filters
- removed `h2_ctx*` and `h3_ctx*` from `struct HTTP` and thus
the easy handle
- solves the problem of attaching "foreign protocol" easy handles
during connection shutdown

Test 1616 verifies the new hash functions.

Closes #13204

show more ...