lib/src: white space edits to comply better with code style

... as checksrc now finds and complains about these.

Closes #14921

lib, src, tests: added space around ternary expressions

Closes #14912

openssl: certinfo errors now fail correctly

If there is a (memory) error when creating the certinfo data, the code
would previously continue which could lead to a partial/broken response

openssl: certinfo errors now fail correctly

If there is a (memory) error when creating the certinfo data, the code
would previously continue which could lead to a partial/broken response.
Now, the first error aborts and cleans up the entire thing.

A certinfo "collection" error is however still not considered an error
big enough to stop the handshake.

Bonus 1: made two functions static (and removed the Curl_ prefix) that
were not used outside of openssl.c

Bonus 2: removed the unused function Curl_ossl_set_client_cert

Closes #14780

show more ...

lib: use FMT_ as prefix instead of CURL_FORMAT_

For printf format defines used internally. Makes the code slighly
easier to read.

Closes #14764

rand: only provide weak random when needed

builds without TLS and builds using rustls

Closes #14749

cpool: rename "connection cache/conncache" to "Connection Pools/cpool"

This is a better match for what they do and the general "cpool"
var/function prefix works well.

The pool n

cpool: rename "connection cache/conncache" to "Connection Pools/cpool"

This is a better match for what they do and the general "cpool"
var/function prefix works well.

The pool now handles very long hostnames correctly.

The following changes have been made:

* 'struct connectdata', e.g. connections, keep new members
named `destination` and ' destination_len' that fully specifies
interface+port+hostname of where the connection is going to.
This is used in the pool for "bundling" of connections with
the same destination. There is no limit on the length any more.
* Locking: all locks are done inside conncache.c when calling
into the pool and released on return. This eliminates hazards
of the callers keeping track.
* 'struct connectbundle' is now internal to the pool. It is no
longer referenced by a connection.
* 'bundle->multiuse' no longer exists. HTTP/2 and 3 and TLS filters
no longer need to set it. Instead, the multi checks on leaving
MSTATE_CONNECT or MSTATE_CONNECTING if the connection is now
multiplexed and new, e.g. not conn->bits.reuse. In that case
the processing of pending handles is triggered.
* The pool's init is provided with a callback to invoke on all
connections being discarded. This allows the cleanups in
`Curl_disconnect` to run, wherever it is decided to retire
a connection.
* Several pool operations can now be fully done with one call.
Pruning dead connections, upkeep and checks on pool limits
can now directly discard connections and need no longer return
those to the caller for doing that (as we have now the callback
described above).
* Finding a connection for reuse is now done via `Curl_cpool_find()`
and the caller provides callbacks to evaluate the connection
candidates.
* The 'Curl_cpool_check_limits()' now directly uses the max values
that may be set in the transfer's multi. No need to pass them
around. Curl_multi_max_host_connections() and
Curl_multi_max_total_connections() are gone.
* Add method 'Curl_node_llist()' to get the llist a node is in.
Used in cpool to verify connection are indeed in the list (or
not in any list) as they need to.

I left the conncache.[ch] as is for now and also did not touch the
documentation. If we update that outside the feature window, we can
do this in a separate PR.

Multi-thread safety is not achieved by this PR, but since more details
on how pools operate are now "internal" it is a better starting
point to go for this in the future.

Closes #14662

show more ...

spnego_gssapi: implement TLS channel bindings for openssl

Channel Bindings are used to tie the session context to a specific TLS
channel. This is to provide additional proof of valid ide

spnego_gssapi: implement TLS channel bindings for openssl

Channel Bindings are used to tie the session context to a specific TLS
channel. This is to provide additional proof of valid identity,
mitigating authentication relay attacks.

Major web servers have the ability to require (None/Accept/Require)
GSSAPI channel binding, rendering Curl unable to connect to such
websites unless support for channel bindings is implemented.

IIS calls this feature Extended Protection (EPA), which is used in
Enterprise environments using Kerberos for authentication.

This change require krb5 >= 1.19, otherwise channel bindings won't be
forwarded through SPNEGO.

Co-Authored-By: Steffen Kieß <947515+steffen-kiess@users.noreply.github.com>
Closes #13098

show more ...

vtls: stop offering alpn http/1.1 for http2-prior-knowledge

- For HTTPS if http2-prior-knowledge is set then only offer h2 (HTTP/2)
alpn to the server for protocol negotiation.

vtls: stop offering alpn http/1.1 for http2-prior-knowledge

- For HTTPS if http2-prior-knowledge is set then only offer h2 (HTTP/2)
alpn to the server for protocol negotiation.

Prior to this change both HTTP/2 ("h2") and HTTP/1.1 ("http/1.1") were
offered for ALPN when http2-prior-knowledge was set.

CURL_HTTP_VERSION_2_PRIOR_KNOWLEDGE (tool: --http2-prior-knowledge) is
meant to send non-TLS HTTP requests HTTP/2 when it is known the server
supports them. However when HTTPS is used then it attempts to first
negotiate the connection with ALPN. In that case the user likely does
not want to offer http/1.1 to the server as an acceptable protocol.

Reported-by: kit-ty-kate@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/9963
Closes https://github.com/curl/curl/pull/14266

show more ...

lib: add eos flag to send methods

Adds a `bool eos` flag to send methods to indicate that the data
is the last chunk the invovled transfer wants to send to the server.

This will

lib: add eos flag to send methods

Adds a `bool eos` flag to send methods to indicate that the data
is the last chunk the invovled transfer wants to send to the server.

This will help protocol filters like HTTP/2 and 3 to forward the
stream's EOF flag and also allow to EAGAIN such calls when buffers
are not yet fully flushed.

Closes #14220

show more ...

vtls: init ssl peer only once

- check that `struct ssl_peer` is only initialized once
- fix vtls peer init to run only once
- check in peer init that hostname is not empty, fail othe

vtls: init ssl peer only once

- check that `struct ssl_peer` is only initialized once
- fix vtls peer init to run only once
- check in peer init that hostname is not empty, fail otherwise

Closes #14152

show more ...

x509asn1: raise size limit for x509 certification information

Raise the limit for certification information from 10 thousand to 100
thousand bytes. Certificates can be larger than 10k.

x509asn1: raise size limit for x509 certification information

Raise the limit for certification information from 10 thousand to 100
thousand bytes. Certificates can be larger than 10k.

Change the infof() debug output to add '...' at the end when the max
limit it can handle is exceeded.

Reported-by: Sergio Durigan Junior
Fixes #14352
Closes #14354

show more ...

lib: convert some debugf()s into traces

Use CURL_TRC_CF() for some useful tracing information instead of
DEBUGF().

Closes #14322

vtls: avoid forward declaration in MultiSSL builds

The MSVC compiler cannot have forward declaration with const and static
variable, causing this error:
```
curl\lib\vtls\vtls.c(

vtls: avoid forward declaration in MultiSSL builds

The MSVC compiler cannot have forward declaration with const and static
variable, causing this error:
```
curl\lib\vtls\vtls.c(417,44): warning C4132: 'Curl_ssl_multi': const object should be initialized
```

Ref: #14276
Closes #14305

show more ...

Revert "lib: send eos flag"

This reverts commit be93299f10ef0b2bf7fe5c82140120073831867a.

lib: send eos flag

Adds a `bool eos` flag to send methods to indicate that the data is the
last chunk the invovled transfer wants to send to the server.

This will help protocol

lib: send eos flag

Adds a `bool eos` flag to send methods to indicate that the data is the
last chunk the invovled transfer wants to send to the server.

This will help protocol filters like HTTP/2 and 3 to forward the
stream's EOF flag and also allow to EAGAIN such calls when buffers are
not yet fully flushed.

Closes #14220

show more ...

vtls: replace addsessionid with set_sessionid

- deduplicate the code in many tls backends that check
for an existing id and delete it before adding the new one
- rename ssl_primary

vtls: replace addsessionid with set_sessionid

- deduplicate the code in many tls backends that check
for an existing id and delete it before adding the new one
- rename ssl_primary_config's `sessionid` bool to `cache_session`

Closes #14121

show more ...

code: language cleanup in comments

Based on the standards and guidelines we use for our documentation.

- expand contractions (they're => they are etc)
- host name = > hostname

code: language cleanup in comments

Based on the standards and guidelines we use for our documentation.

- expand contractions (they're => they are etc)
- host name = > hostname
- file name => filename
- user name = username
- man page => manpage
- run-time => runtime
- set-up => setup
- back-end => backend
- a HTTP => an HTTP
- Two spaces after a period => one space after period

Closes #14073

show more ...

lib: graceful connection shutdown

When libcurl discards a connection there are two phases this may go
through: "shutdown" and "closing". If a connection is aborted, the
shutdown phas

lib: graceful connection shutdown

When libcurl discards a connection there are two phases this may go
through: "shutdown" and "closing". If a connection is aborted, the
shutdown phase is skipped and it is closed right away.

The connection filters attached to the connection implement the phases
in their `do_shutdown()` and `do_close()` callbacks. Filters carry now a
`shutdown` flags next to `connected` to keep track of the shutdown
operation.

Filters are shut down from top to bottom. If a filter is not connected,
its shutdown is skipped. Notable filters that *do* something during
shutdown are HTTP/2 and TLS. HTTP/2 sends the GOAWAY frame. TLS sends
its close notify and expects to receive a close notify from the server.

As sends and receives may EAGAIN on the network, a shutdown is often not
successful right away and needs to poll the connection's socket(s). To
facilitate this, such connections are placed on a new shutdown list
inside the connection cache.

Since managing this list requires the cooperation of a multi handle,
only the connection cache belonging to a multi handle is used. If a
connection was in another cache when being discarded, it is removed
there and added to the multi's cache. If no multi handle is available at
that time, the connection is shutdown and closed in a one-time,
best-effort attempt.

When a multi handle is destroyed, all connection still on the shutdown
list are discarded with a final shutdown attempt and close. In curl
debug builds, the environment variable `CURL_GRACEFUL_SHUTDOWN` can be
set to make this graceful with a timeout in milliseconds given by the
variable.

The shutdown list is limited to the max number of connections configured
for a multi cache. Set via CURLMOPT_MAX_TOTAL_CONNECTIONS. When the
limit is reached, the oldest connection on the shutdown list is
discarded.

- In multi_wait() and multi_waitfds(), collect all connection caches
involved (each transfer might carry its own) into a temporary list.
Let each connection cache on the list contribute sockets and
POLLIN/OUT events it's connections are waiting for.

- in multi_perform() collect the connection caches the same way and let
them peform their maintenance. This will make another non-blocking
attempt to shutdown all connections on its shutdown list.

- for event based multis (multi->socket_cb set), add the sockets and
their poll events via the callback. When `multi_socket()` is invoked
for a socket not known by an active transfer, forward this to the
multi's cache for processing. On closing a connection, remove its
socket(s) via the callback.

TLS connection filters MUST NOT send close nofity messages in their
`do_close()` implementation. The reason is that a TLS close notify
signals a success. When a connection is aborted and skips its shutdown
phase, the server needs to see a missing close notify to detect
something has gone wrong.

A graceful shutdown of FTP's data connection is performed implicitly
before regarding the upload/download as complete and continuing on the
control connection. For FTP without TLS, there is just the socket close
happening. But with TLS, the sent/received close notify signals that the
transfer is complete and healthy. Servers like `vsftpd` verify that and
reject uploads without a TLS close notify.

- added test_19_* for shutdown related tests
- test_19_01 and test_19_02 test for TCP RST packets
which happen without a graceful shutdown and should
no longer appear otherwise.
- add test_19_03 for handling shutdowns by the server
- add test_19_04 for handling shutdowns by curl
- add test_19_05 for event based shutdowny by server
- add test_30_06/07 and test_31_06/07 for shutdown checks
on FTP up- and downloads.

Closes #13976

show more ...

connection: shutdown TLS (for FTP) better

This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a

connection: shutdown TLS (for FTP) better

This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a
blocking way with a 2sec timeout.

- add cfilter `Curl_cft_shutdown` callback
- keep a shutdown start timestamp and timeout at connectdata
- provide shutdown timeout default and member in
`data->set.shutdowntimeout`.
- provide methods for starting, interrogating and clearing
shutdown timers
- provide `Curl_conn_shutdown_blocking()` to shutdown the
`sockindex` filter chain in a blocking way. Use that in FTP.
- add `Curl_conn_cf_poll()` to wait for socket events during
shutdown of a connection filter chain.
This gets the monitoring sockets and events via the filters
"adjust_pollset()" methods. This gives correct behaviour when
shutting down a TLS connection through a HTTP/2 proxy.
- Implement shutdown for all socket filters
- for HTTP/2 and h2 proxying to send GOAWAY
- for TLS backends to the best of their capabilities
- for tcp socket filter to make a final, nonblocking
receive to avoid unwanted RST states
- add shutdown forwarding to happy eyeballers and
https connect ballers when applicable.

Closes #13904

show more ...

vtls: new io_need flags for poll handling

- decouple need to recv/send from negotiation state, we need
this later in shutdown handling as well
- move ssl enums from urldata.h to vt

vtls: new io_need flags for poll handling

- decouple need to recv/send from negotiation state, we need
this later in shutdown handling as well
- move ssl enums from urldata.h to vtls_int.h
- implement use of `connssl->io_need` in vtls.c. and all backends

Closes #13879

show more ...

lib/v*: tidy up types and casts

Also add a couple of negative checks.

Cherry-picked from #13489
Closes #13622

vtls: deprioritize Secure Transport

Moved Secure Transport behind OpenSSL, so we can build CURL with both
and prefer using OpenSSL over Secure Transport by default.

Closes #13547

wolfssl: support CA caching

As a bonus, add SSLSUPP_CA_CACHE to let TLS backends signal its support
for this so that *setopt() return error if there is no support.

Closes #13786

multi: add multi->proto_hash, a key-value store for protocol data

- add `Curl_hash_add2()` that passes a destructor function for
the element added. Call element destructor instead of h

multi: add multi->proto_hash, a key-value store for protocol data

- add `Curl_hash_add2()` that passes a destructor function for
the element added. Call element destructor instead of hash
destructor if present.
- multi: add `proto_hash` for protocol related information,
remove `struct multi_ssl_backend_data`.
- openssl: use multi->proto_hash to keep x509 shared store
- schannel: use multi->proto_hash to keep x509 shared store
- vtls: remove Curl_free_multi_ssl_backend_data() and its
equivalents in the TLS backends

Closes #13345

show more ...

vtls: remove duplicate assign

Curl_ssl_peer_cleanup() already clears the ->sni field, no point in
assigning it again.

Spotted by CodeSonar

Closes #13626