urldata: shrink *select_bits int => unsigned char

- dselect_bits
- cselect_bits

... are using less than 8 bits. Changed types and moved them towards
the end of the structs t

urldata: shrink *select_bits int => unsigned char

- dselect_bits
- cselect_bits

... are using less than 8 bits. Changed types and moved them towards
the end of the structs to fit better.

Closes #11025

show more ...

http3: expire unpaused transfers in all HTTP/3 backends

Closes #11005

h2/h3: replace `state.drain` counter with `state.dselect_bits`

- `drain` was used by http/2 and http/3 implementations to indicate
that the transfer requires send/recv independant from

h2/h3: replace `state.drain` counter with `state.dselect_bits`

- `drain` was used by http/2 and http/3 implementations to indicate
that the transfer requires send/recv independant from its socket
poll state. Intended as a counter, it was used as bool flag only.
- a similar mechanism exists on `connectdata->cselect_bits` where
specific protocols can indicate something similar, only for the
whole connection.
- `cselect_bits` are cleard in transfer.c on use and, importantly,
also set when the transfer loop expended its `maxloops` tries.
`drain` was not cleared by transfer and the http2/3 implementations
had to take care of that.
- `dselect_bits` is cleared *and* set by the transfer loop. http2/3
does no longer clear it, only set when new events happen.

This change unifies the handling of socket poll overrides, extending
`cselect_bits` by a easy handle specific value and a common treatment in
transfers.

Closes #11005

show more ...

http2: support HTTP/2 to forward proxies, non-tunneling

- with `--proxy-http2` allow h2 ALPN negotiation to
forward proxies
- applies to http: requests against a https: proxy only,

http2: support HTTP/2 to forward proxies, non-tunneling

- with `--proxy-http2` allow h2 ALPN negotiation to
forward proxies
- applies to http: requests against a https: proxy only,
as https: requests will auto-tunnel
- adding a HTTP/1 request parser in http1.c
- removed h2h3.c
- using new request parser in nghttp2 and all h3 backends
- adding test 2603 for request parser
- adding h2 proxy test cases to test_10_*

scorecard.py: request scoring accidentally always run curl
with '-v'. Removed that, expect double numbers.

labeller: added http1.* and h2-proxy sources to detection

Closes #10967

show more ...

http3: check stream_ctx more thoroughly in all backends

- callbacks and filter methods might be invoked at unexpected
times, e.g. when the transfer's stream_ctx has not been initialize

http3: check stream_ctx more thoroughly in all backends

- callbacks and filter methods might be invoked at unexpected
times, e.g. when the transfer's stream_ctx has not been initialized
yet or, more likely, has already been taken down.
- check for existance of stream_ctx in such places and return
an error or silently succeed the call.

Closes #10951

show more ...

http2: move HTTP/2 stream vars into local context

- remove NGHTTP2 members of `struct HTTP`
- add `void *h2_ctx` to `struct HTTP`
- add `void *h3_ctx` to `struct HTTP`
- separate

http2: move HTTP/2 stream vars into local context

- remove NGHTTP2 members of `struct HTTP`
- add `void *h2_ctx` to `struct HTTP`
- add `void *h3_ctx` to `struct HTTP`
- separate h2/h3 pointers are needed for eyeballing
- manage local stream_ctx in http implementations

Closes #10877

show more ...

http3: improvements across backends

- ngtcp2: using bufq for recv stream data
- internal stream_ctx instead of `struct HTTP` members
for quiche, ngtcp2 and msh3
- no more QUIC

http3: improvements across backends

- ngtcp2: using bufq for recv stream data
- internal stream_ctx instead of `struct HTTP` members
for quiche, ngtcp2 and msh3
- no more QUIC related members in `struct HTTP`
- experimental use of recvmmsg(), disabled by default
- testing on my old debian box shows no throughput improvements.
- leaving it in, but disabled, for future revisit
- vquic: common UDP receive code for ngtcp2 and quiche
- vquic: common UDP send code for ngtcp2 and quiche
- added pytest skips for known msh3 failures
- fix unit2601 to survive torture testing
- quiche: using latest `master` from quiche and enabling large download
tests, now that key change is supported
- fixing test_07_21 where retry handling of starting a stream
was faulty
- msh3: use bufq for recv buffering headers and data
- msh3: replace fprintf debug logging with LOG_CF where possible
- msh3: force QUIC expire timers on recv/send to have more than
1 request per second served

Closes #10772

show more ...

url: fix logic in connection reuse to deny reuse on "unclean" connections

- add parameter to `conn_is_alive()` cfilter method that returns
if there is input data waiting on the connect

url: fix logic in connection reuse to deny reuse on "unclean" connections

- add parameter to `conn_is_alive()` cfilter method that returns
if there is input data waiting on the connection
- refrain from re-using connnection from the cache that have
input pending
- adapt http/2 and http/3 alive checks to digest pending input
to check the connection state
- remove check_cxn method from openssl as that was just doing
what the socket filter now does.
- add tests for connection reuse with special server configs

Closes #10690

show more ...

connect: fix time_connect and time_appconnect timer statistics

- time_connect was not updated when the overall connection failed,
e.g. when SSL verification was unsuccessful, refs #106

connect: fix time_connect and time_appconnect timer statistics

- time_connect was not updated when the overall connection failed,
e.g. when SSL verification was unsuccessful, refs #10670
- rework gather those values to interrogate involved filters,
also from all eyeballing attempts, to report the maximum of
those values.
- added 3 test cases in test_06 to check reported values on
successful, partially failed and totally failed connections.

Reported-by: Master Inspire
Fixes #10670
Closes #10671

show more ...

connections: introduce http/3 happy eyeballs

New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
the equivalent CURL

connections: introduce http/3 happy eyeballs

New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
succeed after 100ms (subject to change), a parallel attempt
is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
expires, the HTTPS-CONNECT filter checks if the QUIC filter
has received any data from the server. If not, it will start
the HTTP/2 attempt.

HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
implementation into the udp socket cfilter. Because unconnected
UDP sockets are weird. For example they error when adding to a
pollset.

HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue

HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.

GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
and closes immediately (so the local conn goes into DRAIN phase), the
connection is torn down and a another attempt is made after a short grace
period.
This is the behaviour observed with nghttpx when we tell it to shut
down gracefully. Tested in pytest test_03_02.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).

- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
Invoke:
python3 tests/tests-httpd/scorecard.py --help
for usage.

Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
report connection statistics. This is triggered when the connection
has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
a timestamp of when it happend. This allows for updating timers
"later", e.g. a connect statistic after full connectivity has been
reached.
- in case of HTTP eyeballing, the previous changes will update
statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
by a filter chain.
Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
checks during the handshake are made.

HTTP/3 documentation on how https eyeballing works.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).

Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy

pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.

Closes #10349

show more ...

curl_log: for failf/infof and debug logging implementations

- new functions and macros for cfilter debugging
- set CURL_DEBUG with names of cfilters where debug logging should be

curl_log: for failf/infof and debug logging implementations

- new functions and macros for cfilter debugging
- set CURL_DEBUG with names of cfilters where debug logging should be
enabled
- use GNUC __attribute__ to enable printf format checks during compile

Closes #10271

show more ...

msh3: update to v0.6

Closes #10192

msh3: run again in its cfilter

- test 2500, single GET works
- test 2501, single POST stalls
- test 2502, multiple, sequential GETs each use a new connection since
MsH3Connecti

msh3: run again in its cfilter

- test 2500, single GET works
- test 2501, single POST stalls
- test 2502, multiple, sequential GETs each use a new connection since
MsH3ConnectionGetState(qconn) no longer reports CONNECTED after one
GET.

Closes #10204

show more ...

quic: rename vquic implementations, fix for quiche build.

- quiche in debug mode did not build, fixed.
- moved all vquic implementation files to prefix curl_* to avoid
the potentia

quic: rename vquic implementations, fix for quiche build.

- quiche in debug mode did not build, fixed.
- moved all vquic implementation files to prefix curl_* to avoid
the potential mixups between provided .h files and our own.
- quich passes test 2500 and 2502. 2501, the POST, fail with
the body being rejected. Quich bug?

Closes #10242

show more ...