#
49f83c30 |
| 11-Apr-2024 |
Viktor Szakats |
lib: merge `ENABLE_QUIC` C macro into `USE_HTTP3` Before this patch `lib/curl_setup.h` defined these two macros right next to each other, then the source code used them interchangeably.
lib: merge `ENABLE_QUIC` C macro into `USE_HTTP3` Before this patch `lib/curl_setup.h` defined these two macros right next to each other, then the source code used them interchangeably. After this patch, `USE_HTTP3` guards all HTTP/3 / QUIC features. (Like `USE_HTTP2` does for HTTP/2.) `ENABLE_QUIC` is no longer used. This patch doesn't change the way HTTP/3 is enabled via autotools or CMake. Builders who enabled HTTP/3 manually by defining both of these macros via `CPPFLAGS` can now delete `-DENABLE_QUIC`. Closes #13352
show more ...
|
#
fcef00db |
| 08-Mar-2024 |
Stefan Eissing |
lib: keep conn IP information together new struct ip_quadruple for holding local/remote addr+port - used in data->info and conn and cf-socket.c - copy back and forth complete st
lib: keep conn IP information together new struct ip_quadruple for holding local/remote addr+port - used in data->info and conn and cf-socket.c - copy back and forth complete struct - add 'secondary' to conn - use secondary in reporting success for ftp 2nd connection Reported-by: DasKutti on github Fixes #13084 Closes #13090
show more ...
|
#
0535f6ec |
| 18-Jan-2024 |
Stefan Eissing |
http3: initial support for OpenSSL 3.2 QUIC stack - HTTP/3 for curl using OpenSSL's own QUIC stack together with nghttp3 - configure with `--with-openssl-quic` to enable curl to
http3: initial support for OpenSSL 3.2 QUIC stack - HTTP/3 for curl using OpenSSL's own QUIC stack together with nghttp3 - configure with `--with-openssl-quic` to enable curl to build this. This requires the nghttp3 library - implementation with the following restrictions: * macOS has to use an unconnected UDP socket due to an issue in OpenSSL's datagram implementation See https://github.com/openssl/openssl/issues/23251 This makes connections to non-reponsive servers hang. * GET requests will send the indicator that they have no body in a separate QUIC packet. This may result in processing delays or Transfer-Encodings on proxied requests * uploads that encounter blocks will use 100% cpu as detection of these flow control issue is not working (we have not figured out to pry that from OpenSSL). Closes #12734
show more ...
|
#
5d044ad9 |
| 09-Jan-2024 |
Stefan Eissing |
vquic: extract TLS setup into own source - separate ngtcp2 specific parts out - provide callback during init to allow ngtcp2 to apply its defaults Closes #12678
|
#
3829759b |
| 08-Dec-2023 |
Viktor Szakats |
build: enable missing OpenSSF-recommended warnings, with fixes https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html as of 2023-11-29 [1]
build: enable missing OpenSSF-recommended warnings, with fixes https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html as of 2023-11-29 [1]. Enable new recommended warnings (except `-Wsign-conversion`): - enable `-Wformat=2` for clang (in both cmake and autotools). - add `CURL_PRINTF()` internal attribute and mark functions accepting printf arguments with it. This is a copy of existing `CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible with redefinting the `printf` symbol: https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94 - fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for mingw-w64 and enable it on this platform. - enable `-Wimplicit-fallthrough`. - enable `-Wtrampolines`. - add `-Wsign-conversion` commented with a FIXME. - cmake: enable `-pedantic-errors` the way we do it with autotools. Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747 - lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format checks. Previously it was always disabled due to the internal `printf` macro. Fix them: - fix bug where an `set_ipv6_v6only()` call was missed in builds with `--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`. - add internal `FALLTHROUGH()` macro. - replace obsolete fall-through comments with `FALLTHROUGH()`. - fix fallthrough markups: Delete redundant ones (showing up as warnings in most cases). Add missing ones. Fix indentation. - silence `-Wformat-nonliteral` warnings with llvm/clang. - fix one `-Wformat-nonliteral` warning. - fix new `-Wformat` and `-Wformat-security` warnings. - fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its definition to `lib/curl_setup.h` allowing use in `tests/server`. - lib: fix two wrongly passed string arguments in log outputs. Co-authored-by: Jay Satiro - fix new `-Wformat` warnings on mingw-w64. [1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895bfc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C%2B%2B.md Closes #12489
show more ...
|
#
9cc57875 |
| 10-Oct-2023 |
Stefan Eissing |
quic: manage connection idle timeouts - configure a 120s idle timeout on our side of the connection - track the timestamp when actual socket IO happens - check IO timestamp to our *a
quic: manage connection idle timeouts - configure a 120s idle timeout on our side of the connection - track the timestamp when actual socket IO happens - check IO timestamp to our *and* the peer's idle timeouts in "is this connection alive" checks Reported-by: calvin2021y on github Fixes #12064 Closes #12077
show more ...
|
#
331b89a3 |
| 29-Aug-2023 |
Stefan Eissing |
http2: polish things around POST - added test cases for various code paths - fixed handling of blocked write when stream had been closed inbetween attempts - re-enabled DEBUGAS
http2: polish things around POST - added test cases for various code paths - fixed handling of blocked write when stream had been closed inbetween attempts - re-enabled DEBUGASSERT on send with smaller data size - in debug builds, environment variables can be set to simulate a slow network when sending data. cf-socket.c and vquic.c support * CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be answered with a EAGAIN. TCP/UNIX sockets. This is chosen randomly. * CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written to the network. TCP/UNIX sockets. Example: 80 means a send with 1000 bytes would only send 800 This is applied to every send. * CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be answered with EAGAIN. QUIC only. This is chosen randomly. Closes #11756
show more ...
|
#
d50fe6be |
| 25-Aug-2023 |
Viktor Szakats |
lib: fix null ptr derefs and uninitialized vars (h2/h3) Fixing compiler warnings with gcc 13.2.0 in unity builds. Assisted-by: Jay Satiro Assisted-by: Stefan Eissing Closes
lib: fix null ptr derefs and uninitialized vars (h2/h3) Fixing compiler warnings with gcc 13.2.0 in unity builds. Assisted-by: Jay Satiro Assisted-by: Stefan Eissing Closes #11739
show more ...
|
#
199fb4b9 |
| 07-Aug-2023 |
Stefan Eissing |
http3/ngtcp2: shorten handshake, trace cleanup - shorten handshake timing by delayed x509 store load (OpenSSL) as we do for HTTP/2 - cleanup of trace output, align with HTTP/2 outp
http3/ngtcp2: shorten handshake, trace cleanup - shorten handshake timing by delayed x509 store load (OpenSSL) as we do for HTTP/2 - cleanup of trace output, align with HTTP/2 output Closes #11609
show more ...
|
#
ae91bb7e |
| 03-Aug-2023 |
Goro FUJI |
vquic: show stringified messages for errno Closes #11584
|
#
e12b39e1 |
| 03-Aug-2023 |
Stefan Eissing |
trace: make tracing available in non-debug builds Add --trace-config to curl Add curl_global_trace() to libcurl Closes #11421
|
#
fd306e55 |
| 16-Jun-2023 |
Jay Satiro |
lib: fix some format specifiers - Use CURL_FORMAT_CURL_OFF_T where %zd was erroneously used for some curl_off_t variables. - Use %zu where %zd was erroneously used for some si
lib: fix some format specifiers - Use CURL_FORMAT_CURL_OFF_T where %zd was erroneously used for some curl_off_t variables. - Use %zu where %zd was erroneously used for some size_t variables. Prior to this change some of the Windows CI tests were failing because in Windows 32-bit targets have a 32-bit size_t and a 64-bit curl_off_t. When %zd was used for some curl_off_t variables then only the lower 32-bits was read and the upper 32-bits would be read for part or all of the next specifier. Fixes https://github.com/curl/curl/issues/11327 Closes https://github.com/curl/curl/pull/11321
show more ...
|
#
8e85764b |
| 22-May-2023 |
Daniel Stenberg |
lib: remove unused functions, make single-use static Closes #11174
|
#
5b4bcc6e |
| 18-May-2023 |
Daniel Stenberg |
vquic.c: make recvfrom_packets static, avoid compiler warning warning: no previous prototype for 'recvfrom_packets' Reported-by: Keitagit-kun on github Fixes #11146 Closes #
vquic.c: make recvfrom_packets static, avoid compiler warning warning: no previous prototype for 'recvfrom_packets' Reported-by: Keitagit-kun on github Fixes #11146 Closes #11148
show more ...
|
#
cab2d56e |
| 21-Apr-2023 |
Stefan Eissing |
h2/h3: replace `state.drain` counter with `state.dselect_bits` - `drain` was used by http/2 and http/3 implementations to indicate that the transfer requires send/recv independant from
h2/h3: replace `state.drain` counter with `state.dselect_bits` - `drain` was used by http/2 and http/3 implementations to indicate that the transfer requires send/recv independant from its socket poll state. Intended as a counter, it was used as bool flag only. - a similar mechanism exists on `connectdata->cselect_bits` where specific protocols can indicate something similar, only for the whole connection. - `cselect_bits` are cleard in transfer.c on use and, importantly, also set when the transfer loop expended its `maxloops` tries. `drain` was not cleared by transfer and the http2/3 implementations had to take care of that. - `dselect_bits` is cleared *and* set by the transfer loop. http2/3 does no longer clear it, only set when new events happen. This change unifies the handling of socket poll overrides, extending `cselect_bits` by a easy handle specific value and a common treatment in transfers. Closes #11005
show more ...
|
#
544abeea |
| 30-Mar-2023 |
Stefan Eissing |
http3: improvements across backends - ngtcp2: using bufq for recv stream data - internal stream_ctx instead of `struct HTTP` members for quiche, ngtcp2 and msh3 - no more QUIC
http3: improvements across backends - ngtcp2: using bufq for recv stream data - internal stream_ctx instead of `struct HTTP` members for quiche, ngtcp2 and msh3 - no more QUIC related members in `struct HTTP` - experimental use of recvmmsg(), disabled by default - testing on my old debian box shows no throughput improvements. - leaving it in, but disabled, for future revisit - vquic: common UDP receive code for ngtcp2 and quiche - vquic: common UDP send code for ngtcp2 and quiche - added pytest skips for known msh3 failures - fix unit2601 to survive torture testing - quiche: using latest `master` from quiche and enabling large download tests, now that key change is supported - fixing test_07_21 where retry handling of starting a stream was faulty - msh3: use bufq for recv buffering headers and data - msh3: replace fprintf debug logging with LOG_CF where possible - msh3: force QUIC expire timers on recv/send to have more than 1 request per second served Closes #10772
show more ...
|
#
78f73f79 |
| 23-Mar-2023 |
Stefan Eissing |
ngtcp2: adjust config and code checks for ngtcp2 without nghttp3 - make configure show on HTTP3 feature that both ngtcp2 and nghttp3 are in play - define ENABLE_QUIC only when USE_
ngtcp2: adjust config and code checks for ngtcp2 without nghttp3 - make configure show on HTTP3 feature that both ngtcp2 and nghttp3 are in play - define ENABLE_QUIC only when USE_NGTCP2 and USE_NGHTTP3 are defined - add USE_NGHTTP3 in the ngtcp2 implementation Fixes #10793 Closes #10821
show more ...
|
#
a4d015e6 |
| 28-Feb-2023 |
Stefan Eissing |
http: fix unix domain socket use in https connects - when h2/h3 eyeballing was involved, unix domain socket configurations were not honoured - configuring --unix-socket will disabl
http: fix unix domain socket use in https connects - when h2/h3 eyeballing was involved, unix domain socket configurations were not honoured - configuring --unix-socket will disable HTTP/3 as candidate for eyeballing - combinatino of --unix-socket and --http3-only will fail during initialisation - adding pytest test_11 to reproduce Reported-by: Jelle van der Waa Fixes #10633 Closes #10641
show more ...
|
#
51211a31 |
| 27-Feb-2023 |
Viktor Szakats |
quic/schannel: fix compiler warnings Fixes #10603 Closes #10616
|
#
7ce140ba |
| 12-Feb-2023 |
Viktor Szakats |
h3: silence compiler warnings Reviewed-by: Daniel Stenberg Fixes #10485 Closes #10486
|
#
c96f9821 |
| 09-Feb-2023 |
Stefan Eissing |
vquic: stabilization and improvements vquic stabilization - udp send code shared between ngtcp2 and quiche - quiche handling of data and events improved ngtcp2 and pytest im
vquic: stabilization and improvements vquic stabilization - udp send code shared between ngtcp2 and quiche - quiche handling of data and events improved ngtcp2 and pytest improvements - fixes handling of "drain" situations, discovered in scorecard tests with the Caddy server. - improvements in handling transfers that have already data or are already closed to make an early return on recv pytest - adding caddy tests when available scorecard improvemnts. - using correct caddy port - allowing tests for only httpd or caddy Closes #10451
show more ...
|
#
67115824 |
| 01-Feb-2023 |
Stefan Eissing |
connections: introduce http/3 happy eyeballs New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing. - filter is installed when `--http3` in the tool is used (or the equivalent CURL
connections: introduce http/3 happy eyeballs New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing. - filter is installed when `--http3` in the tool is used (or the equivalent CURLOPT_ done in the library) - starts a QUIC/HTTP/3 connect right away. Should that not succeed after 100ms (subject to change), a parallel attempt is started for HTTP/2 and HTTP/1.1 via TCP - both attempts are subject to IPv6/IPv4 eyeballing, same as happens for other connections - tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT - use a `soft` timeout at half the value. When the soft timeout expires, the HTTPS-CONNECT filter checks if the QUIC filter has received any data from the server. If not, it will start the HTTP/2 attempt. HTTP/3(ngtcp2) improvements. - setting call_data in all cfilter calls similar to http/2 and vtls filters for use in callback where no stream data is available. - returning CURLE_PARTIAL_FILE for prematurely terminated transfers - enabling pytest test_05 for h3 - shifting functionality to "connect" UDP sockets from ngtcp2 implementation into the udp socket cfilter. Because unconnected UDP sockets are weird. For example they error when adding to a pollset. HTTP/3(quiche) improvements. - fixed upload bug in quiche implementation, now passes 251 and pytest - error codes on stream RESET - improved debug logs - handling of DRAIN during connect - limiting pending event queue HTTP/2 cfilter improvements. - use LOG_CF macros for dynamic logging in debug build - fix CURLcode on RST streams to be CURLE_PARTIAL_FILE - enable pytest test_05 for h2 - fix upload pytests and improve parallel transfer performance. GOAWAY handling for ngtcp2/quiche - during connect, when the remote server refuses to accept new connections and closes immediately (so the local conn goes into DRAIN phase), the connection is torn down and a another attempt is made after a short grace period. This is the behaviour observed with nghttpx when we tell it to shut down gracefully. Tested in pytest test_03_02. TLS improvements - ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces copy of logic in all tls backends. - standardized the infof logging of offered ALPNs - ALPN negotiated: have common function for all backends that sets alpn proprty and connection related things based on the negotiated protocol (or lack thereof). - new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation. Invoke: python3 tests/tests-httpd/scorecard.py --help for usage. Improvements on gathering connect statistics and socket access. - new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters report connection statistics. This is triggered when the connection has completely connected. - new void Curl_pgrsTimeWas(..) method to report a timer update with a timestamp of when it happend. This allows for updating timers "later", e.g. a connect statistic after full connectivity has been reached. - in case of HTTP eyeballing, the previous changes will update statistics only from the filter chain that "won" the eyeballing. - new cfilter query CF_QUERY_SOCKET for retrieving the socket used by a filter chain. Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket() for convenient use of this query. - Change VTLS backend to query their sub-filters for the socket when checks during the handshake are made. HTTP/3 documentation on how https eyeballing works. TLS improvements - ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces copy of logic in all tls backends. - standardized the infof logging of offered ALPNs - ALPN negotiated: have common function for all backends that sets alpn proprty and connection related things based on the negotiated protocol (or lack thereof). Scorecard with Caddy. - configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing - tests/tests-httpd/scorecard.py now measures download speeds with caddy pytest improvements - adding Makfile to clean gen dir - adding nghttpx rundir creation on start - checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old. - catch exception when checking for caddy existance on system. Closes #10349
show more ...
|
#
446267c5 |
| 05-Jan-2023 |
Stefan Eissing |
quic: rename vquic implementations, fix for quiche build. - quiche in debug mode did not build, fixed. - moved all vquic implementation files to prefix curl_* to avoid the potentia
quic: rename vquic implementations, fix for quiche build. - quiche in debug mode did not build, fixed. - moved all vquic implementation files to prefix curl_* to avoid the potential mixups between provided .h files and our own. - quich passes test 2500 and 2502. 2501, the POST, fail with the body being rejected. Quich bug? Closes #10242
show more ...
|
#
2bc1d775 |
| 02-Jan-2023 |
Daniel Stenberg |
copyright: update all copyright lines and remove year ranges - they are mostly pointless in all major jurisdictions - many big corporations and projects already don't use them - save
copyright: update all copyright lines and remove year ranges - they are mostly pointless in all major jurisdictions - many big corporations and projects already don't use them - saves us from pointless churn - git keeps history for us - the year range is kept in COPYING checksrc is updated to allow non-year using copyright statements Closes #10205
show more ...
|
#
71b7e016 |
| 30-Dec-2022 |
Stefan Eissing |
lib: connect/h2/h3 refactor Refactoring of connection setup and happy eyeballing. Move nghttp2. ngtcp2, quiche and msh3 into connection filters. - eyeballing cfilter that uses
lib: connect/h2/h3 refactor Refactoring of connection setup and happy eyeballing. Move nghttp2. ngtcp2, quiche and msh3 into connection filters. - eyeballing cfilter that uses sub-filters for performing parallel connects - socket cfilter for all transport types, including QUIC - QUIC implementations in cfilter, can now participate in eyeballing - connection setup is more dynamic in order to adapt to what filter did really connect. Relevant to see if a SSL filter needs to be added or if SSL has already been provided - HTTP/3 test cases similar to HTTP/2 - multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche - Fix for data attach/detach in VTLS filters that could lead to crashes during parallel transfers. - Eliminating setup() methods in cfilters, no longer needed. - Improving Curl_conn_is_alive() to replace Curl_connalive() and integrated ssl alive checks into cfilter. - Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update connection into and persist it at the easy handle. - Several more cfilter related cleanups and moves: - stream_weigth and dependency info is now wrapped in struct Curl_data_priority - Curl_data_priority members depend is available in HTTP2|HTTP3 - Curl_data_priority members depend on NGHTTP2 support - handling init/reset/cleanup of priority part of url.c - data->state.priority same struct, but shallow copy for compares only - PROTOPT_STREAM has been removed - Curl_conn_is_mulitplex() now available to check on capability - Adding query method to connection filters. - ngtcp2+quiche: implementing query for max concurrent transfers. - Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event. - setting keepalive timestamp on connect - DATA_SETUP is called after the connection has been completely setup (but may not connected yet) to allow filters to initialize data members they use. - there is no socket to be had with msh3, it is unclear how select shall work - manual test via "curl --http3 https://curl.se" fail with "empty reply from server". - Various socket/conn related cleanups: - Curl_socket is now Curl_socket_open and in cf-socket.c - Curl_closesocket is now Curl_socket_close and in cf-socket.c - Curl_ssl_use has been replaced with Cur_conn_is_ssl - Curl_conn_tcp_accepted_set has been split into Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set with a clearer purpose Closes #10141
show more ...
|