lib: remove unused functions, make single-use static

Closes #11174

http2: remove unused Curl_http2_strerror function declaration

Curl_http2_strerror was renamed to http2_strerror in
05b100aee247bb9bec8e9a1b0 and then http2_strerror was removed in
58

http2: remove unused Curl_http2_strerror function declaration

Curl_http2_strerror was renamed to http2_strerror in
05b100aee247bb9bec8e9a1b0 and then http2_strerror was removed in
5808a0d0f5ea0399d4a2a2

This also fixes the following compiler error

lib/http2.h:41:33: error: unknown type name 'uint32_t'
lib/http2.h:1:1: note: 'uint32_t' is defined in header '<stdint.h>'

Closes #10912

show more ...

connections: introduce http/3 happy eyeballs

New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
the equivalent CURL

connections: introduce http/3 happy eyeballs

New cfilter HTTP-CONNECT for h3/h2/http1.1 eyeballing.
- filter is installed when `--http3` in the tool is used (or
the equivalent CURLOPT_ done in the library)
- starts a QUIC/HTTP/3 connect right away. Should that not
succeed after 100ms (subject to change), a parallel attempt
is started for HTTP/2 and HTTP/1.1 via TCP
- both attempts are subject to IPv6/IPv4 eyeballing, same
as happens for other connections
- tie timeout to the ip-version HAPPY_EYEBALLS_TIMEOUT
- use a `soft` timeout at half the value. When the soft timeout
expires, the HTTPS-CONNECT filter checks if the QUIC filter
has received any data from the server. If not, it will start
the HTTP/2 attempt.

HTTP/3(ngtcp2) improvements.
- setting call_data in all cfilter calls similar to http/2 and vtls filters
for use in callback where no stream data is available.
- returning CURLE_PARTIAL_FILE for prematurely terminated transfers
- enabling pytest test_05 for h3
- shifting functionality to "connect" UDP sockets from ngtcp2
implementation into the udp socket cfilter. Because unconnected
UDP sockets are weird. For example they error when adding to a
pollset.

HTTP/3(quiche) improvements.
- fixed upload bug in quiche implementation, now passes 251 and pytest
- error codes on stream RESET
- improved debug logs
- handling of DRAIN during connect
- limiting pending event queue

HTTP/2 cfilter improvements.
- use LOG_CF macros for dynamic logging in debug build
- fix CURLcode on RST streams to be CURLE_PARTIAL_FILE
- enable pytest test_05 for h2
- fix upload pytests and improve parallel transfer performance.

GOAWAY handling for ngtcp2/quiche
- during connect, when the remote server refuses to accept new connections
and closes immediately (so the local conn goes into DRAIN phase), the
connection is torn down and a another attempt is made after a short grace
period.
This is the behaviour observed with nghttpx when we tell it to shut
down gracefully. Tested in pytest test_03_02.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).

- new tests/tests-httpd/scorecard.py for testing h3/h2 protocol implementation.
Invoke:
python3 tests/tests-httpd/scorecard.py --help
for usage.

Improvements on gathering connect statistics and socket access.
- new CF_CTRL_CONN_REPORT_STATS cfilter control for having cfilters
report connection statistics. This is triggered when the connection
has completely connected.
- new void Curl_pgrsTimeWas(..) method to report a timer update with
a timestamp of when it happend. This allows for updating timers
"later", e.g. a connect statistic after full connectivity has been
reached.
- in case of HTTP eyeballing, the previous changes will update
statistics only from the filter chain that "won" the eyeballing.
- new cfilter query CF_QUERY_SOCKET for retrieving the socket used
by a filter chain.
Added methods Curl_conn_cf_get_socket() and Curl_conn_get_socket()
for convenient use of this query.
- Change VTLS backend to query their sub-filters for the socket when
checks during the handshake are made.

HTTP/3 documentation on how https eyeballing works.

TLS improvements
- ALPN selection for SSL/SSL-PROXY filters in one vtls set of functions, replaces
copy of logic in all tls backends.
- standardized the infof logging of offered ALPNs
- ALPN negotiated: have common function for all backends that sets alpn proprty
and connection related things based on the negotiated protocol (or lack thereof).

Scorecard with Caddy.
- configure can be run with `--with-test-caddy=path` to specify which caddy to use for testing
- tests/tests-httpd/scorecard.py now measures download speeds with caddy

pytest improvements
- adding Makfile to clean gen dir
- adding nghttpx rundir creation on start
- checking httpd version 2.4.55 for test_05 cases where it is needed. Skipping with message if too old.
- catch exception when checking for caddy existance on system.

Closes #10349

show more ...

curl_log: for failf/infof and debug logging implementations

- new functions and macros for cfilter debugging
- set CURL_DEBUG with names of cfilters where debug logging should be

curl_log: for failf/infof and debug logging implementations

- new functions and macros for cfilter debugging
- set CURL_DEBUG with names of cfilters where debug logging should be
enabled
- use GNUC __attribute__ to enable printf format checks during compile

Closes #10271

show more ...

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- save

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205

show more ...

lib: connect/h2/h3 refactor

Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.

- eyeballing cfilter that uses

lib: connect/h2/h3 refactor

Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.

- eyeballing cfilter that uses sub-filters for performing parallel connects
- socket cfilter for all transport types, including QUIC
- QUIC implementations in cfilter, can now participate in eyeballing
- connection setup is more dynamic in order to adapt to what filter did
really connect. Relevant to see if a SSL filter needs to be added or
if SSL has already been provided
- HTTP/3 test cases similar to HTTP/2
- multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche

- Fix for data attach/detach in VTLS filters that could lead to crashes
during parallel transfers.
- Eliminating setup() methods in cfilters, no longer needed.
- Improving Curl_conn_is_alive() to replace Curl_connalive() and
integrated ssl alive checks into cfilter.
- Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update
connection into and persist it at the easy handle.

- Several more cfilter related cleanups and moves:
- stream_weigth and dependency info is now wrapped in struct
Curl_data_priority
- Curl_data_priority members depend is available in HTTP2|HTTP3
- Curl_data_priority members depend on NGHTTP2 support
- handling init/reset/cleanup of priority part of url.c
- data->state.priority same struct, but shallow copy for compares only

- PROTOPT_STREAM has been removed
- Curl_conn_is_mulitplex() now available to check on capability

- Adding query method to connection filters.
- ngtcp2+quiche: implementing query for max concurrent transfers.

- Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event.
- setting keepalive timestamp on connect
- DATA_SETUP is called after the connection has been completely
setup (but may not connected yet) to allow filters to initialize
data members they use.

- there is no socket to be had with msh3, it is unclear how select
shall work

- manual test via "curl --http3 https://curl.se" fail with "empty
reply from server".

- Various socket/conn related cleanups:
- Curl_socket is now Curl_socket_open and in cf-socket.c
- Curl_closesocket is now Curl_socket_close and in cf-socket.c
- Curl_ssl_use has been replaced with Cur_conn_is_ssl
- Curl_conn_tcp_accepted_set has been split into
Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set
with a clearer purpose

Closes #10141

show more ...

lib: rewind BEFORE request instead of AFTER previous

This makes a big difference for cases when the rewind is not actually
necessary to perofm (for example HTTP response code 301 convert

lib: rewind BEFORE request instead of AFTER previous

This makes a big difference for cases when the rewind is not actually
necessary to perofm (for example HTTP response code 301 converts to GET)
and therefore the rewind can be avoided. In particular for situations
when that rewind fails, for example when reading from a pipe or similar.

Reported-by: Ali Utku Selen

Fixes #9735
Closes #9958

show more ...

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869

show more ...

version: turn version number functions into returning void

... as we never use the return codes from them.

Reviewed-by: Daniel Gustafsson
Closes #7319

http2: move the stream error field to the per-transfer storage

Storing a stream error in the per-connection struct was an error that lead to
race conditions as subsequent stream handling

http2: move the stream error field to the per-transfer storage

Storing a stream error in the per-connection struct was an error that lead to
race conditions as subsequent stream handling could overwrite the error code
before it was used for the stream with the actual problem.

Closes #6910

show more ...

h2: allow 100 streams by default

instead of 13, before the server has told how many streams it
accepts. The server can always reject new streams anyway if we go above
what it accepts

h2: allow 100 streams by default

instead of 13, before the server has told how many streams it
accepts. The server can always reject new streams anyway if we go above
what it accepts.

Ref: #6826
Closes #6852

show more ...

http2: remove conn->data use

... but instead use a private alternative that points to the "driving
transfer" from the connection. We set the "user data" associated with
the connectio

http2: remove conn->data use

... but instead use a private alternative that points to the "driving
transfer" from the connection. We set the "user data" associated with
the connection to be the connectdata struct, but when we drive transfers
the code still needs to know the pointer to the transfer. We can change
the user data to become the Curl_easy handle, but with older nghttp2
version we cannot dynamically update that pointer properly when
different transfers are used over the same connection.

Closes #6520

show more ...

lib: pass in 'struct Curl_easy *' to most functions

... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and no

lib: pass in 'struct Curl_easy *' to most functions

... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and not connections.

- We need the transfer handle to log, store data and more. Everything in
libcurl is driven by a transfer (the CURL * in the public API).

- This work clarifies and separates the transfers from the connections
better.

- We should avoid "conn->data". Since individual connections can be used
by many transfers when multiplexing, making sure that conn->data
points to the current and correct transfer at all times is difficult
and has been notoriously error-prone over the years. The goal is to
ultimately remove the conn->data pointer for this reason.

Closes #6425

show more ...

curl.se: new home

Closes #6172

dynbuf: introduce internal generic dynamic buffer functions

A common set of functions instead of many separate implementations for
creating buffers that can grow when appending data to t

dynbuf: introduce internal generic dynamic buffer functions

A common set of functions instead of many separate implementations for
creating buffers that can grow when appending data to them. Existing
functionality has been ported over.

In my early basic testing, the total number of allocations seem at
roughly the same amount as before, possibly a few less.

See docs/DYNBUF.md for a description of the API.

Closes #5300

show more ...

http2: make pausing/unpausing set/clear local stream window

This reduces the HTTP/2 window size to 32 MB since libcurl might have to
buffer up to this amount of data in memory and yet we

http2: make pausing/unpausing set/clear local stream window

This reduces the HTTP/2 window size to 32 MB since libcurl might have to
buffer up to this amount of data in memory and yet we don't want it set
lower to potentially impact tranfer performance on high speed networks.

Requires nghttp2 commit b3f85e2daa629
(https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end
up in the next release after 1.40.0.

Fixes #4939
Closes #4940

show more ...

conncache: fix multi-thread use of shared connection cache

It could accidentally let the connection get used by more than one
thread, leading to double-free and more.

Reported-b

conncache: fix multi-thread use of shared connection cache

It could accidentally let the connection get used by more than one
thread, leading to double-free and more.

Reported-by: Christopher Reid
Fixes #4544
Closes #4557

show more ...

headers: Remove no longer exported functions

There were a leftover few prototypes of Curl_ functions that we used to
export but no longer do, this removes those prototypes and cleans up

headers: Remove no longer exported functions

There were a leftover few prototypes of Curl_ functions that we used to
export but no longer do, this removes those prototypes and cleans up any
comments still referring to them.

Curl_write32_le(), Curl_strcpy_url(), Curl_strlen_url(), Curl_up_free()
Curl_concat_url(), Curl_detach_connnection(), Curl_http_setup_conn()
were made static in 05b100aee247bb9bec8e9a1b0166496aa4248d1c.
Curl_http_perhapsrewind() made static in 574aecee208f79d391f10d57520b3.

For the remainder, I didn't trawl the Git logs hard enough to capture
their exact time of deletion, but they were all gone: Curl_splayprint(),
Curl_http2_send_request(), Curl_global_host_cache_dtor(),
Curl_scan_cache_used(), Curl_hostcache_destroy(), Curl_second_connect(),
Curl_http_auth_stage() and Curl_close_connections().

Closes #4096
Reviewed-by: Daniel Stenberg <daniel@haxx.se>

show more ...

cleanup: make local functions static

urlapi: turn three local-only functions into statics

conncache: make conncache_find_first_connection static

multi: make detach_connnect

cleanup: make local functions static

urlapi: turn three local-only functions into statics

conncache: make conncache_find_first_connection static

multi: make detach_connnection static

connect: make getaddressinfo static

curl_ntlm_core: make hmac_md5 static

http2: make two functions static

http: make http_setup_conn static

connect: make tcpnodelay static

tests: make UNITTEST a thing to mark functions with, so they can be static for
normal builds and non-static for unit test builds

... and mark Curl_shuffle_addr accordingly.

url: make up_free static

setopt: make vsetopt static

curl_endian: make write32_le static

rtsp: make rtsp_connisdead static

warnless: remove unused functions

memdebug: remove one unused function, made another static

show more ...

Upon HTTP_1_1_REQUIRED, retry the request with HTTP/1.1

This is a companion patch to cbea2fd2c (NTLM: force the connection to
HTTP/1.1, 2018-12-06): with NTLM, we can switch to HTTP/1.1

Upon HTTP_1_1_REQUIRED, retry the request with HTTP/1.1

This is a companion patch to cbea2fd2c (NTLM: force the connection to
HTTP/1.1, 2018-12-06): with NTLM, we can switch to HTTP/1.1
preemptively. However, with other (Negotiate) authentication it is not
clear to this developer whether there is a way to make it work with
HTTP/2, so let's try HTTP/2 first and fall back in case we encounter the
error HTTP_1_1_REQUIRED.

Note: we will still keep the NTLM workaround, as it avoids an extra
round trip.

Daniel Stenberg helped a lot with this patch, in particular by
suggesting to introduce the Curl_h2_http_1_1_error() function.

Closes #3349

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

show more ...

whitespace fixes

- replace tabs with spaces where possible
- remove line ending spaces
- remove double/triple newlines at EOF
- fix a non-UTF-8 character
- cleanup a few inde

whitespace fixes

- replace tabs with spaces where possible
- remove line ending spaces
- remove double/triple newlines at EOF
- fix a non-UTF-8 character
- cleanup a few indentations/line continuations
in manual examples

Closes https://github.com/curl/curl/pull/3037

show more ...

http2: several cleanups

- separate easy handle from connections better
- added asserts on a number of places
- added sanity check of pipelines for debug builds

Closes #2751

http2: fix OOM crash

torture mode with test 1021 found it

http2: Fix crashes when parent stream gets aborted

Closes #1125

http2: return EOF when done uploading without known size

Fixes #982