tests: enable additional ruff Python lint options

These all seem reasonable to enable for this code.

tests: change Python code style to pass ruff checks

Most of the changes consisted of removing unused imports and unnecessary
f-strings.

tests: fix some Python typing issues

These otherwise raise errors in pytype. A few problematic methods
weren't being used and are deleted.

tests/http: add HTTP/2 Upgrade and prior knowledge tests

Adds test cases to check that plain http: with HTTP/2 works
via 'Upgrade: h2c' or --http2-prior-knowledge'.

Also added t

tests/http: add HTTP/2 Upgrade and prior knowledge tests

Adds test cases to check that plain http: with HTTP/2 works
via 'Upgrade: h2c' or --http2-prior-knowledge'.

Also added tests to check connection reused in these situations.

Closes #14694

show more ...

tests: improve test_17_07_ssl_ciphers

Change TLS proto version on the test httpd server to test setting
combinations of --tls13-ciphers and --ciphers.

To not let the changed con

tests: improve test_17_07_ssl_ciphers

Change TLS proto version on the test httpd server to test setting
combinations of --tls13-ciphers and --ciphers.

To not let the changed config of the httpd server bleed into the next
test, clean and reload on each test. Because a reload is slow, only
do this if the config is different than the loaded config. For this
the httpd.reload_if_config_changed() method is added.

Overloading of autouse fixtures does not seem to work. For the test
httpd server to be reloaded with a clean config in test_18_methods,
to not be affected by the config changes in test_17_ssl_use, the two
class scope fixtures of test_18_methods are now combined.

Closes #14589

show more ...

test httpd, tweak cipher list

Configure the AES 256 instead of the AES 128 cipher in the test httpd to
make scorecard testing between httpd and caddy more comparable.

Adapt test

test httpd, tweak cipher list

Configure the AES 256 instead of the AES 128 cipher in the test httpd to
make scorecard testing between httpd and caddy more comparable.

Adapt test_17 expectations, now that AES 128 can no longer, but 256 can
now be negotiated.

Closes #14502

show more ...

revert "tests/http: configure test httpd to honor client cipher order"

revert f6cb3c63 #14338

Setting SSLHonorCipherOrder to on means it honors the server cipher
order. From the

revert "tests/http: configure test httpd to honor client cipher order"

revert f6cb3c63 #14338

Setting SSLHonorCipherOrder to on means it honors the server cipher
order. From the documentation: "When choosing a cipher during an SSLv3
or TLSv1 handshake, normally the client's preference is used. If this
directive is enabled, the server's preference will be used instead."

Also the commit inhibits test_17_07_ssl_ciphers. The test tries to
tests if all the ciphers specified, and only those, are properly set
in curl. For that to work we need have cases where some or all ciphers
do no intersect with the cipher-set of the server. We need to be able
to assert a failed connection based on a cipher set mismatch.

That is why a restricted set of ciphers is used on the server. This
set is so chosen that it contains the well known most secure ciphers.
Except with the slower aes256 variant intentionally left out, to be
able to test above described.

As test_17_07_ssl_ciphers is currently the only test that tests the
functioning of the --ciphers and --tls13-ciphers options, it is
important that its coverage is as good as possible.

Closes #14381

show more ...

tests/http: configure test httpd to honor client cipher order

Let the client, e.g. curl, influence the cipher selected in a TLS
handshake. TLS backends have different preferences and hon

tests/http: configure test httpd to honor client cipher order

Let the client, e.g. curl, influence the cipher selected in a TLS
handshake. TLS backends have different preferences and honor that
in httpd the same as Caddy does.

Also makes for a more fair compare of different TLS backends.

Closes #14338

show more ...

pytests: scorecard upload tests

- add upload tests to scorecard, invoke with
> python3 tests/http/scorecard.py -u h1|h2|h3
- add a reverse proxy setup from Caddy to httpd for

pytests: scorecard upload tests

- add upload tests to scorecard, invoke with
> python3 tests/http/scorecard.py -u h1|h2|h3
- add a reverse proxy setup from Caddy to httpd for
upload tests since Caddy does not have other PUT/POST handling
- add caddy tests in test_08 for POST/PUT
- increase read buffer in mod_curltest for larger reads

Closes #14208

show more ...

tests/httpd: adjust ReadBufferSize for better performance

- list httpd and caddy versions in scorecard run

Closes #14039

tests: add pytest for --ciphers and --tls13-ciphers options

Closes #13530

vtls: TLS session storage overhaul

- add session with destructor callback
- remove vtls `session_free` method
- let `Curl_ssl_addsessionid()` take ownership
of session object,

vtls: TLS session storage overhaul

- add session with destructor callback
- remove vtls `session_free` method
- let `Curl_ssl_addsessionid()` take ownership
of session object, freeing it also on failures
- change tls backend use
- test_17, add tests for SSL session resumption

Closes #13386

show more ...

CURLINFO_REQUEST_SIZE: fixed, add tests for transfer infos reported

- tests for 'size_request' and other stats reported, for
presence and consistency

Reported-by: Jonatan Vela

CURLINFO_REQUEST_SIZE: fixed, add tests for transfer infos reported

- tests for 'size_request' and other stats reported, for
presence and consistency

Reported-by: Jonatan Vela
Fixes #13269
Closes #13275

show more ...

content_encoding: brotli and others, pass through 0-length writes

- curl's transfer handling may write 0-length chunks at the end of the
download with an EOS flag. (HTTP/2 does this co

content_encoding: brotli and others, pass through 0-length writes

- curl's transfer handling may write 0-length chunks at the end of the
download with an EOS flag. (HTTP/2 does this commonly)

- content encoders need to pass-through such a write and not count this
as error in case they are finished decoding

Fixes #13209
Fixes #13212
Closes #13219

show more ...

vtls: revert "receive max buffer" + add test case

- add test_05_04 for requests using http/1.0, http/1.1 and h2 against an
Apache resource that does an unclean TLS shutdown.
- reve

vtls: revert "receive max buffer" + add test case

- add test_05_04 for requests using http/1.0, http/1.1 and h2 against an
Apache resource that does an unclean TLS shutdown.
- revert special workarund in openssl.c for suppressing shutdown errors
on multiplexed connections
- vlts.c restore to its state before 9a90c9dd64d2f03601833a70786d485851bd1b53

Fixes #12885
Fixes #12844

Closes #12848

show more ...

h2: testcase and fix for pausing h2 streams

- refs #11982 where it was noted that paused transfers may
close successfully without delivering the complete data
- made sample poc int

h2: testcase and fix for pausing h2 streams

- refs #11982 where it was noted that paused transfers may
close successfully without delivering the complete data
- made sample poc into tests/http/client/h2-pausing.c and
added test_02_27 to reproduce

Closes #11989
Fixes #11982
Reported-by: Harry Sintonen

show more ...

http2: polish things around POST

- added test cases for various code paths
- fixed handling of blocked write when stream had
been closed inbetween attempts
- re-enabled DEBUGAS

http2: polish things around POST

- added test cases for various code paths
- fixed handling of blocked write when stream had
been closed inbetween attempts
- re-enabled DEBUGASSERT on send with smaller data size

- in debug builds, environment variables can be set to simulate a slow
network when sending data. cf-socket.c and vquic.c support
* CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be
answered with a EAGAIN. TCP/UNIX sockets.
This is chosen randomly.
* CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written
to the network. TCP/UNIX sockets.
Example: 80 means a send with 1000 bytes would only send 800
This is applied to every send.
* CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be
answered with EAGAIN. QUIC only.
This is chosen randomly.

Closes #11756

show more ...

bearssl: handshake fix, provide proper get_select_socks() implementation

- bring bearssl handshake times down from +200ms down to other TLS backends
- vtls: improve generic get_select_so

bearssl: handshake fix, provide proper get_select_socks() implementation

- bring bearssl handshake times down from +200ms down to other TLS backends
- vtls: improve generic get_select_socks() implementation
- tests: provide Apache with a suitable ssl session cache

Closes #11675

show more ...

http2: upgrade tests and add fix for non-existing stream

- check in h2 filter recv that stream actually exists
and return error if not
- add test for parallel, extreme h2 upgrades

http2: upgrade tests and add fix for non-existing stream

- check in h2 filter recv that stream actually exists
and return error if not
- add test for parallel, extreme h2 upgrades that fail if
connections get reused before fully switched
- add h2 upgrade upload test just for completeness

Closes #11563

show more ...

tests/http: Add mod_h2 directive `H2ProxyRequests`

master of mod_h2 now requires H2ProxyRequests directives for forward
proxying with HTTP/2 to work.

Ref: https://github.com/ici

tests/http: Add mod_h2 directive `H2ProxyRequests`

master of mod_h2 now requires H2ProxyRequests directives for forward
proxying with HTTP/2 to work.

Ref: https://github.com/icing/mod_h2/commit/3897a7086

Closes https://github.com/curl/curl/pull/11392

show more ...

http2: error stream resets with code CURLE_HTTP2_STREAM

- refs #11357, where it was reported that HTTP/1.1 downgrades
no longer works
- fixed with suggested change
- added test

http2: error stream resets with code CURLE_HTTP2_STREAM

- refs #11357, where it was reported that HTTP/1.1 downgrades
no longer works
- fixed with suggested change
- added test_05_03 and a new handler in the curltest module
to reproduce that downgrades work

Fixes #11357
Closes #11362
Reported-by: Jay Satiro

show more ...

http2: fix EOF handling on uploads with auth negotiation

- doing a POST with `--digest` does an override on the initial request
with `Content-Length: 0`, but the http2 filter was unawa

http2: fix EOF handling on uploads with auth negotiation

- doing a POST with `--digest` does an override on the initial request
with `Content-Length: 0`, but the http2 filter was unaware of that
and expected the originally request body. It did therefore not
send a final DATA frame with EOF flag to the server.
- The fix overrides any initial notion of post size when the `done_send`
event is triggered by the transfer loop, leading to the EOF that
is necessary.
- refs #11194. The fault did not happen in testing, as Apache httpd
never tries to read the request body of the initial request,
sends the 401 reply and closes the stream. The server used in the
reported issue however tried to read the EOF and timed out on the
request.

Reported-by: Aleksander Mazur
Fixes #11194
Cloes #11200

show more ...

tests/http: more tests with specific clients

- Makefile support for building test specific clients in tests/http/clients
- auto-make of clients when invoking pytest
- added test_09_0

tests/http: more tests with specific clients

- Makefile support for building test specific clients in tests/http/clients
- auto-make of clients when invoking pytest
- added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush
- added test_02_21 for lib based downloads and pausing/unpausing transfers

curl url parser:
- added internal method `curl_url_set_authority()` for setting the
authority part of a url (used for PUSH_PROMISE)

http2:
- made logging of PUSH_PROMISE handling nicer

Placing python test requirements in requirements.txt files
- separate files to base test suite and http tests since use
and module lists differ
- using the files in the gh workflows

websocket test cases, fixes for we and bufq
- bufq: account for spare chunks in space calculation
- bufq: reset chunks that are skipped empty
- ws: correctly encode frames with 126 bytes payload
- ws: update frame meta information on first call of collect
callback that fills user buffer
- test client ws-data: some test/reporting improvements

Closes #11006

show more ...

proxy: http2 proxy tunnel implementation

- currently only on debug build and when env variable
CURL_PROXY_TUNNEL_H2 is present.
- will ALPN negotiate with the proxy server and swit

proxy: http2 proxy tunnel implementation

- currently only on debug build and when env variable
CURL_PROXY_TUNNEL_H2 is present.
- will ALPN negotiate with the proxy server and switch
tunnel filter based on the protocol negotiated.
- http/1.1 tunnel code moved into cf-h1-proxy.[ch]
- http/2 tunnel code implemented in cf-h2-proxy.[ch]
- tunnel start and ALPN set remains in http_proxy.c
- moving all haproxy related code into cf-haproxy.[ch]

VTLS changes
- SSL filters rely solely on the "alpn" specification they
are created with and no longer check conn->bits.tls_enable_alpn.
- checks on which ALPN specification to use (or none at all) are
done in vtls.c when creating the filter.

Testing
- added a nghttpx forward proxy to the pytest setup that
speaks HTTP/2 and forwards all requests to the Apache httpd
forward proxy server.
- extending test coverage in test_10 cases
- adding proxy tests for direct/tunnel h1/h2 use of basic auth.
- adding test for http/1.1 and h2 proxy tunneling to pytest

Closes #10780

show more ...

tests/http: add timeout to running curl in test cases

- we had a CI case once where `curl` seemingly did not
return and it was hard to guess what happened.
- make curl execution in

tests/http: add timeout to running curl in test cases

- we had a CI case once where `curl` seemingly did not
return and it was hard to guess what happened.
- make curl execution in test cases time out after 60 seconds

Closes #10783

show more ...