pytest: fixes for recent python, add FTP tests

Fixes:
- in uds tests, abort also silently on os errors
- be conservative on the h3 goaway duration
- detect curl debug build and u

pytest: fixes for recent python, add FTP tests

Fixes:
- in uds tests, abort also silently on os errors
- be conservative on the h3 goaway duration
- detect curl debug build and use in checks
- fix caddy version check for slight difference under linux
- set caddy default path fitting for linux
- fix deprecation warnings in valid time checks

FTP tests:
- add '--with-test-vsftpd=path' to configure
- use vsftpd default path suitable for linux
- add test_30 with plain FTP tests
- add test_31 with --ssl-reqd FTP tests
- add vsftpd to linux GHA for pytest workflows

Closes #13661

show more ...

tidy-up: whitespace [ci skip]

vtls: revert "receive max buffer" + add test case

- add test_05_04 for requests using http/1.0, http/1.1 and h2 against an
Apache resource that does an unclean TLS shutdown.
- reve

vtls: revert "receive max buffer" + add test case

- add test_05_04 for requests using http/1.0, http/1.1 and h2 against an
Apache resource that does an unclean TLS shutdown.
- revert special workarund in openssl.c for suppressing shutdown errors
on multiplexed connections
- vlts.c restore to its state before 9a90c9dd64d2f03601833a70786d485851bd1b53

Fixes #12885
Fixes #12844

Closes #12848

show more ...

tidy-up: mostly whitespace nits

- delete completed TODO from `./CMakeLists.txt`.
- convert a C++ comment to C89 in `./CMake/CurlTests.c`.
- delete duplicate EOLs from EOF.
- add

tidy-up: mostly whitespace nits

- delete completed TODO from `./CMakeLists.txt`.
- convert a C++ comment to C89 in `./CMake/CurlTests.c`.
- delete duplicate EOLs from EOF.
- add missing EOL at EOF.
- delete whitespace at EOL (except from expected test results).
- convert tabs to spaces.
- convert CRLF EOLs to LF in GHA yaml.
- text casing fixes in `./CMakeLists.txt`.
- fix a codespell typo in `packages/OS400/initscript.sh`.

Closes #11772

show more ...

http3/ngtcp2: shorten handshake, trace cleanup

- shorten handshake timing by delayed x509 store load (OpenSSL)
as we do for HTTP/2
- cleanup of trace output, align with HTTP/2 outp

http3/ngtcp2: shorten handshake, trace cleanup

- shorten handshake timing by delayed x509 store load (OpenSSL)
as we do for HTTP/2
- cleanup of trace output, align with HTTP/2 output

Closes #11609

show more ...

http2: error stream resets with code CURLE_HTTP2_STREAM

- refs #11357, where it was reported that HTTP/1.1 downgrades
no longer works
- fixed with suggested change
- added test

http2: error stream resets with code CURLE_HTTP2_STREAM

- refs #11357, where it was reported that HTTP/1.1 downgrades
no longer works
- fixed with suggested change
- added test_05_03 and a new handler in the curltest module
to reproduce that downgrades work

Fixes #11357
Closes #11362
Reported-by: Jay Satiro

show more ...

h2/h3: replace `state.drain` counter with `state.dselect_bits`

- `drain` was used by http/2 and http/3 implementations to indicate
that the transfer requires send/recv independant from

h2/h3: replace `state.drain` counter with `state.dselect_bits`

- `drain` was used by http/2 and http/3 implementations to indicate
that the transfer requires send/recv independant from its socket
poll state. Intended as a counter, it was used as bool flag only.
- a similar mechanism exists on `connectdata->cselect_bits` where
specific protocols can indicate something similar, only for the
whole connection.
- `cselect_bits` are cleard in transfer.c on use and, importantly,
also set when the transfer loop expended its `maxloops` tries.
`drain` was not cleared by transfer and the http2/3 implementations
had to take care of that.
- `dselect_bits` is cleared *and* set by the transfer loop. http2/3
does no longer clear it, only set when new events happen.

This change unifies the handling of socket poll overrides, extending
`cselect_bits` by a easy handle specific value and a common treatment in
transfers.

Closes #11005

show more ...

proxy: http2 proxy tunnel implementation

- currently only on debug build and when env variable
CURL_PROXY_TUNNEL_H2 is present.
- will ALPN negotiate with the proxy server and swit

proxy: http2 proxy tunnel implementation

- currently only on debug build and when env variable
CURL_PROXY_TUNNEL_H2 is present.
- will ALPN negotiate with the proxy server and switch
tunnel filter based on the protocol negotiated.
- http/1.1 tunnel code moved into cf-h1-proxy.[ch]
- http/2 tunnel code implemented in cf-h2-proxy.[ch]
- tunnel start and ALPN set remains in http_proxy.c
- moving all haproxy related code into cf-haproxy.[ch]

VTLS changes
- SSL filters rely solely on the "alpn" specification they
are created with and no longer check conn->bits.tls_enable_alpn.
- checks on which ALPN specification to use (or none at all) are
done in vtls.c when creating the filter.

Testing
- added a nghttpx forward proxy to the pytest setup that
speaks HTTP/2 and forwards all requests to the Apache httpd
forward proxy server.
- extending test coverage in test_10 cases
- adding proxy tests for direct/tunnel h1/h2 use of basic auth.
- adding test for http/1.1 and h2 proxy tunneling to pytest

Closes #10780

show more ...

http3: improvements across backends

- ngtcp2: using bufq for recv stream data
- internal stream_ctx instead of `struct HTTP` members
for quiche, ngtcp2 and msh3
- no more QUIC

http3: improvements across backends

- ngtcp2: using bufq for recv stream data
- internal stream_ctx instead of `struct HTTP` members
for quiche, ngtcp2 and msh3
- no more QUIC related members in `struct HTTP`
- experimental use of recvmmsg(), disabled by default
- testing on my old debian box shows no throughput improvements.
- leaving it in, but disabled, for future revisit
- vquic: common UDP receive code for ngtcp2 and quiche
- vquic: common UDP send code for ngtcp2 and quiche
- added pytest skips for known msh3 failures
- fix unit2601 to survive torture testing
- quiche: using latest `master` from quiche and enabling large download
tests, now that key change is supported
- fixing test_07_21 where retry handling of starting a stream
was faulty
- msh3: use bufq for recv buffering headers and data
- msh3: replace fprintf debug logging with LOG_CF where possible
- msh3: force QUIC expire timers on recv/send to have more than
1 request per second served

Closes #10772

show more ...

pytest: improvements for suitable curl and error output

- will check built curl for http and https support and
skip all tests if not there
- will dump stdout/stderr/trace output on

pytest: improvements for suitable curl and error output

- will check built curl for http and https support and
skip all tests if not there
- will dump stdout/stderr/trace output on errored responses

Closes #10829

show more ...

tests/http: remove year ranges from copyrights

Closes #10763

tests/http: add pytest to GHA and improve tests

- added to: ngtcp2-quictls, ngtcp2-gnutls and the linux varians
quiche, bearssl, libressl, mbedtls, openssl3, rustls
- added disable

tests/http: add pytest to GHA and improve tests

- added to: ngtcp2-quictls, ngtcp2-gnutls and the linux varians
quiche, bearssl, libressl, mbedtls, openssl3, rustls
- added disabled in ngtcp2-wolfssl due to weird SSL_connect() errors
not reproducable locally

Improvements on pytest:

- handling of systems with nghttpx in $PATH
- configure will seach $PATH got nghttpx used in pytest
- pytest fixes for managing nghttpx without h3 support
- ngtcp2-wolfssl: use a fully enabled wolfssl build

- lower parallel count for http/1.1 tests, since we do not
want to test excessive connections.
- check built curl for HTTPS-proxy support in proxy tests
- bearssl does not like one of our critical cert extensions, making
it non-critical now
- bearssl is too slow for test_12, skipping
- making sure we do h3 tests only when curl and server support is there

Closes #10699

show more ...

http2: fix handling of RST and GOAWAY to recognize partial transfers

- a reset transfer (HTTP/2 RST) did not always lead to the proper
error message on receiving its response, leading

http2: fix handling of RST and GOAWAY to recognize partial transfers

- a reset transfer (HTTP/2 RST) did not always lead to the proper
error message on receiving its response, leading to wrong reports
of a successful transfer
- test_05_02 was able to trigger this condition with increased transfer
count. The simulated response errors did not carry a 'Content-Length'
so only proper RST handling could detect the abort
- When doing such transfers in parallel, a connection could enter the
state where
a) it had been closed (GOAWAY received)
b) the RST had not been "seen" for the transfer yet
or c) the GOAWAY announced an error and the last successful
stream id was not checked against ongoing transfers

Closes #10693

show more ...

tests: use dynamic ports numbers in pytest suite

- necessary ports are bound at start of test suite and then
given to server fixtures for use.
- this make parallel use of pytest (i

tests: use dynamic ports numbers in pytest suite

- necessary ports are bound at start of test suite and then
given to server fixtures for use.
- this make parallel use of pytest (in separate directories),
practically safe for use as OS tend to not reuse such port numbers
for a while

Closes #10692

show more ...

tests: rename tests/tests-httpd to tests/http

- httpd is only one server we test with
- the suite coveres the HTTP protocol in general where
the default test cases need a more b

tests: rename tests/tests-httpd to tests/http

- httpd is only one server we test with
- the suite coveres the HTTP protocol in general where
the default test cases need a more beefy environment

Closes #10654

show more ...