pytest: fixes for recent python, add FTP tests

Fixes:
- in uds tests, abort also silently on os errors
- be conservative on the h3 goaway duration
- detect curl debug build and u

pytest: fixes for recent python, add FTP tests

Fixes:
- in uds tests, abort also silently on os errors
- be conservative on the h3 goaway duration
- detect curl debug build and use in checks
- fix caddy version check for slight difference under linux
- set caddy default path fitting for linux
- fix deprecation warnings in valid time checks

FTP tests:
- add '--with-test-vsftpd=path' to configure
- use vsftpd default path suitable for linux
- add test_30 with plain FTP tests
- add test_31 with --ssl-reqd FTP tests
- add vsftpd to linux GHA for pytest workflows

Closes #13661

show more ...

http tests: in CI skip test_02_23* for quiche

For unknown reasons, these tests fail in CI often, but run fine locally.
Skip them in CI to avoid unrelated PRs to have failures.

C

http tests: in CI skip test_02_23* for quiche

For unknown reasons, these tests fail in CI often, but run fine locally.
Skip them in CI to avoid unrelated PRs to have failures.

Closes #13638

show more ...

http2 + ngtcp2: pass CURLcode errors from callbacks

- errors returned by Curl_xfer_write_resp() and the header variant are
not errors in the protocol. The result needs to be returned o

http2 + ngtcp2: pass CURLcode errors from callbacks

- errors returned by Curl_xfer_write_resp() and the header variant are
not errors in the protocol. The result needs to be returned on the
next recv() from the protocol filter.

- make xfer write errors for response data cause the stream to be
cancelled

- added pytest test_02_14 and test_02_15 to verify that also for
parallel processing

Reported-by: Laramie Leavitt
Fixes #13411
Closes #13424

show more ...

http3: extend download abort tests, fixes in ngtcp2

- fix flow handling in ngtcp2 to ACK data on streams
we abort ourself.
- extend test_02_23* cases to also run for h3
- skip

http3: extend download abort tests, fixes in ngtcp2

- fix flow handling in ngtcp2 to ACK data on streams
we abort ourself.
- extend test_02_23* cases to also run for h3
- skip test_02_23* for OpenSSL QUIC as it gets stalled
on progressing the connection

Closes #13374

show more ...

cw-out: improved error handling

- remember error encountered in invoking write callback and always fail
afterwards without further invokes

- check behaviour in test_02_17 with

cw-out: improved error handling

- remember error encountered in invoking write callback and always fail
afterwards without further invokes

- check behaviour in test_02_17 with h2-pausing client

Reported-by: Pavel Kropachev
Fixes #13337
Closes #13340

show more ...

tidy-up: whitespace [ci skip]

tls: use shared init code for TCP+QUIC

Closes #13172

http2: emit RST when client write fails

- When the writing of response data fails, reset the stream
and do not return a callback error to nghttp2. That would
be a fatal error for

http2: emit RST when client write fails

- When the writing of response data fails, reset the stream
and do not return a callback error to nghttp2. That would
be a fatal error for the connection and harm other requests.
- add test cases for various abort scenarios

Reported-by: Konstantin Kuzov
Fixes #13292
Closes #13298

show more ...

content_encoding: brotli and others, pass through 0-length writes

- curl's transfer handling may write 0-length chunks at the end of the
download with an EOS flag. (HTTP/2 does this co

content_encoding: brotli and others, pass through 0-length writes

- curl's transfer handling may write 0-length chunks at the end of the
download with an EOS flag. (HTTP/2 does this commonly)

- content encoders need to pass-through such a write and not count this
as error in case they are finished decoding

Fixes #13209
Fixes #13212
Closes #13219

show more ...

test_02_download: fix paramters to test_02_27

- it is a special client that only ever uses http/2

Closes #12467

pytest: use lower count in repeat tests

- lower large iteration counts in some tests somewhat for
the same coverage with less duration

Closes #12248

h2: testcase and fix for pausing h2 streams

- refs #11982 where it was noted that paused transfers may
close successfully without delivering the complete data
- made sample poc int

h2: testcase and fix for pausing h2 streams

- refs #11982 where it was noted that paused transfers may
close successfully without delivering the complete data
- made sample poc into tests/http/client/h2-pausing.c and
added test_02_27 to reproduce

Closes #11989
Fixes #11982
Reported-by: Harry Sintonen

show more ...

pytest: improvements

- set CURL_CI for pytest runs in CI environments
- exclude timing sensitive tests from CI runs
- for failed results, list only the log and stat of
the fail

pytest: improvements

- set CURL_CI for pytest runs in CI environments
- exclude timing sensitive tests from CI runs
- for failed results, list only the log and stat of
the failed transfer

- fix type in http.c comment

Closes #11812

show more ...

pytest: fix check for slow_network skips to only apply when intended

Closes #11801

http2: polish things around POST

- added test cases for various code paths
- fixed handling of blocked write when stream had
been closed inbetween attempts
- re-enabled DEBUGAS

http2: polish things around POST

- added test cases for various code paths
- fixed handling of blocked write when stream had
been closed inbetween attempts
- re-enabled DEBUGASSERT on send with smaller data size

- in debug builds, environment variables can be set to simulate a slow
network when sending data. cf-socket.c and vquic.c support
* CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be
answered with a EAGAIN. TCP/UNIX sockets.
This is chosen randomly.
* CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written
to the network. TCP/UNIX sockets.
Example: 80 means a send with 1000 bytes would only send 800
This is applied to every send.
* CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be
answered with EAGAIN. QUIC only.
This is chosen randomly.

Closes #11756

show more ...

wolfssl, use master again in CI

- with the shared session update fix landed in master, it
is time to use that in our CI again

misc: spellfixes

Closes #11730

tests: TLS session sharing test

- test TLS session sharing with special test client
- expect failure with wolfSSL
- disable flaky wolfSSL test_02_07b

Closes #11675

http2: upgrade tests and add fix for non-existing stream

- check in h2 filter recv that stream actually exists
and return error if not
- add test for parallel, extreme h2 upgrades

http2: upgrade tests and add fix for non-existing stream

- check in h2 filter recv that stream actually exists
and return error if not
- add test for parallel, extreme h2 upgrades that fail if
connections get reused before fully switched
- add h2 upgrade upload test just for completeness

Closes #11563

show more ...

CI ngtcp2+quictls: use nghttpx cache as in quiche build

http2: better support for --limit-rate

- leave transfer loop when --limit-rate is in effect and has
been received
- adjust stream window size to --limit-rate plus some slack

http2: better support for --limit-rate

- leave transfer loop when --limit-rate is in effect and has
been received
- adjust stream window size to --limit-rate plus some slack
to make the server observe the pacing we want
- add test case to confirm behaviour

Closes #11115

show more ...

http2: do flow window accounting for cancelled streams

- nghttp2 does not free connection level window flow for
aborted streams
- when closing transfers, make sure that any buffere

http2: do flow window accounting for cancelled streams

- nghttp2 does not free connection level window flow for
aborted streams
- when closing transfers, make sure that any buffered
response data is "given back" to the flow control window
- add tests test_02_22 and test_02_23 to reproduce

Closes #11052

show more ...

tests/http: more tests with specific clients

- Makefile support for building test specific clients in tests/http/clients
- auto-make of clients when invoking pytest
- added test_09_0

tests/http: more tests with specific clients

- Makefile support for building test specific clients in tests/http/clients
- auto-make of clients when invoking pytest
- added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush
- added test_02_21 for lib based downloads and pausing/unpausing transfers

curl url parser:
- added internal method `curl_url_set_authority()` for setting the
authority part of a url (used for PUSH_PROMISE)

http2:
- made logging of PUSH_PROMISE handling nicer

Placing python test requirements in requirements.txt files
- separate files to base test suite and http tests since use
and module lists differ
- using the files in the gh workflows

websocket test cases, fixes for we and bufq
- bufq: account for spare chunks in space calculation
- bufq: reset chunks that are skipped empty
- ws: correctly encode frames with 126 bytes payload
- ws: update frame meta information on first call of collect
callback that fills user buffer
- test client ws-data: some test/reporting improvements

Closes #11006

show more ...

proxy: http2 proxy tunnel implementation

- currently only on debug build and when env variable
CURL_PROXY_TUNNEL_H2 is present.
- will ALPN negotiate with the proxy server and swit

proxy: http2 proxy tunnel implementation

- currently only on debug build and when env variable
CURL_PROXY_TUNNEL_H2 is present.
- will ALPN negotiate with the proxy server and switch
tunnel filter based on the protocol negotiated.
- http/1.1 tunnel code moved into cf-h1-proxy.[ch]
- http/2 tunnel code implemented in cf-h2-proxy.[ch]
- tunnel start and ALPN set remains in http_proxy.c
- moving all haproxy related code into cf-haproxy.[ch]

VTLS changes
- SSL filters rely solely on the "alpn" specification they
are created with and no longer check conn->bits.tls_enable_alpn.
- checks on which ALPN specification to use (or none at all) are
done in vtls.c when creating the filter.

Testing
- added a nghttpx forward proxy to the pytest setup that
speaks HTTP/2 and forwards all requests to the Apache httpd
forward proxy server.
- extending test coverage in test_10 cases
- adding proxy tests for direct/tunnel h1/h2 use of basic auth.
- adding test for http/1.1 and h2 proxy tunneling to pytest

Closes #10780

show more ...

http3: improvements across backends

- ngtcp2: using bufq for recv stream data
- internal stream_ctx instead of `struct HTTP` members
for quiche, ngtcp2 and msh3
- no more QUIC

http3: improvements across backends

- ngtcp2: using bufq for recv stream data
- internal stream_ctx instead of `struct HTTP` members
for quiche, ngtcp2 and msh3
- no more QUIC related members in `struct HTTP`
- experimental use of recvmmsg(), disabled by default
- testing on my old debian box shows no throughput improvements.
- leaving it in, but disabled, for future revisit
- vquic: common UDP receive code for ngtcp2 and quiche
- vquic: common UDP send code for ngtcp2 and quiche
- added pytest skips for known msh3 failures
- fix unit2601 to survive torture testing
- quiche: using latest `master` from quiche and enabling large download
tests, now that key change is supported
- fixing test_07_21 where retry handling of starting a stream
was faulty
- msh3: use bufq for recv buffering headers and data
- msh3: replace fprintf debug logging with LOG_CF where possible
- msh3: force QUIC expire timers on recv/send to have more than
1 request per second served

Closes #10772

show more ...