http2: auto reset stream on server eos

When a server signals EOS from its side and the curl upload is
unfinished and the server has not given a positive HTTP status response,
auto RS

http2: auto reset stream on server eos

When a server signals EOS from its side and the curl upload is
unfinished and the server has not given a positive HTTP status response,
auto RST the stream to signal that the upload is incomplete and that the
whole transfer can be stopped.

Fixes the case where the server responds with 413 on an upload but does
not RST the stream from its side, as httpd and others do.

Reported-by: jkamp-aws on github
Fixes #15316
Closes #15325

show more ...

TLS: TLSv1.3 earlydata support for curl

Based on #14135, implement TLSv1.3 earlydata support for the curl
command line, libcurl and its implementation in GnuTLS.

If a known TLS

TLS: TLSv1.3 earlydata support for curl

Based on #14135, implement TLSv1.3 earlydata support for the curl
command line, libcurl and its implementation in GnuTLS.

If a known TLS session announces early data support, and the feature is
enabled *and* it is not a "connect-only" transfer, delay the TLS
handshake until the first request is being sent.

- Add --tls-earldata as new boolean command line option for curl.
- Add CURLSSLOPT_EARLYDATA to libcurl to enable use of the feature.
- Add CURLINFO_EARLYDATA_SENT_T to libcurl, reporting the amount of
bytes sent and accepted/rejected by the server.

Implementation details:
- store the ALPN protocol selected at the SSL session.
- When reusing the session and enabling earlydata, use exactly
that ALPN protocol for negoptiation with the server. When the
sessions ALPN does not match the connections ALPN, earlydata
will not be enabled.
- Check that the server selected the correct ALPN protocol for
an earlydata connect. If the server does not confirm or reports
something different, the connect fails.
- HTTP/2: delay sending the initial SETTINGS frames during connect,
if not connect-only.

Verification:
- add test_02_32 to verify earlydata GET with nghttpx.
- add test_07_70 to verify earlydata PUT with nghttpx.
- add support in 'hx-download', 'hx-upload' clients for the feature

Assisted-by: ad-chaos on github
Closes #15211

show more ...

tests: change Python code style to pass ruff checks

Most of the changes consisted of removing unused imports and unnecessary
f-strings.

tests: fix some Python typing issues

These otherwise raise errors in pytype. A few problematic methods
weren't being used and are deleted.

pytest: improve pytest_07_42a reliability

Due to timings on paused response receive, the error code may vary due
to the location where it is detected that the server closed the transfer

pytest: improve pytest_07_42a reliability

Due to timings on paused response receive, the error code may vary due
to the location where it is detected that the server closed the transfer
prematurely.

Be more lenient in always allowing PARTIAL_FILE as ok.

Closes #15138

show more ...

request: correctly reset the eos_sent flag

Add test cases

Bug: https://marc.info/?l=git&m=172620452502747&w=2
Reported-by: Patrick Steinhardt
Closes #14895

http2: when uploading data from stdin, fix eos forwarding

When uploading data from stdin ('-T -'), and the EOS was only detected
on a 0-length read, the EOS was not forwarded to the filt

http2: when uploading data from stdin, fix eos forwarding

When uploading data from stdin ('-T -'), and the EOS was only detected
on a 0-length read, the EOS was not forwarded to the filters. This led
HTTP/2 to hang on not forwarding this to the server.

Added test_07_14 to reproduce and verify.

Fixes #14870
Reported-by: nekopsykose on github
Closes #14877

show more ...

pytest: tweak counts

Lower some iteration counts for tests where they do not seem to add
value. Raise speed limits somewhat to shave off a second or two.

Closes #14689

http2: improved upload eos handling

- replace the counting of upload lengths with the new eos send flag
- improve frequency of stream draining to happen less on events where it
is

http2: improved upload eos handling

- replace the counting of upload lengths with the new eos send flag
- improve frequency of stream draining to happen less on events where it
is not needed
- this PR is based on #14220

http2, cf-h2-proxy: fix EAGAINed out buffer
- in adjust pollset and shutdown handling, a non-empty `ctx->outbufq`
must trigger send polling, irregardless of http/2 flow control
- in http2, fix retry handling of blocked GOAWAY frame

test case improvement:
- let client 'upload-pausing' handle http versions

Closes #14253

show more ...

transfer: conn close on paused upload

- add 2 variations on test_07_42 which PAUSEs uploads
and response connections terminating either right away
or after the 100-continue respo

transfer: conn close on paused upload

- add 2 variations on test_07_42 which PAUSEs uploads
and response connections terminating either right away
or after the 100-continue response
- when detecting the connection being closed in transfer.c
readwrite_data(), clear ALL send bits in data->req.keepon.
It no longer makes send to wait for a KEEP_SEND_PAUSE or HOLD.
- in the protocol client writer add the check for incomplete
response bodies. When an EOS is seen and the length is known,
check that and fail if bytes are missing.

Reported-by: Sergey Bronnikov
Fixes #13740
Closes #13750

show more ...

tidy-up: whitespace [ci skip]

request: paused upload on completed download, assess connection

A transfer with a completed download that is still uploading needs to
check the connection state when it is PAUSEd, since

request: paused upload on completed download, assess connection

A transfer with a completed download that is still uploading needs to
check the connection state when it is PAUSEd, since connection
close/errors would otherwise go unnoticed.

Reported-by: Sergey Bronnikov
Fixes #13260
Closes #13271

show more ...

http: expect 100 rework

Move all handling of HTTP's `Expect: 100-continue` feature into a client
reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer
sending on general

http: expect 100 rework

Move all handling of HTTP's `Expect: 100-continue` feature into a client
reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer
sending on general events like a timer.

HTTP installs a `CURL_CR_PROTOCOL` reader when announcing `Expect:
100-continue`. That reader works as follows:

- on first invocation, records time, starts the `EXPIRE_100_TIMEOUT`
timer, disables `KEEP_SEND`, enables `KEEP_SEND_TIMER` and returns 0,
eos=FALSE like a paused upload.

- on subsequent invocation it checks if the timer has expired. If so, it
enables `KEEP_SEND` and switches to passing through reads to the
underlying readers.

Transfer handling's `readwrite()` will be invoked when a timer expires
(like `EXPIRE_100_TIMEOUT`) or when data from the server arrives. Seeing
`KEEP_SEND_TIMER`, it will try to upload more data, which triggers
reading from the client readers again. Which then may lead to a new
pausing or cause the upload to start.

Flags and timestamps connected to this have been moved from
`SingleRequest` into the reader's context.

Closes #13110

show more ...

pytest: Scorecard tracking CPU and RSS

Closes #12765

transfer: fix upload rate limiting, add test cases

- add test cases for rate limiting uploads for all
http versions
- fix transfer loop handling of limits. Signal a re-receive

transfer: fix upload rate limiting, add test cases

- add test cases for rate limiting uploads for all
http versions
- fix transfer loop handling of limits. Signal a re-receive
attempt only on exhausting maxloops without an EAGAIN
- fix `data->state.selectbits` forcing re-receive to also
set re-sending when transfer is doing this.

Reported-by: Karthikdasari0423 on github
Fixes #12559
Closes #12586

show more ...

http2: improved on_stream_close/data_done handling

- there seems to be a code path that cleans up easy handles without
triggering DONE or DETACH events to the connection filters. This

http2: improved on_stream_close/data_done handling

- there seems to be a code path that cleans up easy handles without
triggering DONE or DETACH events to the connection filters. This
would explain wh nghttp2 still holds stream user data
- add GOOD check to easy handle used in on_close_callback to
prevent crashes, ASSERTs in debug builds.
- NULL the stream user data early before submitting RST
- add checks in on_stream_close() to identify UNGOOD easy handles

Reported-by: Hans-Christian Egtvedt
Fixes #10936
Closes #12562

show more ...

pytest: use lower count in repeat tests

- lower large iteration counts in some tests somewhat for
the same coverage with less duration

Closes #12248

http: fix sending of large requests

- refs #11342 where errors with git https interactions
were observed
- problem was caused by 1st sends of size larger than 64KB
which resu

http: fix sending of large requests

- refs #11342 where errors with git https interactions
were observed
- problem was caused by 1st sends of size larger than 64KB
which resulted in later retries of 64KB only
- limit sending of 1st block to 64KB
- adjust h2/h3 filters to cope with parsing the HTTP/1.1
formatted request in chunks

- introducing Curl_nwrite() as companion to Curl_write()
for the many cases where the sockindex is already known

Fixes #11342 (again)
Closes #11803

show more ...

http2: polish things around POST

- added test cases for various code paths
- fixed handling of blocked write when stream had
been closed inbetween attempts
- re-enabled DEBUGAS

http2: polish things around POST

- added test cases for various code paths
- fixed handling of blocked write when stream had
been closed inbetween attempts
- re-enabled DEBUGASSERT on send with smaller data size

- in debug builds, environment variables can be set to simulate a slow
network when sending data. cf-socket.c and vquic.c support
* CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be
answered with a EAGAIN. TCP/UNIX sockets.
This is chosen randomly.
* CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written
to the network. TCP/UNIX sockets.
Example: 80 means a send with 1000 bytes would only send 800
This is applied to every send.
* CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be
answered with EAGAIN. QUIC only.
This is chosen randomly.

Closes #11756

show more ...

tidy-up: mostly whitespace nits

- delete completed TODO from `./CMakeLists.txt`.
- convert a C++ comment to C89 in `./CMake/CurlTests.c`.
- delete duplicate EOLs from EOF.
- add

tidy-up: mostly whitespace nits

- delete completed TODO from `./CMakeLists.txt`.
- convert a C++ comment to C89 in `./CMake/CurlTests.c`.
- delete duplicate EOLs from EOF.
- add missing EOL at EOF.
- delete whitespace at EOL (except from expected test results).
- convert tabs to spaces.
- convert CRLF EOLs to LF in GHA yaml.
- text casing fixes in `./CMakeLists.txt`.
- fix a codespell typo in `packages/OS400/initscript.sh`.

Closes #11772

show more ...

test_07_upload.py: fix test_07_34 curl args

- Pass correct filename to --data-binary.

Prior to this change --data-binary was passed an incorrect filename due
to a missing separa

test_07_upload.py: fix test_07_34 curl args

- Pass correct filename to --data-binary.

Prior to this change --data-binary was passed an incorrect filename due
to a missing separator in the arguments list. Since aacbeae7 curl will
error on incorrect filenames for POST.

Fixes https://github.com/curl/curl/issues/11761
Closes https://github.com/curl/curl/pull/11763

show more ...

http2: upgrade tests and add fix for non-existing stream

- check in h2 filter recv that stream actually exists
and return error if not
- add test for parallel, extreme h2 upgrades

http2: upgrade tests and add fix for non-existing stream

- check in h2 filter recv that stream actually exists
and return error if not
- add test for parallel, extreme h2 upgrades that fail if
connections get reused before fully switched
- add h2 upgrade upload test just for completeness

Closes #11563

show more ...

http2: fix EOF handling on uploads with auth negotiation

- doing a POST with `--digest` does an override on the initial request
with `Content-Length: 0`, but the http2 filter was unawa

http2: fix EOF handling on uploads with auth negotiation

- doing a POST with `--digest` does an override on the initial request
with `Content-Length: 0`, but the http2 filter was unaware of that
and expected the originally request body. It did therefore not
send a final DATA frame with EOF flag to the server.
- The fix overrides any initial notion of post size when the `done_send`
event is triggered by the transfer loop, leading to the EOF that
is necessary.
- refs #11194. The fault did not happen in testing, as Apache httpd
never tries to read the request body of the initial request,
sends the 401 reply and closes the stream. The server used in the
reported issue however tried to read the EOF and timed out on the
request.

Reported-by: Aleksander Mazur
Fixes #11194
Cloes #11200

show more ...

http/2: unstick uploads

- refs #11157 and #11175 where uploads get stuck or lead to RST streams
- fixes our h2 send behaviour to continue sending in the nghttp2 session
as long as

http/2: unstick uploads

- refs #11157 and #11175 where uploads get stuck or lead to RST streams
- fixes our h2 send behaviour to continue sending in the nghttp2 session
as long as it wants to. This will empty our send buffer as long as
the remote stream/connection window allows.
- in case the window is exhausted, the data remaining in the send buffer
will wait for a WINDOW_UPDATE from the server. Which is a socket event
that engages our transfer loop again
- the problem in the issue was that we did not exhaust the window, but
left data in the sendbuffer and no further socket events did happen.
The server was just waiting for us to send more.
- relatedly, there was an issue fixed that closing a stream with KEEP_HOLD
set kept the transfer from shutting down - as it should have - leading
to a timeout.

Closes #11176

show more ...

http2: upload improvements

Make send buffer smaller to have progress and "upload done" reporting
closer to reality. Fix handling of send "drain" condition to no longer
trigger once t

http2: upload improvements

Make send buffer smaller to have progress and "upload done" reporting
closer to reality. Fix handling of send "drain" condition to no longer
trigger once the transfer loop reports it is done sending. Also do not
trigger the send "drain" on RST streams.

Background:
- a upload stall was reported in #11157 that timed out
- test_07_33a reproduces a problem with such a stall if the
server 404s the request and RSTs the stream.
- test_07_33b verifies a successful PUT, using the parameters
from #11157 and checks success

Ref: #11157
Closes #11165

show more ...