transfer: do not use EXPIRE_NOW while blocked

- When a transfer sets `data->state.select_bits`, it is
scheduled for rerun with EXPIRE_NOW. If such a transfer
is blocked (due to P

transfer: do not use EXPIRE_NOW while blocked

- When a transfer sets `data->state.select_bits`, it is
scheduled for rerun with EXPIRE_NOW. If such a transfer
is blocked (due to PAUSE, for example), this will lead to
a busy loop.
- multi.c: check for transfer block
- sendf.*: add Curl_xfer_is_blocked()
- sendf.*: add client reader `is_paused()` callback
- implement is_paused()` callback where needed

Closes #13908

show more ...

http: remove "struct HTTP"

It is not actually used anymore and only contained a dummy struct field.
Remove all traces and uses of it.

Closes #13927

lib: xfer_setup and non-blocking shutdown

- clarify Curl_xfer_setup() with RECV/SEND flags and different calls for
which socket they operate on. Add a shutdown flag for secondary

lib: xfer_setup and non-blocking shutdown

- clarify Curl_xfer_setup() with RECV/SEND flags and different calls for
which socket they operate on. Add a shutdown flag for secondary
sockets
- change Curl_xfer_setup() calls to new functions
- implement non-blocking connection shutdown at the end of receiving or
sending a transfer

Closes #13913

show more ...

build: untangle `CURLDEBUG` and `DEBUGBUILD` macros

`CURLDEBUG` is meant to enable memory tracking, but in a bunch of cases,
it was protecting debug features that were supposed to be gua

build: untangle `CURLDEBUG` and `DEBUGBUILD` macros

`CURLDEBUG` is meant to enable memory tracking, but in a bunch of cases,
it was protecting debug features that were supposed to be guarded with
`DEBUGBUILD`.

Replace these uses with `DEBUGBUILD`.

This leaves `CURLDEBUG` uses solely for its intended purpose: to enable
the memory tracking debug feature.

Also:
- autotools: rely on `DEBUGBUILD` to enable `checksrc`.
Instead of `CURLDEBUG`, which worked in most cases because debug
builds enable `CURLDEBUG` by default, but it's not accurate.
- include `lib/easyif.h` instead of keeping a copy of a declaration.
- add CI test jobs for the build issues discovered.

Ref: https://github.com/curl/curl/pull/13694#issuecomment-2120311894
Closes #13718

show more ...

http: write last header line late

- HEADERFUNCTIONS might inspect response properties like
CURLINFO_CONTENT_LENGTH_DOWNLOAD_T on seeing the last header line. If
the line is being

http: write last header line late

- HEADERFUNCTIONS might inspect response properties like
CURLINFO_CONTENT_LENGTH_DOWNLOAD_T on seeing the last header line. If
the line is being written before this is initialized, values are not
available.

- write the last header line late when analyzing a HTTP response so that
all information is available at the time of the writing.

- add test1485 to verify that CURLINFO_CONTENT_LENGTH_DOWNLOAD_T works
on seeing the last header.

Fixes #13752
Reported-by: Harry Sintonen
Closes #13757

show more ...

http: HEAD response body tolerance

- as reported in #13725, some servers wrongly send body bytes in
responses to a HEAD request. This used to be tolerated in curl
8.4 and before

http: HEAD response body tolerance

- as reported in #13725, some servers wrongly send body bytes in
responses to a HEAD request. This used to be tolerated in curl
8.4 and before and leads to failed transfers in newer versions.
- restore previous behaviour for HTTP/1.1 and HTTP/2:
* 1.1: do not add 'Transfer-Encoding' writers from HEAD
responses. RFC 9112 says they do not apply.
* 2: when the transfer expects 'no_body', to not report stream
resets as error when all response headers have been received.

Reported-by: Jeroen Ooms
Fixes #13725
Closes #13732

show more ...

http: remove redundant check

Spotted by CodeSonar

Closes #13582

lib: make protocol handlers store scheme name lowercase

- saves a lowercase operation when the "[scheme]_proxy" name is
generated
- appears less "shouting"
- update test 970, 9

lib: make protocol handlers store scheme name lowercase

- saves a lowercase operation when the "[scheme]_proxy" name is
generated
- appears less "shouting"
- update test 970, 972, 1438 and 1536

Closes #13553

show more ...

http: acknowledge a returned error code

... and do not overwrite it with a new value that could then hide the
problem.

Spotted by CodeSonar

Closes #13434

http: reject HTTP major version switch mid connection

A connection that has seen an HTTP major version now refuses any other
major HTTP version in future responses. Previously, a HTTP/1.

http: reject HTTP major version switch mid connection

A connection that has seen an HTTP major version now refuses any other
major HTTP version in future responses. Previously, a HTTP/1.x
connection would just silently accept HTTP/2 or HTTP/3 in the status
lines as long as it had support for those built-in. It would then just
lead to confusion and badness.

Indirectly Spotted by CodeSonar which identified a duplicate assignment
in this function.

Add test 471 to verify

Closes #13421

show more ...

lib: merge `ENABLE_QUIC` C macro into `USE_HTTP3`

Before this patch `lib/curl_setup.h` defined these two macros right
next to each other, then the source code used them interchangeably.

lib: merge `ENABLE_QUIC` C macro into `USE_HTTP3`

Before this patch `lib/curl_setup.h` defined these two macros right
next to each other, then the source code used them interchangeably.

After this patch, `USE_HTTP3` guards all HTTP/3 / QUIC features.
(Like `USE_HTTP2` does for HTTP/2.) `ENABLE_QUIC` is no longer used.

This patch doesn't change the way HTTP/3 is enabled via autotools
or CMake. Builders who enabled HTTP/3 manually by defining both of
these macros via `CPPFLAGS` can now delete `-DENABLE_QUIC`.

Closes #13352

show more ...

lib: add Curl_xfer_write_resp_hd

Add method in protocol handlers to allow writing of a single,
0-terminated header line. Avoids parsing and copying these lines.

Closes #13165

NTLM_WB: drop support

The feature has not worked for months and has been marked as DEPRECATED
for six+ months.

Closes #13249

urldata: remove fields not used depending on used features

Reduced size of dynamically_allocated_data structure.

Reduced number of stored values in enum dupstring and enum dupblob.

urldata: remove fields not used depending on used features

Reduced size of dynamically_allocated_data structure.

Reduced number of stored values in enum dupstring and enum dupblob. This
affects the reduced array placed in the UserDefined structure.

Closes #13188

show more ...

http: with chunked POST forced, disable length check on read callback

- when an application forces HTTP/1.1 chunked transfer encoding
by setting the corresponding header and instructs

http: with chunked POST forced, disable length check on read callback

- when an application forces HTTP/1.1 chunked transfer encoding
by setting the corresponding header and instructs curl to use
the CURLOPT_READFUNCTION, disregard any POST length information.
- this establishes backward compatibility with previous curl versions

Applications are encouraged to not force "chunked", but rather
set length information for a POST. By setting -1, curl will
auto-select chunked on HTTP/1.1 and work properly on other HTTP
versions.

Reported-by: Jeff King
Fixes #13229
Closes #13257

show more ...

http: remove stale comment about rewindbeforesend

... because that struct field exists no more.

Follow-up to 14bcea074a782272.

Closes #13187

http: separate response parsing from response action

- move code that triggers on end-of-response into separate function from
parsing
- simplify some headp/headerlen usage
- ad

http: separate response parsing from response action

- move code that triggers on end-of-response into separate function from
parsing
- simplify some headp/headerlen usage
- add `httpversion` to SingleRequest to indicate the version of the
current response

Closes #13134

show more ...

http: improve response header handling, save cpu cycles

Saving some cpu cycles in http response header processing:
- pass the length of the header line along
- use string constant si

http: improve response header handling, save cpu cycles

Saving some cpu cycles in http response header processing:
- pass the length of the header line along
- use string constant sizeof() instead of strlen()
- check line length if prefix is possible
- switch on first header char to limit checks

Closes #13143

show more ...

http: expect 100 rework

Move all handling of HTTP's `Expect: 100-continue` feature into a client
reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer
sending on general

http: expect 100 rework

Move all handling of HTTP's `Expect: 100-continue` feature into a client
reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer
sending on general events like a timer.

HTTP installs a `CURL_CR_PROTOCOL` reader when announcing `Expect:
100-continue`. That reader works as follows:

- on first invocation, records time, starts the `EXPIRE_100_TIMEOUT`
timer, disables `KEEP_SEND`, enables `KEEP_SEND_TIMER` and returns 0,
eos=FALSE like a paused upload.

- on subsequent invocation it checks if the timer has expired. If so, it
enables `KEEP_SEND` and switches to passing through reads to the
underlying readers.

Transfer handling's `readwrite()` will be invoked when a timer expires
(like `EXPIRE_100_TIMEOUT`) or when data from the server arrives. Seeing
`KEEP_SEND_TIMER`, it will try to upload more data, which triggers
reading from the client readers again. Which then may lead to a new
pausing or cause the upload to start.

Flags and timestamps connected to this have been moved from
`SingleRequest` into the reader's context.

Closes #13110

show more ...

http: revisit http_perhapsrewind()

- use facilities provided by client readers better
- work also for non-uploading requests like GET/HEAD
- update documentation

Closes #131

http: revisit http_perhapsrewind()

- use facilities provided by client readers better
- work also for non-uploading requests like GET/HEAD
- update documentation

Closes #13117

show more ...

lib: move 'done' parameter to SingleRequests

A transfer may do several `SingleRequest`s for its success. This happens
regularly for authentication, follows and retries on failed connecti

lib: move 'done' parameter to SingleRequests

A transfer may do several `SingleRequest`s for its success. This happens
regularly for authentication, follows and retries on failed connections.
The "readwrite()" calls and functions connected to those carried a `bool
*done` parameter to indicate that the current `SingleRequest` is over.
This may happen before `upload_done` or `download_done` bits of
`SingleRequest` are set.

The problem with that is now `write_resp()` protocol handlers are
invoked in places where the `bool *done` cannot be passed up to the
caller. Instead of being a bool in the call chain, it needs to become a
member of `SingleRequest`, reflecting its state.

This removes the `bool *done` parameter and adds the `done` bit to
`SingleRequest` instead. It adds `Curl_req_soft_reset()` for using a
`SingleRequest` in a follow up, clearing `done` and other
flags/counters.

Closes #13096

show more ...

lib: client reader polish

- seek_func/seek_client, use transfer values only
- remove copies held in `struct connectdata`, use only
ever `data->set.seek_func`
- reso

lib: client reader polish

- seek_func/seek_client, use transfer values only
- remove copies held in `struct connectdata`, use only
ever `data->set.seek_func`
- resolves possible issues in multiuse connections
- new mime post reader eliminates need to ever overwriting this

- websockets, remove empty Curl_ws_done() function

Closes #13079

show more ...

http: fix dead code in setting post client reader

- postsize was always 0, thus the check's else never happened
after the mime client reader was introduced

Follow-up to 0ba471

http: fix dead code in setting post client reader

- postsize was always 0, thus the check's else never happened
after the mime client reader was introduced

Follow-up to 0ba47146f7ff3d
Closes #13060

show more ...

mime: add client reader

Add `mime` client reader. Encapsulates reading from mime parts, getting
their length, rewinding and unpausing.

- remove special mime handling from sendf.

mime: add client reader

Add `mime` client reader. Encapsulates reading from mime parts, getting
their length, rewinding and unpausing.

- remove special mime handling from sendf.c and easy.c
- add general "unpause" method to client readers
- use new reader in http/imap/smtp
- make some mime functions static that are now only used internally

In addition:
- remove flag 'forbidchunk' as no longer needed

Closes #13039

show more ...

http: better error message for HTTP/1.x response without status line

If a response without a status line is received, and the connection is
known to use HTTP/1.x (not HTTP/0.9), report t

http: better error message for HTTP/1.x response without status line

If a response without a status line is received, and the connection is
known to use HTTP/1.x (not HTTP/0.9), report the error "Invalid status
line" instead of "Received HTTP/0.9 when not allowed".

Closes #13045

show more ...