request: on shutdown send, proceed normally on timeout

When ending an FTP upload, we shut down the connection gracefully, since
the server should be notified we had send all bytes. Mostl

request: on shutdown send, proceed normally on timeout

When ending an FTP upload, we shut down the connection gracefully, since
the server should be notified we had send all bytes. Mostly, this is a
NOP without TLS involved. With TLS, close-notify messages should be
exchanged.

As reported in #14843, not all servers seem to do that. Since it is the
server's responsiblity to check it has received everything, we just log
the timeout and proceed as if everything is fine.

In the receive direction, we still fail the transfer if the server does
not shut down its direction properly.

Fixes #14843
Reported-by: Rasmus Melchior Jacobsen
Closes #14848

show more ...

lib: fix unity builds with BearSSL, MSH3, Quiche, OmniOS

- fix MSH3 static symbol clash.
- fix Quiche static symbol clash.
- fix local macro clash with BearSSL header.
- fix loca

lib: fix unity builds with BearSSL, MSH3, Quiche, OmniOS

- fix MSH3 static symbol clash.
- fix Quiche static symbol clash.
- fix local macro clash with BearSSL header.
- fix local macro clash with OmniOS system header.
```
In file included from ../../lib/urldata.h:197,
from ../../lib/altsvc.c:32,
from libcurlall.c:2:
../../lib/cf-socket.h:55:25: error: expected ':', ',', ';', '}' or '__attribute__' before '.' token
55 | #define sa_addr _sa_ex_u.addr
| ^
In file included from ../../lib/urldata.h:197,
from ../../lib/altsvc.c:32,
from libcurlall.c:2:
../../lib/cf-socket.h:55:25: error: expected ':', ',', ';', '}' or '__attribute__' before '.' token
55 | #define sa_addr _sa_ex_u.addr
| ^
```
Ref: https://github.com/curl/curl/actions/runs/10738314933/job/29781644299?pr=14772#step:3:6115

Discovered while adding support for "unity" builds for autotools.

Required-by: #14922
Cherry-picked from #14815
Closes #14932

show more ...

lib/src: white space edits to comply better with code style

... as checksrc now finds and complains about these.

Closes #14921

lib, src, tests: added space around ternary expressions

Closes #14912

FTP: partly revert eeb7c1280742f5c8fa48a4340fc1e1a1a2c7075a

Since ASCII transfers on FTP means sending CRLF line endings, we should
still keep converting them to LF-only on platforms whe

FTP: partly revert eeb7c1280742f5c8fa48a4340fc1e1a1a2c7075a

Since ASCII transfers on FTP means sending CRLF line endings, we should
still keep converting them to LF-only on platforms where text files
typically do not use CRLF.

This also DOES NOT convert existing CRLF line endings on ASCII uploads
but only does stand-alone LF => CRLF.

Regression from eeb7c1280742f5c8 shipped in 8.10.0

Reported-by: finkjsc on github
Fixes #14873
Closes #14875

show more ...

lib: use FMT_ as prefix instead of CURL_FORMAT_

For printf format defines used internally. Makes the code slighly
easier to read.

Closes #14764

ftp: always offer line end conversions

Previously this functionality was limited to platforms that not already
use CRLF as native line endings.

TODO: 4.5 ASCII support now consi

ftp: always offer line end conversions

Previously this functionality was limited to platforms that not already
use CRLF as native line endings.

TODO: 4.5 ASCII support now considered fixed

Closes #14717

show more ...

urldata: remove crlf_conversions counter

Since the introduction of client writers, we check the body length in
the PROTOCOL phase and do FTP lineend conversions laster in the
CONTENT

urldata: remove crlf_conversions counter

Since the introduction of client writers, we check the body length in
the PROTOCOL phase and do FTP lineend conversions laster in the
CONTENT_DECODING phase. This means we no longer need to count the
conversions for length checks.

Closes #14709

show more ...

connect: limit update IP info

Update IP related information at the connection and the transfer in two
places only: once the filter chain connects and when a transfer is added
to a co

connect: limit update IP info

Update IP related information at the connection and the transfer in two
places only: once the filter chain connects and when a transfer is added
to a connection. The latter only updates on reuse when the filters
already are connected.

The only user of that information before a full connect is the HAProxy
filter. Add cfilter CF_QUERY_IP_INFO query to let it find the
information from the filters "below".

This solves two issues with the previous version:
- updates where often done twice with the same info
- happy eyeballing filter "forks" could overwrite each others
updates before the full winner was determined.

Closes #14699

show more ...

tidy-up: misc spelling (bit, ASCII)

Closes #14559

llist: remove direct struct accesses, use only functions

- Turned them all into functions to also do asserts etc.

- The llist related structs got all their fields renamed in order t

llist: remove direct struct accesses, use only functions

- Turned them all into functions to also do asserts etc.

- The llist related structs got all their fields renamed in order to make
sure no existing code remains using direct access.

- Each list node struct now points back to the list it "lives in", so
Curl_node_remove() no longer needs the list pointer.

- Rename the node struct and some of the access functions.

- Added lots of ASSERTs to verify API being used correctly

- Fix some cases of API misuse

Add docs/LLIST.md documenting the internal linked list API.

Closes #14485

show more ...

ftp: flush pingpong before response

Fix FTP protocol to flush the pingpong's send buffer before receiving a
response from the server, as it may never come otherwise.

Fixes FTP/F

ftp: flush pingpong before response

Fix FTP protocol to flush the pingpong's send buffer before receiving a
response from the server, as it may never come otherwise.

Fixes FTP/FTPS tests with `CURL_DBG_SOCK_WBLOCK=90` set.

Closes #14452

show more ...

url: dns_entry related improvements

Replace Curl_resolv_unlock() with Curl_resolv_unlink():

-replace inuse member with refcount in Curl_dns_entry

- pass Curl_dns_entry ** t

url: dns_entry related improvements

Replace Curl_resolv_unlock() with Curl_resolv_unlink():

-replace inuse member with refcount in Curl_dns_entry

- pass Curl_dns_entry ** to unlink, so it gets always cleared

- solve potential (but unlikley) UAF in FTP's handling of looked up
Curl_dns_entry. Esp. do not use addr information after unlinking an entry.
In reality, the unlink will not free memory, as the dns entry is still
referenced by the hostcache. But this is not safe and relying on no other
code pruning the cache in the meantime.

- pass permanent flag when adding a dns entry instead of fixing timestamp
afterwards.

url.c: fold several static *resolve_* functions into one.

Closes #14195

show more ...

ftp: remove redundant null pointer check in loop condition

Closes #14234

code: language cleanup in comments

Based on the standards and guidelines we use for our documentation.

- expand contractions (they're => they are etc)
- host name = > hostname

code: language cleanup in comments

Based on the standards and guidelines we use for our documentation.

- expand contractions (they're => they are etc)
- host name = > hostname
- file name => filename
- user name = username
- man page => manpage
- run-time => runtime
- set-up => setup
- back-end => backend
- a HTTP => an HTTP
- Two spaces after a period => one space after period

Closes #14073

show more ...

lib: xfer_setup and non-blocking shutdown

- clarify Curl_xfer_setup() with RECV/SEND flags and different calls for
which socket they operate on. Add a shutdown flag for secondary

lib: xfer_setup and non-blocking shutdown

- clarify Curl_xfer_setup() with RECV/SEND flags and different calls for
which socket they operate on. Add a shutdown flag for secondary
sockets
- change Curl_xfer_setup() calls to new functions
- implement non-blocking connection shutdown at the end of receiving or
sending a transfer

Closes #13913

show more ...

connection: shutdown TLS (for FTP) better

This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a

connection: shutdown TLS (for FTP) better

This adds connection shutdown infrastructure and first use for FTP. FTP
data connections, when not encountering an error, are now shut down in a
blocking way with a 2sec timeout.

- add cfilter `Curl_cft_shutdown` callback
- keep a shutdown start timestamp and timeout at connectdata
- provide shutdown timeout default and member in
`data->set.shutdowntimeout`.
- provide methods for starting, interrogating and clearing
shutdown timers
- provide `Curl_conn_shutdown_blocking()` to shutdown the
`sockindex` filter chain in a blocking way. Use that in FTP.
- add `Curl_conn_cf_poll()` to wait for socket events during
shutdown of a connection filter chain.
This gets the monitoring sockets and events via the filters
"adjust_pollset()" methods. This gives correct behaviour when
shutting down a TLS connection through a HTTP/2 proxy.
- Implement shutdown for all socket filters
- for HTTP/2 and h2 proxying to send GOAWAY
- for TLS backends to the best of their capabilities
- for tcp socket filter to make a final, nonblocking
receive to avoid unwanted RST states
- add shutdown forwarding to happy eyeballers and
https connect ballers when applicable.

Closes #13904

show more ...

cf-socket: improve SO_SNDBUF update for Winsock

- Rename: Curl_sndbufset => Curl_sndbuf_init

- Rename: win_update_buffer_size => win_update_sndbuf_size

- Save the last set

cf-socket: improve SO_SNDBUF update for Winsock

- Rename: Curl_sndbufset => Curl_sndbuf_init

- Rename: win_update_buffer_size => win_update_sndbuf_size

- Save the last set SO_SNDBUF size to compare against so that we can
avoid setsockopt calls every second.

This is a follow-up to 0b520e12 which moved the SO_SNDBUF update check
into cf-socket. This change improves it further by making the function
names easier to understand and reducing the amount of setsockopt calls.

Closes https://github.com/curl/curl/pull/13827

show more ...

ftp: fix build for CURL_DISABLE_VERBOSE_STRINGS

This is a follow-up to b7c7dffe which changed the FTP state change
verbose debug text (aka infof) to tracing debug text (aka trc).

ftp: fix build for CURL_DISABLE_VERBOSE_STRINGS

This is a follow-up to b7c7dffe which changed the FTP state change
verbose debug text (aka infof) to tracing debug text (aka trc).

Prior to this change if libcurl was without DEBUGBUILD and built with
CURL_DISABLE_VERBOSE_STRINGS (ie --disable-verbose) the build would
error.

Caught by Circle CI job openssl-no-verbose.

show more ...

ftp: add tracing support

- add `Curl_trc_feat_ftp` for tracing via trace config
- add macro CURL_TRC_FTP(data, fmt, ...)
- replace DEBUGF(infof()) statements in ftp.c by CURL_TRC_FTP

ftp: add tracing support

- add `Curl_trc_feat_ftp` for tracing via trace config
- add macro CURL_TRC_FTP(data, fmt, ...)
- replace DEBUGF(infof()) statements in ftp.c by CURL_TRC_FTP()
- always trace FTP connection state

Closes #13580

show more ...

lib: make protocol handlers store scheme name lowercase

- saves a lowercase operation when the "[scheme]_proxy" name is
generated
- appears less "shouting"
- update test 970, 9

lib: make protocol handlers store scheme name lowercase

- saves a lowercase operation when the "[scheme]_proxy" name is
generated
- appears less "shouting"
- update test 970, 972, 1438 and 1536

Closes #13553

show more ...

ftp: fix socket leak on rare error

In the function AcceptServerConnect() the newly created socket would
leak if Curl_conn_tcp_accepted_set() returns error. Which basically
should nev

ftp: fix socket leak on rare error

In the function AcceptServerConnect() the newly created socket would
leak if Curl_conn_tcp_accepted_set() returns error. Which basically
should never happen.

Spotted by CodeSonar.

Closes #13417

show more ...

build: prefer `USE_IPV6` macro internally (was: `ENABLE_IPV6`)

Before this patch, two macros were used to guard IPv6 features in curl
sources: `ENABLE_IPV6` and `USE_IPV6`. This patch ma

build: prefer `USE_IPV6` macro internally (was: `ENABLE_IPV6`)

Before this patch, two macros were used to guard IPv6 features in curl
sources: `ENABLE_IPV6` and `USE_IPV6`. This patch makes the source use
the latter for consistency with other similar switches.

`-DENABLE_IPV6` remains accepted for compatibility as a synonym for
`-DUSE_IPV6`, when passed to the compiler.

`ENABLE_IPV6` also remains the name of the CMake and `Makefile.vc`
options to control this feature.

Closes #13349

show more ...

lib: add Curl_xfer_write_resp_hd

Add method in protocol handlers to allow writing of a single,
0-terminated header line. Avoids parsing and copying these lines.

Closes #13165

lib: keep conn IP information together

new struct ip_quadruple for holding local/remote addr+port

- used in data->info and conn and cf-socket.c
- copy back and forth complete st

lib: keep conn IP information together

new struct ip_quadruple for holding local/remote addr+port

- used in data->info and conn and cf-socket.c
- copy back and forth complete struct
- add 'secondary' to conn
- use secondary in reporting success for ftp 2nd connection

Reported-by: DasKutti on github
Fixes #13084
Closes #13090

show more ...