windows: simplify detecting and using system headers

- autotools, cmake: assume that if we detect Windows, `windows.h`,
`winsock2.h` and `ws2tcpip.h` do exist.
- lib: fix 3 outlier

windows: simplify detecting and using system headers

- autotools, cmake: assume that if we detect Windows, `windows.h`,
`winsock2.h` and `ws2tcpip.h` do exist.
- lib: fix 3 outlier `#if` conditions to use `USE_WINSOCK` instead of
looking for `winsock2.h`.
- autotools: merge 3 Windows check methods into one.
- move Watt-32 and lwIP socket support to `setup-win32.h` from
`config-win32.h`. It opens up using these with all build tools. Also
merge logic with Windows Sockets.
- fix to assume Windows sockets with the mingw32ce toolchain.
Follow-up to: 2748c64d605b19fb419ae56810ad8da36487a2d4
- cmake: delete unused variable `signature_call_conv` since
eb33ccd5332435fa50f1758e5debb869c6942b7f.
- autotools: simplify `CURL_CHECK_WIN32_LARGEFILE` detection.
- examples/externalsocket: fix header order.
- cmake/OtherTests.cmake: delete Windows-specific `_source_epilogue`
that wasn't used anymore.
- cmake/OtherTests.cmake: set `WIN32_LEAN_AND_MEAN` for test
`SIZEOF_STRUCT_SOCKADDR_STORAGE`.

After this patch curl universally uses `_WIN32` to guard
Windows-specific logic. It guards Windows Sockets-specific logic with
`USE_WINSOCK` (this might need further work).

Reviewed-by: Jay Satiro
Closes #12495

show more ...

build: enable missing OpenSSF-recommended warnings, with fixes

https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html
as of 2023-11-29 [1]

build: enable missing OpenSSF-recommended warnings, with fixes

https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html
as of 2023-11-29 [1].

Enable new recommended warnings (except `-Wsign-conversion`):

- enable `-Wformat=2` for clang (in both cmake and autotools).
- add `CURL_PRINTF()` internal attribute and mark functions accepting
printf arguments with it. This is a copy of existing
`CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible
with redefinting the `printf` symbol:
https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94
- fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for
mingw-w64 and enable it on this platform.
- enable `-Wimplicit-fallthrough`.
- enable `-Wtrampolines`.
- add `-Wsign-conversion` commented with a FIXME.
- cmake: enable `-pedantic-errors` the way we do it with autotools.
Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747
- lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format
checks. Previously it was always disabled due to the internal `printf`
macro.

Fix them:

- fix bug where an `set_ipv6_v6only()` call was missed in builds with
`--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`.
- add internal `FALLTHROUGH()` macro.
- replace obsolete fall-through comments with `FALLTHROUGH()`.
- fix fallthrough markups: Delete redundant ones (showing up as
warnings in most cases). Add missing ones. Fix indentation.
- silence `-Wformat-nonliteral` warnings with llvm/clang.
- fix one `-Wformat-nonliteral` warning.
- fix new `-Wformat` and `-Wformat-security` warnings.
- fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its
definition to `lib/curl_setup.h` allowing use in `tests/server`.
- lib: fix two wrongly passed string arguments in log outputs.
Co-authored-by: Jay Satiro
- fix new `-Wformat` warnings on mingw-w64.

[1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895bfc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C%2B%2B.md

Closes #12489

show more ...

Makefile.mk: drop Windows support

And DLL-support with it. This leaves `Makefile.mk` for MS-DOS and Amiga.

We recommend CMake instead. With unity mode it's much faster, and about

Makefile.mk: drop Windows support

And DLL-support with it. This leaves `Makefile.mk` for MS-DOS and Amiga.

We recommend CMake instead. With unity mode it's much faster, and about
the same without.

Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806
Reviewed-by: Daniel Stenberg
Closes #12224

show more ...

cmdline-docs: use .IP consistently

Remove use of .TP and some .B. The idea is to reduce nroff syntax as
much as possible and to use it consistently. Ultimately, we should be
able to

cmdline-docs: use .IP consistently

Remove use of .TP and some .B. The idea is to reduce nroff syntax as
much as possible and to use it consistently. Ultimately, we should be
able to introduce our own easier-to-use-and-read syntax/formatting and
convert on generation time.

Closes #12535

show more ...

http: fix off-by-one error in request method length check

It should allow one more byte.

Closes #12534

curl: show ipfs and ipns as supported "protocols"

They are accepted schemes in URLs passed to curl (the tool, not the
library).

Also makes curl-config show the same list.

curl: show ipfs and ipns as supported "protocols"

They are accepted schemes in URLs passed to curl (the tool, not the
library).

Also makes curl-config show the same list.

Co-Authored-by: Jay Satiro
Reported-by: Chara White
Bug: https://curl.se/mail/archive-2023-12/0026.html
Closes #12508

show more ...

Revert "urldata: move async resolver state from easy handle to connectdata"

This reverts commit 56a4db2e4e2bcb9a0dcb75b83560a78ef231fcc8 (#12198)

We want the c-ares channel to be he

Revert "urldata: move async resolver state from easy handle to connectdata"

This reverts commit 56a4db2e4e2bcb9a0dcb75b83560a78ef231fcc8 (#12198)

We want the c-ares channel to be held in the easy handle, not per
connection - for performance.

Closes #12524

show more ...

openssl: re-match LibreSSL deinit with init

Earlier we switched to use modern initialization with LibreSSL v2.7.0
and up, but did not touch deinitialization [1]. Fix it in this patch.

openssl: re-match LibreSSL deinit with init

Earlier we switched to use modern initialization with LibreSSL v2.7.0
and up, but did not touch deinitialization [1]. Fix it in this patch.

Regression from bec0c5bbf34369920598678161d2df8bea0e243b #11611

[1] https://github.com/curl/curl/pull/11611#issuecomment-1668654014

Reported-by: Mike Hommey
Reviewed-by: Daniel Stenberg
Fixes #12525
Closes #12526

show more ...

libssh: supress warnings without version check

Define unconditionally.

Follow-up from d21bd2190c46ad7fa

Closes #12523

hostip: return error immediately when Curl_ip2addr() fails

Closes #12522

libssh: improve the deprecation warning dismissal

Previous code was compiler dependant, and dismissed all deprecation warnings
indiscriminately.

libssh provides a way to disable

libssh: improve the deprecation warning dismissal

Previous code was compiler dependant, and dismissed all deprecation warnings
indiscriminately.

libssh provides a way to disable the deprecation warnings for libssh only, and
naturally this is the preferred way.

This commit uses that, to prevent the erroneous hiding of potential, unrelated
deprecation warnings.

Fixes #12519
Closes #12520

show more ...

test1474: removed

The test was already somewhat flaky and disabled on several platforms,
and after 1da640abb688 even more unstable.

readwrite_data: loop less

This function is made to loop in order to drain incoming data
faster. Completely removing the loop has a measerably negative impact on
transfer speeds.

readwrite_data: loop less

This function is made to loop in order to drain incoming data
faster. Completely removing the loop has a measerably negative impact on
transfer speeds.

Downsides with the looping include

- it might call the progress callback much more seldom. Especially if
the write callback is slow.

- rate limiting becomes less exact

- a single transfer might "starve out" other parallel transfers

- QUIC timers for other connections can't be maintained correctly

The long term fix should be to remove the loop and optimize coming back
to avoid the transfer speed penalty.

This fix lower the max loop count to reduce the starvation problem, and
avoids the loop completely for when rate-limiting is in progress.

Ref: #12488
Ref: https://curl.se/mail/lib-2023-12/0012.html
Closes #12504

show more ...

lib: eliminate `conn->cselect_bits`

- use `data->state.dselect_bits` everywhere instead
- remove `bool *comeback` parameter as non-zero
`data->state.dselect_bits` will indicate tha

lib: eliminate `conn->cselect_bits`

- use `data->state.dselect_bits` everywhere instead
- remove `bool *comeback` parameter as non-zero
`data->state.dselect_bits` will indicate that IO is
incomplete.

Closes #12512

show more ...

connect: refactor `Curl_timeleft()`

- less local vars, "better" readability
- added documentation

Closes #12518

cookie: avoid fopen with empty file name

Closes #12514

tests/server: delete workaround for old-mingw

mingw-w64 1.0 comes with w32api v3.12, thus doesn't need this.

Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625

Re

tests/server: delete workaround for old-mingw

mingw-w64 1.0 comes with w32api v3.12, thus doesn't need this.

Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625

Reviewed-by: Jay Satiro
Closes #12510

show more ...

cmake: delete obsolete TODOs more [ci skip]

- manual completed: 898b012a9bf388590c4be7f526815b5ab74feca1 #1288
- soname completed: 5de6848f104d7cb0017080e31216265ac19d0dde #10023
- b

cmake: delete obsolete TODOs more [ci skip]

- manual completed: 898b012a9bf388590c4be7f526815b5ab74feca1 #1288
- soname completed: 5de6848f104d7cb0017080e31216265ac19d0dde #10023
- bunch of others that are completed
- `NTLM_WB_ENABLED` is implemented in a basic form, and now also
scheduled for removal, so a TODO at this point isn't useful.

And this 'to-check' item:

Q: "The cmake build selected to run gcc with -fPIC on my box while the
plain configure script did not."

A: With CMake, since 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546
and fc9bfb14520712672b4784e8b48256fb29204011 #11627, we explicitly
enable PIC for libcurl shared lib. Or when building libcurl for
shared and static lib in a single pass. We do this by default for
Windows or when enabled by the user via `SHARE_LIB_OBJECT`.
Otherwise we don't touch this setting. Meaning the default set by
CMake (if any) or the toolchain is used. On Debian Bookworm, this
means that PIC is disabled for static libs by default. Some platforms
(like macOS), has PIC enabled by default.
autotools supports the double-pass mode only, and in that case
CMake seems to match PIC behaviour now (as tested on Linux with gcc.)

Follow-up to 5d5dfdbd1a6c40bd75e982b66f49e1fa3a7eeae7 #12500

Reviewed-by: Jay Satiro
Closes #12509

show more ...

CLIENT-WRITERS: design and use documentation

Closes #12507

cmake: delete obsolete TODO items [ci skip]

There is always room for improvement, but CMake is up to par now with
autotools, so there is no longer a good reason to keep around these

cmake: delete obsolete TODO items [ci skip]

There is always room for improvement, but CMake is up to par now with
autotools, so there is no longer a good reason to keep around these
inline TODO items.

Answering one of questions:

Q: "The gcc command line use neither -g nor any -O options. As a
developer, I also treasure our configure scripts's --enable-debug
option that sets a long range of "picky" compiler options."

A: CMake offers the `CMAKE_BUILD_TYPE` variable to control debug info
and optimization level. E.g.:
- `Release` = `-O3` + no debug info
- `MinSizeRel` = `-Os` + no debug info
- `Debug` = `-O0` + debug info

https://stackoverflow.com/questions/48754619/what-are-cmake-build-type-debug-release-relwithdebinfo-and-minsizerel/59314670#59314670
https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#default-and-custom-configurations

For picky warnings we have the `PICKY_COMPILER` options, enabled by
default.

Closes #12500

show more ...

CONNECTION-FILTERS: update documentation

Closes #12497

lib: reduce use of strncpy

- bearssl: select cipher without buffer copies
- http_aws_sigv4: avoid strncpy, require exact timestamp length
- http_aws_sigv4: use memcpy isntead of strn

lib: reduce use of strncpy

- bearssl: select cipher without buffer copies
- http_aws_sigv4: avoid strncpy, require exact timestamp length
- http_aws_sigv4: use memcpy isntead of strncpy
- openssl: avoid strncpy calls
- schannel: check for 1.3 algos without buffer copies
- strerror: avoid strncpy calls
- telnet: avoid strncpy, return error on too long inputs
- vtls: avoid strncpy in multissl_version()

Closes #12499

show more ...

CI/distcheck: run full tests

To be able to detect missing files better, this now runs the full CI
test suite. If done before, it would have detected #12462 before
release.

C

CI/distcheck: run full tests

To be able to detect missing files better, this now runs the full CI
test suite. If done before, it would have detected #12462 before
release.

Closes #12503

show more ...

docs: clean up Protocols: for cmdline options

... and some other minor polish.

Closes #12496

cmdline/gen: fix the sorting of the man page options

They were previously sorted based on the file names, which use a .d
extension, making "data" get placed after "data-binary" etc. Maki

cmdline/gen: fix the sorting of the man page options

They were previously sorted based on the file names, which use a .d
extension, making "data" get placed after "data-binary" etc. Making the
sort ignore the extention fixes the ordering.

Reported-by: Boris Verkhovskiy
Bug: https://curl.se/mail/archive-2023-12/0014.html
Closes #12494

show more ...