idn: add native AppleIDN (icucore) support for macOS/iOS

I implemented the IDN functions for macOS and iOS using Unicode
libraries coming with macOS and iOS.

Builds and runs her

idn: add native AppleIDN (icucore) support for macOS/iOS

I implemented the IDN functions for macOS and iOS using Unicode
libraries coming with macOS and iOS.

Builds and runs here on macOS 14.2.1. Also verified to load and
run on older macOS version 10.13.

Build requires macOS SDK 13 or equivalent.

Set `-DUSE_APPLE_IDN=ON` CMake option to enable it.
With autotools and other build tools, set these manual options:
```
CPPFLAGS=-DUSE_APPLE_IDN
LIBS=-licucore
```

Completes TODO 1.6.

TODO: add autotools option and feature-detection.

Refs: #5330 #5371
Co-authored-by: Viktor Szakats
Closes #13246

show more ...

version: add "ECH" as a feature

If available

Follow-up to a362962b7
Closes #13378

curl_version_info: provide librtmp version

Ref: https://github.com/curl/curl/pull/13364#issuecomment-2054151942
Reported-by: talregev on github
Closes #13368

lib: merge `ENABLE_QUIC` C macro into `USE_HTTP3`

Before this patch `lib/curl_setup.h` defined these two macros right
next to each other, then the source code used them interchangeably.

lib: merge `ENABLE_QUIC` C macro into `USE_HTTP3`

Before this patch `lib/curl_setup.h` defined these two macros right
next to each other, then the source code used them interchangeably.

After this patch, `USE_HTTP3` guards all HTTP/3 / QUIC features.
(Like `USE_HTTP2` does for HTTP/2.) `ENABLE_QUIC` is no longer used.

This patch doesn't change the way HTTP/3 is enabled via autotools
or CMake. Builders who enabled HTTP/3 manually by defining both of
these macros via `CPPFLAGS` can now delete `-DENABLE_QUIC`.

Closes #13352

show more ...

build: prefer `USE_IPV6` macro internally (was: `ENABLE_IPV6`)

Before this patch, two macros were used to guard IPv6 features in curl
sources: `ENABLE_IPV6` and `USE_IPV6`. This patch ma

build: prefer `USE_IPV6` macro internally (was: `ENABLE_IPV6`)

Before this patch, two macros were used to guard IPv6 features in curl
sources: `ENABLE_IPV6` and `USE_IPV6`. This patch makes the source use
the latter for consistency with other similar switches.

`-DENABLE_IPV6` remains accepted for compatibility as a synonym for
`-DUSE_IPV6`, when passed to the compiler.

`ENABLE_IPV6` also remains the name of the CMake and `Makefile.vc`
options to control this feature.

Closes #13349

show more ...

NTLM_WB: drop support

The feature has not worked for months and has been marked as DEPRECATED
for six+ months.

Closes #13249

version: allow building with ancient libpsl

The psl_check_version_number() API was added in libpsl 0.11.0. CentOS 7
ships with version 0.7.0 which lacks this API. Revert to using the old

version: allow building with ancient libpsl

The psl_check_version_number() API was added in libpsl 0.11.0. CentOS 7
ships with version 0.7.0 which lacks this API. Revert to using the older
versioning API if we detect an old libpsl version.

Follow-up to 72bd88adde0e8cf6e63644a7d6df1da01a399db4
Bug: https://curl.se/mail/archive-2024-02/0004.html
Reported-by: Scott Mutter
Closes #12872

show more ...

version: show only the libpsl version, not its dependencies

The libpsl version output otherwise also includes version number for its
dependencies, like IDN lib, but since libcurl does no

version: show only the libpsl version, not its dependencies

The libpsl version output otherwise also includes version number for its
dependencies, like IDN lib, but since libcurl does not use libpsl's IDN
functionality those components are not important.

Ref: https://github.com/curl/curl-for-win/issues/63
Closes #12700

show more ...

windows: use built-in `_WIN32` macro to detect Windows

Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. T

windows: use built-in `_WIN32` macro to detect Windows

Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. The
agreement seems to be that `_WIN32` is the preferred practice here.
Make the source code rely on that to detect we're building for Windows.

Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for
Windows detection, next to the official `_WIN32`. After this patch it
only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`.

There is a slight chance these break compatibility with Windows
compilers that fail to define `_WIN32`. I'm not aware of any obsolete
or modern compiler affected, but in case there is one, one possible
solution is to define this macro manually.

grepping for `WIN32` remains useful to discover Windows-specific code.

Also:

- extend `checksrc` to ensure we're not using `WIN32` anymore.

- apply minor formatting here and there.

- delete unnecessary checks for `!MSDOS` when `_WIN32` is present.

Co-authored-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Closes #12376

show more ...

hyper: temporarily remove HTTP/2 support

The current design of the Hyper integration requires rebuilding the
Hyper clientconn for each request. However, building the clientconn
requi

hyper: temporarily remove HTTP/2 support

The current design of the Hyper integration requires rebuilding the
Hyper clientconn for each request. However, building the clientconn
requires resending the HTTP/2 connection preface, which is incorrect
from a protocol perspective. That in turn causes servers to send GOAWAY
frames, effectively degrading performance to "no connection reuse" in
the best case. It may also be triggering some bugs where requests get
dropped entirely and reconnects take too long.

This doesn't rule out HTTP/2 support with Hyper, but it may take a
redesign of the Hyper integration in order to make things work.

Closes #12191

show more ...

configure: better --disable-http

- disable HTTPS-proxy as well, since it can't work without HTTP

- curl_setup: when HTTP is disabled, also disable all features that are
HTTP-o

configure: better --disable-http

- disable HTTPS-proxy as well, since it can't work without HTTP

- curl_setup: when HTTP is disabled, also disable all features that are
HTTP-only

- version: HTTPS-proxy only exists if HTTP support exists

Closes #12223

show more ...

cmake: add support for "unity" builds

Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable

cmake: add support for "unity" builds

Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
`statename`.
- rename `ssl_backend_data` structure to unique names for each
TLS-backend, along with the `ssl_connect_data` struct member
referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
builds when compiling `lib/ldap.c`. To support "unity" builds, we had
to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
`schannel.h` option) _globally_. This caused an indirect inclusion of
Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
as well. This requires `[P]UNICODE_STRING` types, which is apperantly
not defined automatically (as seen with both MSVS and mingw-w64).
This patch includes `<subauth.h>` to fix it.
Ref: https://github.com/curl/curl/runs/13987772013
Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095

show more ...

lib: silence clang/gcc -Wvla warnings in brotli headers

brotli v1.0.0 throughout current latest v1.0.9 and latest master [1]
trigger this warning.

It happened with CMake and GNU

lib: silence clang/gcc -Wvla warnings in brotli headers

brotli v1.0.0 throughout current latest v1.0.9 and latest master [1]
trigger this warning.

It happened with CMake and GNU Make. autotools builds avoid it with
the `convert -I options to -isystem` macro.

llvm/clang:
```
In file included from ./curl/lib/content_encoding.c:36:
./brotli/x64-ucrt/usr/include/brotli/decode.h:204:34: warning: variable length array used [-Wvla]
const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)],
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./brotli/x64-ucrt/usr/include/brotli/port.h:253:34: note: expanded from macro 'BROTLI_ARRAY_PARAM'
^~~~~~
In file included from ./curl/lib/content_encoding.c:36:
./brotli/x64-ucrt/usr/include/brotli/decode.h:206:48: warning: variable length array used [-Wvla]
uint8_t decoded_buffer[BROTLI_ARRAY_PARAM(*decoded_size)]);
~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
./brotli/x64-ucrt/usr/include/brotli/port.h:253:35: note: expanded from macro 'BROTLI_ARRAY_PARAM'
~^~~~~
```

gcc:
```
In file included from ./curl/lib/content_encoding.c:36:
./brotli/x64-ucrt/usr/include/brotli/decode.h:204:5: warning: ISO C90 forbids variable length array 'encoded_buffer' [-Wvla]
204 | const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)],
| ^~~~~
./brotli/x64-ucrt/usr/include/brotli/decode.h:206:5: warning: ISO C90 forbids variable length array 'decoded_buffer' [-Wvla]
206 | uint8_t decoded_buffer[BROTLI_ARRAY_PARAM(*decoded_size)]);
| ^~~~~~~
```

[1] https://github.com/google/brotli/commit/ed1995b6bda19244070ab5d331111f16f67c8054

Reviewed-by: Daniel Stenberg
Reviewed-by: Marcel Raad
Closes #10738

show more ...

misc: remove support for curl_off_t < 8 bytes

Closes #10597

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- save

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205

show more ...

lib: connect/h2/h3 refactor

Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.

- eyeballing cfilter that uses

lib: connect/h2/h3 refactor

Refactoring of connection setup and happy eyeballing. Move
nghttp2. ngtcp2, quiche and msh3 into connection filters.

- eyeballing cfilter that uses sub-filters for performing parallel connects
- socket cfilter for all transport types, including QUIC
- QUIC implementations in cfilter, can now participate in eyeballing
- connection setup is more dynamic in order to adapt to what filter did
really connect. Relevant to see if a SSL filter needs to be added or
if SSL has already been provided
- HTTP/3 test cases similar to HTTP/2
- multiuse of parallel transfers for HTTP/3, tested for ngtcp2 and quiche

- Fix for data attach/detach in VTLS filters that could lead to crashes
during parallel transfers.
- Eliminating setup() methods in cfilters, no longer needed.
- Improving Curl_conn_is_alive() to replace Curl_connalive() and
integrated ssl alive checks into cfilter.
- Adding CF_CNTRL_CONN_INFO_UPDATE to tell filters to update
connection into and persist it at the easy handle.

- Several more cfilter related cleanups and moves:
- stream_weigth and dependency info is now wrapped in struct
Curl_data_priority
- Curl_data_priority members depend is available in HTTP2|HTTP3
- Curl_data_priority members depend on NGHTTP2 support
- handling init/reset/cleanup of priority part of url.c
- data->state.priority same struct, but shallow copy for compares only

- PROTOPT_STREAM has been removed
- Curl_conn_is_mulitplex() now available to check on capability

- Adding query method to connection filters.
- ngtcp2+quiche: implementing query for max concurrent transfers.

- Adding is_alive and keep_alive cfilter methods. Adding DATA_SETUP event.
- setting keepalive timestamp on connect
- DATA_SETUP is called after the connection has been completely
setup (but may not connected yet) to allow filters to initialize
data members they use.

- there is no socket to be had with msh3, it is unclear how select
shall work

- manual test via "curl --http3 https://curl.se" fail with "empty
reply from server".

- Various socket/conn related cleanups:
- Curl_socket is now Curl_socket_open and in cf-socket.c
- Curl_closesocket is now Curl_socket_close and in cf-socket.c
- Curl_ssl_use has been replaced with Cur_conn_is_ssl
- Curl_conn_tcp_accepted_set has been split into
Curl_conn_tcp_listen_set and Curl_conn_tcp_accepted_set
with a clearer purpose

Closes #10141

show more ...

version: add a feature names array to curl_version_info_data

Field feature_names contains a null-terminated sorted array of feature
names. Bitmask field features is deprecated.

version: add a feature names array to curl_version_info_data

Field feature_names contains a null-terminated sorted array of feature
names. Bitmask field features is deprecated.

Documentation is updated. Test 1177 and tests/version-scan.pl updated to
match new documentation format and extended to check feature names too.

Closes #9583

show more ...

lib: connection filters (cfilter) addition to curl:

- general construct/destroy in connectdata
- default implementations of callback functions
- connect: cfilters for connect and

lib: connection filters (cfilter) addition to curl:

- general construct/destroy in connectdata
- default implementations of callback functions
- connect: cfilters for connect and accept
- socks: cfilter for socks proxying
- http_proxy: cfilter for http proxy tunneling
- vtls: cfilters for primary and proxy ssl
- change in general handling of data/conn
- Curl_cfilter_setup() sets up filter chain based on data settings,
if none are installed by the protocol handler setup
- Curl_cfilter_connect() boot straps filters into `connected` status,
used by handlers and multi to reach further stages
- Curl_cfilter_is_connected() to check if a conn is connected,
e.g. all filters have done their work
- Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
indicators for multi select to work
- Curl_cfilter_data_pending() asks filters if the have incoming
data pending for recv
- Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
installed in conn->recv/conn->send for io handling
- Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
and addition/removal of a `data` from their connection
- adding vtl functions to prevent use of Curl_ssl globals directly
in other parts of the code.

Reviewed-by: Daniel Stenberg
Closes #9855

show more ...

tidy-up: delete parallel/unused feature flags

Detecting headers and lib separately makes sense when headers come in
variations or with extra ones, but this wasn't the case here. These we

tidy-up: delete parallel/unused feature flags

Detecting headers and lib separately makes sense when headers come in
variations or with extra ones, but this wasn't the case here. These were
duplicate/parallel macros that we had to keep in sync with each other
for a working build. This patch leaves a single macro for each of these
dependencies:

- Rely on `HAVE_LIBZ`, delete parallel `HAVE_ZLIB_H`.

Also delete CMake logic making sure these two were in sync, along with
a toggle to turn off that logic, called `CURL_SPECIAL_LIBZ`.

Also delete stray `HAVE_ZLIB` defines.

There is also a `USE_ZLIB` variant in `lib/config-dos.h`. This patch
retains it for compatibility and deprecates it.

- Rely on `USE_LIBSSH2`, delete parallel `HAVE_LIBSSH2_H`.

Also delete `LIBSSH2_WIN32`, `LIBSSH2_LIBRARY` from
`winbuild/MakefileBuild.vc`, these have a role when building libssh2
itself. And `CURL_USE_LIBSSH`, which had no use at all.

Also delete stray `HAVE_LIBSSH2` defines.

- Rely on `USE_LIBSSH`, delete parallel `HAVE_LIBSSH_LIBSSH_H`.

Also delete `LIBSSH_WIN32`, `LIBSSH_LIBRARY` and `HAVE_LIBSSH` from
`winbuild/MakefileBuild.vc`, these were the result of copy-pasting the
libssh2 line, and were not having any use.

- Delete unused `HAVE_LIBPSL_H` and `HAVE_LIBPSL`.

Reviewed-by: Daniel Stenberg

Closes #9652

show more ...

tool: remove protocol count limitation

Replace bit mask protocol sets by null-terminated arrays of protocol
tokens. These are the addresses of the protocol names returned by
curl_ver

tool: remove protocol count limitation

Replace bit mask protocol sets by null-terminated arrays of protocol
tokens. These are the addresses of the protocol names returned by
curl_version_info().

Protocol names are sorted case-insensitively before output to satisfy CI
tests matches consistency.

The protocol list returned by curl_version_info() is augmented with all
RTMP protocol variants.

Test 1401 adjusted for new alpha ordered output.

Closes #9546

show more ...

version: add ws + wss

version: rename threadsafe-init to threadsafe

Referring to Daniel's article [1], making the init function thread-safe
was the last bit to make libcurl thread-safe as a whole. So the name

version: rename threadsafe-init to threadsafe

Referring to Daniel's article [1], making the init function thread-safe
was the last bit to make libcurl thread-safe as a whole. So the name of
the feature may as well be the more concise 'threadsafe', also telling
the story that libcurl is now fully thread-safe, not just its init
function. Chances are high that libcurl wants to remain so in the
future, so there is little likelihood of ever needing any other distinct
`threadsafe-<name>` feature flags.

For consistency we also shorten `CURL_VERSION_THREADSAFE_INIT` to
`CURL_VERSION_THREADSAFE`, update its description and reference libcurl's
thread safety documentation.

[1]: https://daniel.haxx.se/blog/2022/06/08/making-libcurl-init-more-thread-safe/

Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Closes #8989

show more ...

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869

show more ...

curl_version_info: add CURL_VERSION_THREADSAFE_INIT

This flag can be used to make sure that curl_global_init() is
thread-safe.

This can be useful for libraries that can't contro

curl_version_info: add CURL_VERSION_THREADSAFE_INIT

This flag can be used to make sure that curl_global_init() is
thread-safe.

This can be useful for libraries that can't control what other
dependencies are doing with Curl.

Closes #8680

show more ...

lib: remove support for CURL_DOES_CONVERSIONS

TPF was the only user and support for that was dropped.

Closes #8378