urlapi: normalize the IPv6 address

As the parsing and address "regeneration" are done anyway, we might as
well use the updated version in the result and thereby A) get a
normalized (

urlapi: normalize the IPv6 address

As the parsing and address "regeneration" are done anyway, we might as
well use the updated version in the result and thereby A) get a
normalized (and lower cased) version of the address and B) avoid a
strcpy().

Updated test 1560 to verify.

Closes #15143

show more ...

WebSockets: make support official (non-experimental)

Inverts the configure/cmake options to instead provide options that
disable WebSockets and have them (ws + wss) enabled by default.

WebSockets: make support official (non-experimental)

Inverts the configure/cmake options to instead provide options that
disable WebSockets and have them (ws + wss) enabled by default.

Closes #14936

show more ...

lib/src: white space edits to comply better with code style

... as checksrc now finds and complains about these.

Closes #14921

urlapi: verify URL *decoded* hostname when set

It was previously wrongly verifying the input in its URL encoded format
when setting the hostname component with curl_url_set(), so it wron

urlapi: verify URL *decoded* hostname when set

It was previously wrongly verifying the input in its URL encoded format
when setting the hostname component with curl_url_set(), so it wrongly
rejected '%'.

Now it URL decodes the name appropriately before the check.

Added tests to lib1560 to verify that a fine %-code is okay and that a
bad %-code (that decodes to '%') is rejected.

Regression from 0a0c9b6dfa0de4a4c, shipped in 8.0.0

Fixes #14656
Reported-by: Venkat Krishna R
Closes #14657

show more ...

urlapi: add CURLU_NO_GUESS_SCHEME

Used for extracting:

- when used asking for a scheme, it will return CURLUE_NO_SCHEME if the
stored information was a guess

- when u

urlapi: add CURLU_NO_GUESS_SCHEME

Used for extracting:

- when used asking for a scheme, it will return CURLUE_NO_SCHEME if the
stored information was a guess

- when used asking for a URL, the URL is returned without a scheme, like
when previously given to the URL parser when it was asked to guess

- as soon as the scheme is set explicitly, it is no longer internally
marked as guessed

The idea being:

1. allow a user to figure out if a URL's scheme was set as a result of
guessing

2. extract the URL without a guessed scheme

3. this makes it work similar to how we already deal with port numbers

Extend test 1560 to verify.

Closes #13616

show more ...

tests: make the unit test result type `CURLcode`

Before this patch, the result code was a mixture of `int` and
`CURLcode`.

Also adjust casts and fix a couple of minor issues fou

tests: make the unit test result type `CURLcode`

Before this patch, the result code was a mixture of `int` and
`CURLcode`.

Also adjust casts and fix a couple of minor issues found along the way.

Cherry-picked from #13489
Closes #13600

show more ...

urlapi: allow setting port number zero

Also set and check errno when strtoul() parsing numbers for better error
checking.

Updated test 1560

Closes #13427

urlapi: add CURLU_GET_EMPTY for empty queries and fragments

By default the API inhibits empty queries and fragments extracted.
Unless this new flag is set.

This also makes the b

urlapi: add CURLU_GET_EMPTY for empty queries and fragments

By default the API inhibits empty queries and fragments extracted.
Unless this new flag is set.

This also makes the behavior more consistent: without it set, zero
length queries and fragments are considered not present in the URL. With
the flag set, they are returned as a zero length strings if they were in
fact present in the URL.

This applies when extracting the individual query and fragment
components and for the full URL.

Closes #13396

show more ...

lib1560: test with leading zeroes and more IPv4 versions

Inspired by WHATWG URL Spec test inputs

Closes #13400

urlapi: fix relative redirects to fragment-only

Using the URL API for a redirect URL when the redirected-to string
starts with a hash, ie is only a fragment, the API would produce the

urlapi: fix relative redirects to fragment-only

Using the URL API for a redirect URL when the redirected-to string
starts with a hash, ie is only a fragment, the API would produce the
wrong final URL.

Adjusted test 1560 to test for several new redirect cases.

Closes #13394

show more ...

idn: add native AppleIDN (icucore) support for macOS/iOS

I implemented the IDN functions for macOS and iOS using Unicode
libraries coming with macOS and iOS.

Builds and runs her

idn: add native AppleIDN (icucore) support for macOS/iOS

I implemented the IDN functions for macOS and iOS using Unicode
libraries coming with macOS and iOS.

Builds and runs here on macOS 14.2.1. Also verified to load and
run on older macOS version 10.13.

Build requires macOS SDK 13 or equivalent.

Set `-DUSE_APPLE_IDN=ON` CMake option to enable it.
With autotools and other build tools, set these manual options:
```
CPPFLAGS=-DUSE_APPLE_IDN
LIBS=-licucore
```

Completes TODO 1.6.

TODO: add autotools option and feature-detection.

Refs: #5330 #5371
Co-authored-by: Viktor Szakats
Closes #13246

show more ...

build: delete/replace clang warning pragmas

- delete redundant warning suppressions for `-Wformat-nonliteral`.
This now relies on `CURL_PRINTF()` and it's theoratically possible

build: delete/replace clang warning pragmas

- delete redundant warning suppressions for `-Wformat-nonliteral`.
This now relies on `CURL_PRINTF()` and it's theoratically possible
that this macro isn't active but the warning is. We're ignoring this
as a corner-case here.

- replace two pragmas with code changes to avoid the warnings.

Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803
Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540
Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489

Reviewed-by: Daniel Stenberg
Closes #12812

show more ...

build: fix some `-Wsign-conversion`/`-Warith-conversion` warnings

- enable `-Wsign-conversion` warnings, but also setting them to not
raise errors.
- fix `-Warith-conversion` warni

build: fix some `-Wsign-conversion`/`-Warith-conversion` warnings

- enable `-Wsign-conversion` warnings, but also setting them to not
raise errors.
- fix `-Warith-conversion` warnings seen in CI.
These are triggered by `-Wsign-converion` and causing errors unless
explicitly silenced. It makes more sense to fix them, there just a few
of them.
- fix some `-Wsign-conversion` warnings.
- hide `-Wsign-conversion` warnings with a `#pragma`.
- add macro `CURL_WARN_SIGN_CONVERSION` to unhide them on a per-build
basis.
- update a CI job to unhide them with the above macro:
https://github.com/curl/curl/actions/workflows/linux.yml -> OpenSSL -O3

Closes #12492

show more ...

build: enable missing OpenSSF-recommended warnings, with fixes

https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html
as of 2023-11-29 [1]

build: enable missing OpenSSF-recommended warnings, with fixes

https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html
as of 2023-11-29 [1].

Enable new recommended warnings (except `-Wsign-conversion`):

- enable `-Wformat=2` for clang (in both cmake and autotools).
- add `CURL_PRINTF()` internal attribute and mark functions accepting
printf arguments with it. This is a copy of existing
`CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible
with redefinting the `printf` symbol:
https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94
- fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for
mingw-w64 and enable it on this platform.
- enable `-Wimplicit-fallthrough`.
- enable `-Wtrampolines`.
- add `-Wsign-conversion` commented with a FIXME.
- cmake: enable `-pedantic-errors` the way we do it with autotools.
Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747
- lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format
checks. Previously it was always disabled due to the internal `printf`
macro.

Fix them:

- fix bug where an `set_ipv6_v6only()` call was missed in builds with
`--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`.
- add internal `FALLTHROUGH()` macro.
- replace obsolete fall-through comments with `FALLTHROUGH()`.
- fix fallthrough markups: Delete redundant ones (showing up as
warnings in most cases). Add missing ones. Fix indentation.
- silence `-Wformat-nonliteral` warnings with llvm/clang.
- fix one `-Wformat-nonliteral` warning.
- fix new `-Wformat` and `-Wformat-security` warnings.
- fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its
definition to `lib/curl_setup.h` allowing use in `tests/server`.
- lib: fix two wrongly passed string arguments in log outputs.
Co-authored-by: Jay Satiro
- fix new `-Wformat` warnings on mingw-w64.

[1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895bfc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C%2B%2B.md

Closes #12489

show more ...

windows: use built-in `_WIN32` macro to detect Windows

Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. T

windows: use built-in `_WIN32` macro to detect Windows

Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. The
agreement seems to be that `_WIN32` is the preferred practice here.
Make the source code rely on that to detect we're building for Windows.

Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for
Windows detection, next to the official `_WIN32`. After this patch it
only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`.

There is a slight chance these break compatibility with Windows
compilers that fail to define `_WIN32`. I'm not aware of any obsolete
or modern compiler affected, but in case there is one, one possible
solution is to define this macro manually.

grepping for `WIN32` remains useful to discover Windows-specific code.

Also:

- extend `checksrc` to ensure we're not using `WIN32` anymore.

- apply minor formatting here and there.

- delete unnecessary checks for `!MSDOS` when `_WIN32` is present.

Co-authored-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Closes #12376

show more ...

urlapi: when URL encoding the fragment, pass in the right length

A benign bug because it would only add an extra null terminator.

Made lib1560 get a test that runs this code.

urlapi: when URL encoding the fragment, pass in the right length

A benign bug because it would only add an extra null terminator.

Made lib1560 get a test that runs this code.

Closes #12250

show more ...

lib1560: verify appending blank URL encoded query string

lib1560: verify setting host to "" with and without URL encode

spelling: fix codespell 2.2.6 typos

Closes #12019

urlapi: setting a blank URL ("") is not an ok URL

Test it in 1560
Fixes #11714
Reported-by: ad0p on github
Closes #11715

urlapi: CURLU_PUNY2IDN - convert from punycode to IDN name

Asssisted-by: Jay Satiro
Closes #11655

urlapi: make sure zoneid is also duplicated in curl_url_dup

Add several curl_url_dup() tests to the general lib1560 test.

Reported-by: Rutger Broekhoff
Bug: https://curl.se/mail

urlapi: make sure zoneid is also duplicated in curl_url_dup

Add several curl_url_dup() tests to the general lib1560 test.

Reported-by: Rutger Broekhoff
Bug: https://curl.se/mail/lib-2023-07/0047.html
Closes #11549

show more ...

urlapi: have *set(PATH) prepend a slash if one is missing

Previously the code would just do that for the path when extracting the
full URL, which made a subsequent curl_url_get() of the

urlapi: have *set(PATH) prepend a slash if one is missing

Previously the code would just do that for the path when extracting the
full URL, which made a subsequent curl_url_get() of the path to
(unexpectedly) still return it without the leading path.

Amend lib1560 to verify this. Clarify the curl_url_set() docs about it.

Bug: https://curl.se/mail/lib-2023-06/0015.html
Closes #11272
Reported-by: Pedro Henrique

show more ...

urlapi: scheme starts with alpha

Add multiple tests to lib1560 to verify

Fixes #11249
Reported-by: ad0p on github
Closes #11250

lib1560: verify more scheme guessing

- on 2nd level domains
- on names without dots

As mentioned in #11161, "imap.com" will be guessed IMAP

Closes #11219