urlapi: allow setting port number zero

Also set and check errno when strtoul() parsing numbers for better error
checking.

Updated test 1560

Closes #13427

urlapi: remove unused flags argument from Curl_url_set_authority

The function is only called from a single place (for HTTP/2 server push)
so might as well just assume this fixed option e

urlapi: remove unused flags argument from Curl_url_set_authority

The function is only called from a single place (for HTTP/2 server push)
so might as well just assume this fixed option every time.

Closes #13409

show more ...

urlapi: add CURLU_GET_EMPTY for empty queries and fragments

By default the API inhibits empty queries and fragments extracted.
Unless this new flag is set.

This also makes the b

urlapi: add CURLU_GET_EMPTY for empty queries and fragments

By default the API inhibits empty queries and fragments extracted.
Unless this new flag is set.

This also makes the behavior more consistent: without it set, zero
length queries and fragments are considered not present in the URL. With
the flag set, they are returned as a zero length strings if they were in
fact present in the URL.

This applies when extracting the individual query and fragment
components and for the full URL.

Closes #13396

show more ...

urlapi: fix relative redirects to fragment-only

Using the URL API for a redirect URL when the redirected-to string
starts with a hash, ie is only a fragment, the API would produce the

urlapi: fix relative redirects to fragment-only

Using the URL API for a redirect URL when the redirected-to string
starts with a hash, ie is only a fragment, the API would produce the
wrong final URL.

Adjusted test 1560 to test for several new redirect cases.

Closes #13394

show more ...

build: prefer `USE_IPV6` macro internally (was: `ENABLE_IPV6`)

Before this patch, two macros were used to guard IPv6 features in curl
sources: `ENABLE_IPV6` and `USE_IPV6`. This patch ma

build: prefer `USE_IPV6` macro internally (was: `ENABLE_IPV6`)

Before this patch, two macros were used to guard IPv6 features in curl
sources: `ENABLE_IPV6` and `USE_IPV6`. This patch makes the source use
the latter for consistency with other similar switches.

`-DENABLE_IPV6` remains accepted for compatibility as a synonym for
`-DUSE_IPV6`, when passed to the compiler.

`ENABLE_IPV6` also remains the name of the CMake and `Makefile.vc`
options to control this feature.

Closes #13349

show more ...

lib: initialize output pointers to NULL before calling strto[ff,l,ul]

In order to make MSAN happy:

==2200945==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x5

lib: initialize output pointers to NULL before calling strto[ff,l,ul]

In order to make MSAN happy:

==2200945==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x596f3b3ed246 in curlx_strtoofft [...]/libcurl/src/lib/strtoofft.c:239:11
#1 0x596f3b402156 in Curl_httpchunk_read [...]/libcurl/src/lib/http_chunks.c:149:12
#2 0x596f3b348550 in readwrite_data [...]/libcurl/src/lib/transfer.c:607:11
[...]

==2202041==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x5a3fab66a72a in Curl_parse_port [...]/libcurl/src/lib/urlapi.c:547:8
#1 0x5a3fab650645 in parse_authority [...]/libcurl/src/lib/urlapi.c:796:12
#2 0x5a3fab6740f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16
#3 0x5a3fab664fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c:1342:12
[...]

==2202320==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x569076a0d6b0 in ipv4_normalize [...]/libcurl/src/lib/urlapi.c:683:12
#1 0x5690769f2820 in parse_authority [...]/libcurl/src/lib/urlapi.c:803:10
#2 0x569076a160f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16
#3 0x569076a06fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c:1342:12
[...]

Signed-off-by: Louis Solofrizzo <lsolofrizzo@scaleway.com>
Closes #12995

show more ...

urlapi: remove assert

This assert triggers wrongly when CURLU_GUESS_SCHEME and
CURLU_NO_AUTHORITY are both set and the URL is a single path.

I think this assert has played out i

urlapi: remove assert

This assert triggers wrongly when CURLU_GUESS_SCHEME and
CURLU_NO_AUTHORITY are both set and the URL is a single path.

I think this assert has played out its role. It was introduced in a
rather big refactor.

Follow-up to 4cfa5bcc9a

Reported-by: promptfuzz_ on hackerone
Closes #12775

show more ...

curl.h: add CURLE_TOO_LARGE

A new error code to be used when an internal field grows too large, like
when a dynbuf reaches its maximum. Previously it would return
CURLE_OUT_OF_MEMORY

curl.h: add CURLE_TOO_LARGE

A new error code to be used when an internal field grows too large, like
when a dynbuf reaches its maximum. Previously it would return
CURLE_OUT_OF_MEMORY for this, which is highly misleading.

Ref: #12268
Closes #12269

show more ...

build: enable missing OpenSSF-recommended warnings, with fixes

https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html
as of 2023-11-29 [1]

build: enable missing OpenSSF-recommended warnings, with fixes

https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html
as of 2023-11-29 [1].

Enable new recommended warnings (except `-Wsign-conversion`):

- enable `-Wformat=2` for clang (in both cmake and autotools).
- add `CURL_PRINTF()` internal attribute and mark functions accepting
printf arguments with it. This is a copy of existing
`CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible
with redefinting the `printf` symbol:
https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94
- fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for
mingw-w64 and enable it on this platform.
- enable `-Wimplicit-fallthrough`.
- enable `-Wtrampolines`.
- add `-Wsign-conversion` commented with a FIXME.
- cmake: enable `-pedantic-errors` the way we do it with autotools.
Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747
- lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format
checks. Previously it was always disabled due to the internal `printf`
macro.

Fix them:

- fix bug where an `set_ipv6_v6only()` call was missed in builds with
`--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`.
- add internal `FALLTHROUGH()` macro.
- replace obsolete fall-through comments with `FALLTHROUGH()`.
- fix fallthrough markups: Delete redundant ones (showing up as
warnings in most cases). Add missing ones. Fix indentation.
- silence `-Wformat-nonliteral` warnings with llvm/clang.
- fix one `-Wformat-nonliteral` warning.
- fix new `-Wformat` and `-Wformat-security` warnings.
- fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its
definition to `lib/curl_setup.h` allowing use in `tests/server`.
- lib: fix two wrongly passed string arguments in log outputs.
Co-authored-by: Jay Satiro
- fix new `-Wformat` warnings on mingw-w64.

[1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895bfc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C%2B%2B.md

Closes #12489

show more ...

lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding

Since the copy does not stop at a null byte, let's not call it anything
that makes you think it works like the common s

lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding

Since the copy does not stop at a null byte, let's not call it anything
that makes you think it works like the common strndup() function.

Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat

Closes #12490

show more ...

windows: use built-in `_WIN32` macro to detect Windows

Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. T

windows: use built-in `_WIN32` macro to detect Windows

Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. The
agreement seems to be that `_WIN32` is the preferred practice here.
Make the source code rely on that to detect we're building for Windows.

Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for
Windows detection, next to the official `_WIN32`. After this patch it
only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`.

There is a slight chance these break compatibility with Windows
compilers that fail to define `_WIN32`. I'm not aware of any obsolete
or modern compiler affected, but in case there is one, one possible
solution is to define this macro manually.

grepping for `WIN32` remains useful to discover Windows-specific code.

Also:

- extend `checksrc` to ensure we're not using `WIN32` anymore.

- apply minor formatting here and there.

- delete unnecessary checks for `!MSDOS` when `_WIN32` is present.

Co-authored-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Closes #12376

show more ...

misc: fix -Walloc-size warnings

GCC 14 introduces a new -Walloc-size included in -Wextra which gives:

```
src/tool_operate.c: In function ‘add_per_transfer’:
src/tool_operat

misc: fix -Walloc-size warnings

GCC 14 introduces a new -Walloc-size included in -Wextra which gives:

```
src/tool_operate.c: In function ‘add_per_transfer’:
src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ for type ‘struct per_transfer’ with size ‘480’ [-Walloc-size]
213 | p = calloc(sizeof(struct per_transfer), 1);
| ^
src/var.c: In function ‘addvariable’:
src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type ‘struct var’ with size ‘32’ [-Walloc-size]
361 | p = calloc(sizeof(struct var), 1);
| ^
```

The calloc prototype is:
```
void *calloc(size_t nmemb, size_t size);
```

So, just swap the number of members and size arguments to match the
prototype, as we're initialising 1 struct of size `sizeof(struct
...)`. GCC then sees we're not doing anything wrong.

Closes #12292

show more ...

lib: add and use Curl_strndup()

The Curl_strndup() function is similar to memdup(), but copies 'n' bytes
then adds a terminating null byte ('\0').

Closes #12251

urlapi: when URL encoding the fragment, pass in the right length

A benign bug because it would only add an extra null terminator.

Made lib1560 get a test that runs this code.

urlapi: when URL encoding the fragment, pass in the right length

A benign bug because it would only add an extra null terminator.

Made lib1560 get a test that runs this code.

Closes #12250

show more ...

urlapi: skip appending NULL pointer query

Reported-by: kirbyn17 on hackerone

Closes #12240

urlapi: avoid null deref if setting blank host to url encode

Reported-by: kirbyn17 on hackerone

Closes #12240

url: protocol handler lookup tidy-up

- rename lookup to what it does
- use ARRAYSIZE instead of NULL check for end
- offer alternate lookup for 0-terminated strings

Closes #

url: protocol handler lookup tidy-up

- rename lookup to what it does
- use ARRAYSIZE instead of NULL check for end
- offer alternate lookup for 0-terminated strings

Closes #12216

show more ...

tidy-up: use more example domains

Also make use of the example TLD:
https://en.wikipedia.org/wiki/.example

Reviewed-by: Daniel Stenberg
Closes #11992

escape: replace Curl_isunreserved with ISUNRESERVED

- Use the ALLCAPS version of the macro so that it is clear a macro is
being called that evaluates the variable multiple times.

escape: replace Curl_isunreserved with ISUNRESERVED

- Use the ALLCAPS version of the macro so that it is clear a macro is
being called that evaluates the variable multiple times.

- Also capitalize macro isurlpuntcs => ISURLPUNTCS since it evaluates
a variable multiple times.

This is a follow-up to 291d225a which changed Curl_isunreserved into an
alias macro for ISUNRESERVED. The problem is the former is not easily
identified as a macro by the caller, which could lead to a bug.

For example, ISUNRESERVED(*foo++) is easily identifiable as wrong but
Curl_isunreserved(*foo++) is not even though they both are the same.

Closes https://github.com/curl/curl/pull/11846

show more ...

urlapi: setting a blank URL ("") is not an ok URL

Test it in 1560
Fixes #11714
Reported-by: ad0p on github
Closes #11715

urlapi: return CURLUE_BAD_HOSTNAME if puny2idn encoding fails

And document it. Only return out of memory when it actually is a memory
problem.

Pointed-out-by: Jacob Mealey
C

urlapi: return CURLUE_BAD_HOSTNAME if puny2idn encoding fails

And document it. Only return out of memory when it actually is a memory
problem.

Pointed-out-by: Jacob Mealey
Closes #11674

show more ...

urlapi: CURLU_PUNY2IDN - convert from punycode to IDN name

Asssisted-by: Jay Satiro
Closes #11655

urlapi: make sure zoneid is also duplicated in curl_url_dup

Add several curl_url_dup() tests to the general lib1560 test.

Reported-by: Rutger Broekhoff
Bug: https://curl.se/mail

urlapi: make sure zoneid is also duplicated in curl_url_dup

Add several curl_url_dup() tests to the general lib1560 test.

Reported-by: Rutger Broekhoff
Bug: https://curl.se/mail/lib-2023-07/0047.html
Closes #11549

show more ...

urlapi: fix heap buffer overflow

`u->path = Curl_memdup(path, pathlen + 1);` accesses bytes after the null-terminator.

```
==2676==ERROR: AddressSanitizer: heap-buffer-overflow

urlapi: fix heap buffer overflow

`u->path = Curl_memdup(path, pathlen + 1);` accesses bytes after the null-terminator.

```
==2676==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x04d48c75 at pc 0x0112708a bp 0x006fb7e0 sp 0x006fb3c4
READ of size 78 at 0x04d48c75 thread T0
#0 0x1127089 in __asan_wrap_memcpy D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:840
#1 0x1891a0e in Curl_memdup C:\actions-runner\_work\client\client\third_party\curl\lib\strdup.c:97
#2 0x18db4b0 in parseurl C:\actions-runner\_work\client\client\third_party\curl\lib\urlapi.c:1297
#3 0x18db819 in parseurl_and_replace C:\actions-runner\_work\client\client\third_party\curl\lib\urlapi.c:1342
#4 0x18d6e39 in curl_url_set C:\actions-runner\_work\client\client\third_party\curl\lib\urlapi.c:1790
#5 0x1877d3e in parseurlandfillconn C:\actions-runner\_work\client\client\third_party\curl\lib\url.c:1768
#6 0x1871acf in create_conn C:\actions-runner\_work\client\client\third_party\curl\lib\url.c:3403
#7 0x186d8dc in Curl_connect C:\actions-runner\_work\client\client\third_party\curl\lib\url.c:3888
#8 0x1856b78 in multi_runsingle C:\actions-runner\_work\client\client\third_party\curl\lib\multi.c:1982
#9 0x18531e3 in curl_multi_perform C:\actions-runner\_work\client\client\third_party\curl\lib\multi.c:2756
```

Closes #11560

show more ...

curl_url_set: enforce the max string length check for all parts

Update the docs and test 1559 accordingly

Closes #11273