lib: silence warnings on comma misuse

Building curl with -Wcomma, I see warnings about "possible misuse of
comma operator here" and moving fields assignment out of the for() fixes
it

lib: silence warnings on comma misuse

Building curl with -Wcomma, I see warnings about "possible misuse of
comma operator here" and moving fields assignment out of the for() fixes
it.

Closes #13392

show more ...

cookie: if psl fails, reject the cookie

A libpsl install without data and no built-in database is now considered
bad enough to reject all cookies since they cannot be checked. It is

cookie: if psl fails, reject the cookie

A libpsl install without data and no built-in database is now considered
bad enough to reject all cookies since they cannot be checked. It is
somewhat of a user error, but still.

Reported-by: Dan Fandrich
Closes #13033

show more ...

lib: convert Curl_get_line to use dynbuf

Create the line in a dynbuf. Aborts the reading of the file on
errors. Avoids having to always allocate maximum amount from the
start. Avoids

lib: convert Curl_get_line to use dynbuf

Create the line in a dynbuf. Aborts the reading of the file on
errors. Avoids having to always allocate maximum amount from the
start. Avoids direct malloc.

Closes #12846

show more ...

build: enable missing OpenSSF-recommended warnings, with fixes

https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html
as of 2023-11-29 [1]

build: enable missing OpenSSF-recommended warnings, with fixes

https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C++.html
as of 2023-11-29 [1].

Enable new recommended warnings (except `-Wsign-conversion`):

- enable `-Wformat=2` for clang (in both cmake and autotools).
- add `CURL_PRINTF()` internal attribute and mark functions accepting
printf arguments with it. This is a copy of existing
`CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible
with redefinting the `printf` symbol:
https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94
- fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for
mingw-w64 and enable it on this platform.
- enable `-Wimplicit-fallthrough`.
- enable `-Wtrampolines`.
- add `-Wsign-conversion` commented with a FIXME.
- cmake: enable `-pedantic-errors` the way we do it with autotools.
Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747
- lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format
checks. Previously it was always disabled due to the internal `printf`
macro.

Fix them:

- fix bug where an `set_ipv6_v6only()` call was missed in builds with
`--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`.
- add internal `FALLTHROUGH()` macro.
- replace obsolete fall-through comments with `FALLTHROUGH()`.
- fix fallthrough markups: Delete redundant ones (showing up as
warnings in most cases). Add missing ones. Fix indentation.
- silence `-Wformat-nonliteral` warnings with llvm/clang.
- fix one `-Wformat-nonliteral` warning.
- fix new `-Wformat` and `-Wformat-security` warnings.
- fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its
definition to `lib/curl_setup.h` allowing use in `tests/server`.
- lib: fix two wrongly passed string arguments in log outputs.
Co-authored-by: Jay Satiro
- fix new `-Wformat` warnings on mingw-w64.

[1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895bfc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options-Hardening-Guide-for-C-and-C%2B%2B.md

Closes #12489

show more ...

cookie: avoid fopen with empty file name

Closes #12514

lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding

Since the copy does not stop at a null byte, let's not call it anything
that makes you think it works like the common s

lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding

Since the copy does not stop at a null byte, let's not call it anything
that makes you think it works like the common strndup() function.

Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat

Closes #12490

show more ...

lib: strndup/memdup instead of malloc, memcpy and null-terminate

- bufref: use strndup
- cookie: use strndup
- formdata: use strndup
- ftp: use strndup
- gtls: use aprin

lib: strndup/memdup instead of malloc, memcpy and null-terminate

- bufref: use strndup
- cookie: use strndup
- formdata: use strndup
- ftp: use strndup
- gtls: use aprintf instead of malloc + strcpy * 2
- http: use strndup
- mbedtls: use strndup
- md4: use memdup
- ntlm: use memdup
- ntlm_sspi: use strndup
- pingpong: use memdup
- rtsp: use strndup instead of malloc, memcpy and null-terminate
- sectransp: use strndup
- socks_gssapi.c: use memdup
- vtls: use dynbuf instead of malloc, snprintf and memcpy
- vtls: use strdup instead of malloc + memcpy
- wolfssh: use strndup

Closes #12453

show more ...

cookie: lowercase the domain names before PSL checks

Reported-by: Harry Sintonen

Closes #12387

urldata: move cookielist from UserDefined to UrlState

1. Because the value is not strictly set with a setopt option.

2. Because otherwise when duping a handle when all the set.* fie

urldata: move cookielist from UserDefined to UrlState

1. Because the value is not strictly set with a setopt option.

2. Because otherwise when duping a handle when all the set.* fields are
first copied and an error happens (think out of memory mid-function),
the function would easily free the list *before* it was deep-copied,
which could lead to a double-free.

Closes #12323

show more ...

misc: fix -Walloc-size warnings

GCC 14 introduces a new -Walloc-size included in -Wextra which gives:

```
src/tool_operate.c: In function ‘add_per_transfer’:
src/tool_operat

misc: fix -Walloc-size warnings

GCC 14 introduces a new -Walloc-size included in -Wextra which gives:

```
src/tool_operate.c: In function ‘add_per_transfer’:
src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ for type ‘struct per_transfer’ with size ‘480’ [-Walloc-size]
213 | p = calloc(sizeof(struct per_transfer), 1);
| ^
src/var.c: In function ‘addvariable’:
src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type ‘struct var’ with size ‘32’ [-Walloc-size]
361 | p = calloc(sizeof(struct var), 1);
| ^
```

The calloc prototype is:
```
void *calloc(size_t nmemb, size_t size);
```

So, just swap the number of members and size arguments to match the
prototype, as we're initialising 1 struct of size `sizeof(struct
...)`. GCC then sees we're not doing anything wrong.

Closes #12292

show more ...

lib: add and use Curl_strndup()

The Curl_strndup() function is similar to memdup(), but copies 'n' bytes
then adds a terminating null byte ('\0').

Closes #12251

cookie: set ->running in cookie_init even if data is NULL

This is a regression introduced in b1b326ec500 (shipped in curl 8.1.0)

Test 3103 verifies.

Fixes #11875
Report

cookie: set ->running in cookie_init even if data is NULL

This is a regression introduced in b1b326ec500 (shipped in curl 8.1.0)

Test 3103 verifies.

Fixes #11875
Reported-by: wangp on github
Closes #11876

show more ...

cookie: reduce variable scope, add const

cookie: do not store the expire or max-age strings

Convert it to an expire time at once and save memory.

Closes #11862

cookie: remove unnecessary struct fields

Plus: reduce the hash table size from 256 to 63. It seems unlikely to
make much of a speed difference for most use cases but saves 1.5KB of
d

cookie: remove unnecessary struct fields

Plus: reduce the hash table size from 256 to 63. It seems unlikely to
make much of a speed difference for most use cases but saves 1.5KB of
data per instance.

Closes #11862

show more ...

cmake: add support for "unity" builds

Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable

cmake: add support for "unity" builds

Aka "jumbo" or "amalgamation" builds. It means to compile all sources
per target as a single C source. This is experimental.

You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake.
It requires CMake 3.16 or newer.

It makes builds (much) faster, allows for better optimizations and tends
to promote less ambiguous code.

Also add a new AppVeyor CI job and convert an existing one to use
"unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job.

Fix related issues:
- add missing include guard to `easy_lock.h`.
- rename static variables and functions (and a macro) with names reused
across sources, or shadowed by local variables.
- add an `#undef` after use.
- add a missing `#undef` before use.
- move internal definitions from `ftp.h` to `ftp.c`.
- `curl_memory.h` fixes to make it work when included repeatedly.
- stop building/linking curlx bits twice for a static-mode curl tool.
These caused doubly defined symbols in unity builds.
- silence missing extern declarations compiler warning for ` _CRT_glob`.
- fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`.
- fix colliding static symbols in debug mode: `debugtime()` and
`statename`.
- rename `ssl_backend_data` structure to unique names for each
TLS-backend, along with the `ssl_connect_data` struct member
referencing them. This required adding casts for each access.
- add workaround for missing `[P]UNICODE_STRING` types in certain Windows
builds when compiling `lib/ldap.c`. To support "unity" builds, we had
to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows
`schannel.h` option) _globally_. This caused an indirect inclusion of
Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled
as well. This requires `[P]UNICODE_STRING` types, which is apperantly
not defined automatically (as seen with both MSVS and mingw-w64).
This patch includes `<subauth.h>` to fix it.
Ref: https://github.com/curl/curl/runs/13987772013
Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&view=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c
- tweak unity builds to compile `lib/memdebug.c` separately in memory
trace builds to avoid PP confusion.
- force-disable unity for test programs.
- do not compile and link libcurl sources to libtests _twice_ when libcurl
is built in static mode.

KNOWN ISSUES:
- running tests with unity builds may fail in cases.
- some build configurations/env may not compile in unity mode. E.g.:
https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfuauwl8q#L250

Ref: https://github.com/libssh2/libssh2/issues/1034
Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html
Ref: https://en.wikipedia.org/wiki/Unity_build

Closes #11095

show more ...

misc: fix spelling mistakes

Reported-by: musvaage on github
Fixes #11171
Closes #11172

checksrc: disallow spaces before labels

Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
th

checksrc: disallow spaces before labels

Out of 415 labels throughout the code base, 86 of those labels were
not at the start of the line. Which means labels always at the start of
the line is the favoured style overall with 329 instances.

Out of the 86 labels not at the start of the line:
* 75 were indented with the same indentation level of the following line
* 8 were indented with exactly one space
* 2 were indented with one fewer indentation level then the following
line
* 1 was indented with the indentation level of the following line minus
three space (probably unintentional)

Co-Authored-By: Viktor Szakats

Closes #11134

show more ...

cookie: address PVS nits

- avoid assigning the same value again
- remove superfluous check of co->domain
- reduce variable scope for namep/valuep

Ref: #10929
Closes #109

cookie: address PVS nits

- avoid assigning the same value again
- remove superfluous check of co->domain
- reduce variable scope for namep/valuep

Ref: #10929
Closes #10954

show more ...

cookie: clarify that init with data set to NULL reads no file

... and make Curl_cookie_add() require 'data' being set proper with an
assert.

The function has not worked with a N

cookie: clarify that init with data set to NULL reads no file

... and make Curl_cookie_add() require 'data' being set proper with an
assert.

The function has not worked with a NULL data for quite some time so this
just corrects the code and comment.

This is a different take than the proposed fixed in #10927

Reported-by: Kvarec Lezki
Ref: #10929
Closes #10930

show more ...

cookie: don't load cookies again when flushing

Reported-by: Sergio Mijatovic
Fixes #10677
Closes #10685

cookie: parse without sscanf()

Saves us from using 2*4096 bytes buffers on stack, the extra copies and
more.

Closes #10550

cookies: fp is always not NULL

Closes #10383

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- save

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205

show more ...

urldata: move the cookefilelist to the 'set' struct

The cookiefile entries are set into the handle and should remain set for
the lifetime of the handle so that duplicating it also duplic

urldata: move the cookefilelist to the 'set' struct

The cookiefile entries are set into the handle and should remain set for
the lifetime of the handle so that duplicating it also duplicates the
list. Therefore, the struct field is moved from 'state' to 'set'.

Fixes #10133
Closes #10134

show more ...