tests: speed up builds with single-binary test bundles

Add support for single-block binaries that contain all libtests and
unit tests respectively.

Enable with:
- autotools:

tests: speed up builds with single-binary test bundles

Add support for single-block binaries that contain all libtests and
unit tests respectively.

Enable with:
- autotools: `--enable-test-bundles`
- cmake: `-DCURL_TEST_BUNDLES=ON`

(They are compatible with `--enable-unity` and `-DCMAKE_UNITY_BUILD=ON`
options, for further speed-up.)

Makes libtests and unit tests build _fast_, needing little disk space
even in static mode. Similar to CMake unity mode, but with a custom
script, also supporting autotools builds.

The price is having to deal with symbols/macros colliding between
`lib*.c` and `unit*.c` sources. Maybe with naming conventions or other
solutions this can be improved gradually and reduce the need for manual
intervention by `mk-bundle.mk`. I've included a script that does the bulk
of detecting name collisions.

Also:
- CI: enable test bundles.
- CI: build tests in more jobs.
- lib2305: fix FILE handle leak.
- unit1661: fix memleak found by torture test by releasing the `bufref`
structure in `unit_stop()` that was allocated in `unit_setup()`.
```
test 1661...[bufref unit tests]
Leak detected: memory still allocated: 13 bytes
allocated by /home/runner/work/curl/curl/tests/unit/unit1661.c:70
1661: torture FAILED: function number 1 in test.
```
Ref: https://github.com/curl/curl/actions/runs/10967279334/job/30456745290?pr=14772#step:8:41

Similar test suite builds with autotools default and cmake+bundle+unity:
- GHA/Linux: 33s vs 7s
https://github.com/curl/curl/actions/runs/10705668823/job/29681617374
- GHA/macOS 34s vs 2s
https://github.com/curl/curl/actions/runs/10705668813/job/29681632885
- GHA/FreeBSD: 15m25 vs 6m21 (full workflow time, ~qemu)
https://github.com/curl/curl/actions/runs/10705668811/job/29681607915
- GHA/Cygwin: 9m52 vs 32s
https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
- GHA/MSYS2: 3m52 vs 14s
https://github.com/curl/curl/actions/runs/10705668808/job/29681624295
- GHA/mingw-w64: 5m45 vs 30s
https://github.com/curl/curl/actions/runs/10705668808/job/29681628787

Autotools test suite builds compared between master -> `--enable-test-bundles`:
- GHA/Linux: 33s -> 9s (run tests: 22m23 -> 20m44)
https://github.com/curl/curl/actions/runs/10710030193/job/29695932185
https://github.com/curl/curl/actions/runs/10967831456/job/30458220344
- GHA/macOS: 25s -> 4s (run tests: 2m58 -> 2m24)
https://github.com/curl/curl/actions/runs/10710030195/job/29695938444
https://github.com/curl/curl/actions/runs/10967831452/job/30458225762
- GHA/non-native (FreeBSD): 4m8 -> 3m12 (full workflow time, ~qemu)
https://github.com/curl/curl/actions/runs/10710030198/job/29695928401
https://github.com/curl/curl/actions/runs/10967831458/job/30458212692
- GHA/Cygwin: 9m25 -> 1m9 (run tests: 9m19 -> 3m28)
https://github.com/curl/curl/actions/runs/10710030212/job/29695928213
https://github.com/curl/curl/actions/runs/10967831453/job/30458213268
- GHA/MSYS2: 3m54 -> 32s (run tests: 6m3 -> 3m59)
https://github.com/curl/curl/actions/runs/10710030190/job/29704850591
https://github.com/curl/curl/actions/runs/10967831449/job/30459280005
- GHA/mingw-w64: 5m42 -> 1m5 (run tests: 7m41 -> 5m36)
https://github.com/curl/curl/actions/runs/10710030190/job/29704852058
https://github.com/curl/curl/actions/runs/10967831449/job/30459280862
- Azure MSYS2 mingw64 openssl: 38m55 -> 11m58
https://dev.azure.com/daniel0244/curl/_build/results?buildId=25546&view=logs&j=b58b8c59-0f61-52e9-0f9e-fad562a1e77f&t=0f9230a7-3b10-53ca-9938-700ece377c5e
https://dev.azure.com/daniel0244/curl/_build/results?buildId=25547&view=logs&jobId=39473db1-3945-55d5-deb5-c218fad88dce&j=b58b8c59-0f61-52e9-0f9e-fad562a1e77f&t=0f9230a7-3b10-53ca-9938-700ece377c5e
- Azure Ubuntu default: 2m15 -> 55s (all build)
https://dev.azure.com/daniel0244/curl/_build/results?buildId=25546&view=logs&j=9d58b9ac-e1e6-53b6-f83a-1f9f1d912522&t=a6b38d83-e7cf-5a9b-c762-a178412717b7
https://dev.azure.com/daniel0244/curl/_build/results?buildId=25547&view=logs&jobId=39473db1-3945-55d5-deb5-c218fad88dce&j=9d58b9ac-e1e6-53b6-f83a-1f9f1d912522&t=a6b38d83-e7cf-5a9b-c762-a178412717b7

Cmake test suite builds compared between master -> `-DCURL_TEST_BUNDLES=ON` + unity:
- GHA/Linux: 29s -> 7s (run tests: 4m50 -> 4m57, 20m43 -> 20m45)
https://github.com/curl/curl/actions/runs/10710030193/job/29695941814
https://github.com/curl/curl/actions/runs/10705668823/job/29681622201
- GHA/Linux old: 44s -> 13s (bundle+no unity) (run tests: 5m5 -> 5m6)
https://github.com/curl/curl/actions/runs/10718264094/job/29719794727
https://github.com/curl/curl/actions/runs/10718653175/job/29721009613
- GHA/macOS: 32s -> 2s (run tests: 2m43 -> 2m40)
https://github.com/curl/curl/actions/runs/10710030195/job/29695931956
https://github.com/curl/curl/actions/runs/10705668813/job/29681638937
- GHA/non-native (*BSD): inconclusive (full workflow time, ~qemu)
https://github.com/curl/curl/actions/runs/10710030198
https://github.com/curl/curl/actions/runs/10705668811
- GHA/Cygwin: 3m9 -> 32s
https://github.com/curl/curl/actions/runs/10710030212/job/29695929075
https://github.com/curl/curl/actions/runs/10705668809/job/29681609965
- GHA/MSYS2: 2m24 -> 14s
https://github.com/curl/curl/actions/runs/10710030190/job/29704850996
https://github.com/curl/curl/actions/runs/10705668808/job/29681624295
- GHA/mingw-w64: 3m56 -> 30s (run tests: 4m2 -> 3m52)
https://github.com/curl/curl/actions/runs/10710030190/job/29704852219
https://github.com/curl/curl/actions/runs/10705668808/job/29681631393
- GHA/mingw-w64-old: 7m19 -> 1m44 (run tests: 3m30 -> 2m53)
https://github.com/curl/curl/actions/runs/10710030190/job/29704849763
https://github.com/curl/curl/actions/runs/10705668808/job/29681622329
- GHA/MSVC: 3m22 -> 13s (run tests: 9m43 -> 4m22)
https://github.com/curl/curl/actions/runs/10710030190/job/29704850411
https://github.com/curl/curl/actions/runs/10705668808/job/29681623313
- AppVeyor CI MSVC 2008: 4m3 -> 45s (full build)
- AppVeyor CI MSVC 2010: 2m56 -> 1m8 (full build)
- AppVeyor CI MSVC 2022: 10m19 -> 2m23 (full build)
https://ci.appveyor.com/project/curlorg/curl/builds/50538455
https://ci.appveyor.com/project/curlorg/curl/builds/50536558
- AppVeyor CI total build time: 10m30 (master) -> 6m48 (unity) -> 4m5 (bundle) -> 3m24 (bundle+unity) -> 5m7 (bundle+unity+all jobs building tests)

Closes #14772

show more ...

tests: delete `libhostname.so` and `chkhostname`

Before this patch, `libhostname.so` and `chkhostname` were a test
facility for overriding `gethostname()` in non-debug builds on
Linu

tests: delete `libhostname.so` and `chkhostname`

Before this patch, `libhostname.so` and `chkhostname` were a test
facility for overriding `gethostname()` in non-debug builds on
Linux and other Unix platforms supporting `LD_PRELOAD`.

`gethostname()` has a single use with SMTP.

The alternative way to override `gethostname()` is building in debug
mode, which allows to do this via the `CURL_GETHOSTNAME` env, on all
platforms.

Drop the `LD_PRELOAD` solution in favour of the above.

Also:
- delete inactive NTLM code with a `gethostname()` call made from it.
- streamline NTLM code by dropping a `printf()` and a macro.
- tests: stop setting `CURL_GETHOSTNAME` where unnecessary.

Closes #14695

show more ...

lib: remove CURLX_NO_MEMORY_CALLBACKS

The only user of this define was 'chkdecimalpoint' - a special purpose
test tool that was built but not used anymore (since 17c18fbc3 - Apr
2020

lib: remove CURLX_NO_MEMORY_CALLBACKS

The only user of this define was 'chkdecimalpoint' - a special purpose
test tool that was built but not used anymore (since 17c18fbc3 - Apr
2020).

Closes #10908

show more ...

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- save

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205

show more ...

test3100: RTSP Basic authentication

Closes #9449

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869

show more ...

CURLOPT_PREREQFUNCTION: add new callback

Triggered before a request is made but after a connection is set up

Changes:

- callback: Update docs and callback for pre-request c

CURLOPT_PREREQFUNCTION: add new callback

Triggered before a request is made but after a connection is set up

Changes:

- callback: Update docs and callback for pre-request callback
- Add documentation for CURLOPT_PREREQDATA and CURLOPT_PREREQFUNCTION,
- Add redirect test and callback failure test
- Note that the function may be called multiple times on a redirection
- Disable new 2086 test due to Windows weirdness

Closes #7477

show more ...

git: ignore libtests in 3XXX area

Currently the file tests/libtest/lib3010 is not getting
ignored by git. This fixes it by adding the 3XXX area to
the according .gitignore file.

git: ignore libtests in 3XXX area

Currently the file tests/libtest/lib3010 is not getting
ignored by git. This fixes it by adding the 3XXX area to
the according .gitignore file.

Closes #5859

show more ...

libtest: Add chkdecimalpoint to .gitignore

Closes #2998

tests/libtest/.gitignore: follow-up fix to ignore lib5* too

tests: update .gitignore for libtests

Closes #2624

gitignore: ignore more generated files

tests: update .gitignore for libtests

tests: adjust .gitignore for new tests

gitignore: Sort for readability

find . -name .gitignore -print0 | xargs -i -0 sort -o '{}' '{}'

libtest: gitignore more binary files

Multiple pipelines and limiting the number of connections.

Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
opt

Multiple pipelines and limiting the number of connections.

Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
optimize the balance between the penalty for opening new
connections and the potential pipelining latency.

Two new options for limiting the number of connections:

CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections
to the same host. When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished, so we can reuse the connection.

CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total.
When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished. The free connection will then be reused, if possible, or
closed if the pending handle can't reuse it.

Several new options for pipelining:

CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a
pipeline is "full" when a connection is to be reused, a new connection
will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it.
If not, the handle will be put in a pending state until a connection is
ready (either free or a pipe got shorter).

CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a transfer with a content
length that is larger than this.

CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a chunk larger than this.

CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow
pipelining.

CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow
pipelining.

See the curl_multi_setopt() man page for details.

show more ...

FTP: prevent the multi interface from blocking

As pointed out in Bug report #3579064, curl_multi_perform() would
wrongly use a blocking mechanism internally for some commands which
c

FTP: prevent the multi interface from blocking

As pointed out in Bug report #3579064, curl_multi_perform() would
wrongly use a blocking mechanism internally for some commands which
could lead to for example a very long block if the LIST response never
showed.

The solution was to make sure to properly continue to use the multi
interface non-blocking state machine.

The new test 1501 verifies the fix.

Bug: http://curl.haxx.se/bug/view.cgi?id=3579064
Reported by: Guido Berhoerster

show more ...

Unit test for curl_multi_wait()

NTLM: verify multiple connections work

Add test2032 to test that NTLM does not switch connections in the middle
of the handshake

Add tests of auth retries

test: remove test 580

Test 580 is removed again for two reasons:

1) Some compilers aren't satisfied by just a data variable called 'test'
when first.o wants a function called 't

test: remove test 580

Test 580 is removed again for two reasons:

1) Some compilers aren't satisfied by just a data variable called 'test'
when first.o wants a function called 'test'. The Solaris compiler says
"ld: warning: symbol `test' has differing types:" while the AIX compiler
downright rejects it.

2) Test case 1119 that was added after this test is way more complete
and cover everything test 580 does and more without introducing the same
problems.

show more ...

ignore: lib580.c is generated by mk580.pl

NTLM tests: boost coverage by forcing the hostname

A shared library tests/libtest/.libs/lihostname.so is preloaded in NTLM
test-cases to override the system implementation of gethostname

NTLM tests: boost coverage by forcing the hostname

A shared library tests/libtest/.libs/lihostname.so is preloaded in NTLM
test-cases to override the system implementation of gethostname(). It
makes it possible to test the NTLM authentication for exact match, and
this way test the implementation of MD4 and DES.

If LD_PRELOAD doesn't work, a debug build willl also workk as debug
builds are now made to prefer a specific environment variable and will
then return that content as host name instead of the actual one.

Kamil wrote the bulk of this, Daniel Stenberg polished it.

show more ...

ignore more files generated when tests run in the source tree