curl: move IPFS code into src/tool_ipfs.[ch]

- convert ensure_trailing into ensure_trailing_slash
- strdup the URL string to own it proper
- use shorter variable names
- combine

curl: move IPFS code into src/tool_ipfs.[ch]

- convert ensure_trailing into ensure_trailing_slash
- strdup the URL string to own it proper
- use shorter variable names
- combine some expressions
- simplify error handling in ipfs_gateway()
- add MAX_GATEWAY_URL_LEN + proper bailout if maximum is reached
- ipfs-gateway.d polish and simplification
- shorten ipfs error message + make them "synthetic"

Closes #12281

show more ...

build: alpha-sort source files for lib and src

Closes #12014

cmake: fix duplicate symbols when linking tests

The linker resolves this automatically in non-unity builds. In unity
builds the linker cannot drop a single object with the duplicates,

cmake: fix duplicate symbols when linking tests

The linker resolves this automatically in non-unity builds. In unity
builds the linker cannot drop a single object with the duplicates,
resulting in these errors. The root issue is that we started including
certain objects both via both libcurlu and libcurltool libs.

Regression from 39e7c22bb459c2e818f079984989a26a09741860

Windows errors:
```
[ 3%] Linking C executable unit1303.exe
[ 3%] Building C object tests/server/CMakeFiles/rtspd.dir/__/__/lib/curl_multibyte.c.obj
../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_UTF8_to_wchar':
C:/projects/curl/lib/curl_multibyte.c:44: multiple definition of `curlx_convert_UTF8_to_wchar'
../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte.c:44: first defined here
../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_wchar_to_UTF8':
C:/projects/curl/lib/curl_multibyte.c:66: multiple definition of `curlx_convert_wchar_to_UTF8'
../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte.c:66: first defined here
../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_open':
C:/projects/curl/lib/curl_multibyte.c:92: multiple definition of `curlx_win32_open'
../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte.c:92: first defined here
../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_fopen':
C:/projects/curl/lib/curl_multibyte.c:120: multiple definition of `curlx_win32_fopen'
../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte.c:120: first defined here
../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_stat':
[...]
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/48110107/job/nvlhpt9aa4ehny5q#L247

macOS errors:
```
[ 56%] Linking C executable unit1302
duplicate symbol '_curlx_sotouz' in:
../../lib/libcurlu.a(unity_0_c.c.o)
../../src/libcurltool.a(unity_0_c.c.o)
duplicate symbol '_curlx_sitouz' in:
../../lib/libcurlu.a(unity_0_c.c.o)
../../src/libcurltool.a(unity_0_c.c.o)
duplicate symbol '_curlx_uztosz' in:
../../lib/libcurlu.a(unity_0_c.c.o)
../../src/libcurltool.a(unity_0_c.c.o)
[...]
```
with config:
```
-DCMAKE_UNITY_BUILD=ON \
-DENABLE_DEBUG=ON -DBUILD_TESTING=ON -DCMAKE_C_FLAGS=-DDEBUGBUILD \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_STATIC_LIBS=OFF
```

Closes #11926

show more ...

spelling: use 'reuse' not 're-use' in code and elsewhere

Unify the spelling as both versions were previously used intermittently

Closes #11717

tool: add "variable" support

Add support for command line variables. Set variables with --variable
name=content or --variable name@file (where "file" can be stdin if set
to a single

tool: add "variable" support

Add support for command line variables. Set variables with --variable
name=content or --variable name@file (where "file" can be stdin if set
to a single dash (-)).

Variable content is expanded in option parameters using "{{name}}"
(without the quotes) if the option name is prefixed with
"--expand-". This gets the contents of the variable "name" inserted, or
a blank if the name does not exist as a variable. Insert "{{" verbatim
in the string by prefixing it with a backslash, like "\\{{".

Import an environment variable with --variable %name. It makes curl exit
with an error if the environment variable is not set. It can also rather
get a default value if the variable does not exist, using =content or
@file like shown above.

Example: get the USER environment variable into the URL:

--variable %USER
--expand-url = "https://example.com/api/{{USER}}/method"

When expanding variables, curl supports a set of functions that can make
the variable contents more convenient to use. It can trim leading and
trailing white space with "trim", output the contents as a JSON quoted
string with "json", URL encode it with "url" and base 64 encode it with
"b64". To apply functions to a variable expansion, add them colon
separated to the right side of the variable. They are then performed in
a left to right order.

Example: get the contents of a file called $HOME/.secret into a variable
called "fix". Make sure that the content is trimmed and percent-encoded
sent as POST data:

--variable %HOME=/home/default
--expand-variable fix@{{HOME}}/.secret
--expand-data "{{fix:trim:url}}"
https://example.com/

Documented. Many new test cases.

Co-brainstormed-by: Emanuele Torre
Assisted-by: Jat Satiro
Closes #11346

show more ...

tool: improve --stderr handling

- freopen stderr with the user-specified file (--stderr file) instead of
using a separate 'errors' stream.

- In tool_setup.h override stdio.h's

tool: improve --stderr handling

- freopen stderr with the user-specified file (--stderr file) instead of
using a separate 'errors' stream.

- In tool_setup.h override stdio.h's stderr macro as global variable
tool_stderr.

Both freopen and overriding the stderr macro are necessary because if
the user-specified filename is "-" then stdout is assigned to
tool_stderr and no freopen takes place. See the PR for more information.

Ref: https://github.com/curl/curl/issues/10491

Closes https://github.com/curl/curl/pull/10673

show more ...

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- save

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205

show more ...

netware: remove leftover traces

Commit 3b16575ae938dec2a29454631a12aa52b6ab9c67 removed support for
building on Novell Netware, but a few leftover traces remained. This
removes the l

netware: remove leftover traces

Commit 3b16575ae938dec2a29454631a12aa52b6ab9c67 removed support for
building on Novell Netware, but a few leftover traces remained. This
removes the last bits.

Closes: #9966
Reviewed-by: Daniel Stenberg <daniel@haxx.se>

show more ...

curl_ctype: convert to macros-only

This no longer provide functions, only macros. Runs faster and produces
smaller output.

The biggest precaution this change brings:

DO

curl_ctype: convert to macros-only

This no longer provide functions, only macros. Runs faster and produces
smaller output.

The biggest precaution this change brings:

DO NOT use post/pre-increments when passing arguments to the macros.

Closes #9429

show more ...

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869

show more ...

timediff.[ch]: add curlx helper functions for timeval conversions

Also move timediff_t definitions from timeval.h to timediff.h and
then make timeval.h include the new standalone-capable

timediff.[ch]: add curlx helper functions for timeval conversions

Also move timediff_t definitions from timeval.h to timediff.h and
then make timeval.h include the new standalone-capable timediff.h.

Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Supersedes #5888
Closes #8595

show more ...

lib: remove support for CURL_DOES_CONVERSIONS

TPF was the only user and support for that was dropped.

Closes #8378

misc: remove the final watcom references

Follow-up to bbf8cae44dedc495e6

We removed support for the watcom builds files back in September
2020. This removes all remaining watcom

misc: remove the final watcom references

Follow-up to bbf8cae44dedc495e6

We removed support for the watcom builds files back in September
2020. This removes all remaining watcom references and ifdefs.

Closes #8287

show more ...

tool_findfile: search for a file in the homedir

The homedir() function is now renamed into findfile() and iterates over
all the environment variables trying to access the file in questio

tool_findfile: search for a file in the homedir

The homedir() function is now renamed into findfile() and iterates over
all the environment variables trying to access the file in question
until it finds it. Last resort is then getpwuid() if
available. Previously it would first try to find a home directory and if
that was set, insist on checking only that directory for the file. This
now returns the full file name it finds.

The Windows specific checks are now done differently too and in this
order:

1 - %USERPROFILE%
2 - %APPDATA%
3 - %USERPROFILE%\\Application Data

The windows order is modified to match how the Windows 10 ssh tool works
when it searches for .ssh/known_hosts.

Reported-by: jeffrson on github
Co-authored-by: Jay Satiro
Fixes #8033
Closes #8035

show more ...

tool_listhelp: easier to generate with gen.pl

tool_listhelp.c is now a separate file with only the command line --help
output, exactly as generated by gen.pl. This makes it easier to gen

tool_listhelp: easier to generate with gen.pl

tool_listhelp.c is now a separate file with only the command line --help
output, exactly as generated by gen.pl. This makes it easier to generate
updates according to what's in the docs/cmdline-opts docs.

cd $srcroot/docs/cmdline-opts
./gen.pl listhelp *.d > $srcroot/src/tool_listhelp.c

With a configure build, this also works:

make -C src listhelp

Closes #7787

show more ...

metalink: remove

Warning: this will make existing curl command lines that use metalink to
stop working.

Reasons for removal:

1. We've found several security problems an

metalink: remove

Warning: this will make existing curl command lines that use metalink to
stop working.

Reasons for removal:

1. We've found several security problems and issues involving the
metalink support in curl. The issues are not detailed here. When
working on those, it become apparent to the team that several of the
problems are due to the system design, metalink library API and what
the metalink RFC says. They are very hard to fix on the curl side
only.

2. The metalink usage with curl was only very briefly documented and was
not following the "normal" curl usage pattern in several ways, making
it surprising and non-intuitive which could lead to further security
issues.

3. The metalink library was last updated 6 years ago and wasn't so
active the years before that either. An unmaintained library means
there's a security problem waiting to happen. This is probably reason
enough.

4. Metalink requires an XML parsing library, which is complex code (even
the smaller alternatives) and to this day often gets security
updates.

5. Metalink is not a widely used curl feature. In the 2020 curl user
survey, only 1.4% of the responders said that they'd are using it. In
2021 that number was 1.2%. Searching the web also show very few
traces of it being used, even with other tools.

6. The torrent format and associated technology clearly won for
downloading large files from multiple sources in parallel.

Cloes #7176

show more ...

curl.se: new home

Closes #6172

curl: use curlx_dynbuf for realloc when loading config files

... fixes an integer overflow at the same time.

Reported-by: ihsinme on github
Assisted-by: Jay Satiro

Clos

curl: use curlx_dynbuf for realloc when loading config files

... fixes an integer overflow at the same time.

Reported-by: ihsinme on github
Assisted-by: Jay Satiro

Closes #5946

show more ...

win32: Add Curl_verify_windows_version() to curlx

Closes https://github.com/curl/curl/pull/5754

curl_multibyte: add to curlx

This will also be needed in the tool and tests.

Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512
Closes https://github.com/curl/cu

curl_multibyte: add to curlx

This will also be needed in the tool and tests.

Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512
Closes https://github.com/curl/curl/pull/3784

show more ...

copyright: fix out-of-date copyright ranges and missing headers

Reported by the new script 'scripts/copyright.pl'. The script has a
regex whitelist for the files that don't need copyrigh

copyright: fix out-of-date copyright ranges and missing headers

Reported by the new script 'scripts/copyright.pl'. The script has a
regex whitelist for the files that don't need copyright headers.

Removed three (mostly usesless) README files from docs/

Closes #5141

show more ...

writeout: support to generate JSON output

This commit adds support to generate JSON via the writeout feature:

-w "%{json}"

It leverages the existing infrastructure as m

writeout: support to generate JSON output

This commit adds support to generate JSON via the writeout feature:

-w "%{json}"

It leverages the existing infrastructure as much as possible. Thus,
generating the JSON on STDERR is possible by:

-w "%{stderr}%{json}"

This implements a variant of
https://github.com/curl/curl/wiki/JSON#--write-out-json.

Closes #4870

show more ...

curl: support parallel transfers

This is done by making sure each individual transfer is first added to a
linked list as then they can be performed serially, or at will, in
parallel.

curl: support parallel transfers

This is done by making sure each individual transfer is first added to a
linked list as then they can be performed serially, or at will, in
parallel.

Closes #3804

show more ...

whitespace fixes

- replace tabs with spaces where possible
- remove line ending spaces
- remove double/triple newlines at EOF
- fix a non-UTF-8 character
- cleanup a few inde

whitespace fixes

- replace tabs with spaces where possible
- remove line ending spaces
- remove double/triple newlines at EOF
- fix a non-UTF-8 character
- cleanup a few indentations/line continuations
in manual examples

Closes https://github.com/curl/curl/pull/3037

show more ...

time-cond: fix reading the file modification time on Windows

On Windows, stat() may adjust the unix file time by a daylight saving time
offset. Avoid this by calling GetFileTime() instea

time-cond: fix reading the file modification time on Windows

On Windows, stat() may adjust the unix file time by a daylight saving time
offset. Avoid this by calling GetFileTime() instead.

Fixes #2164
Closes #2204

show more ...