curl: add --skip-existing

With this option, the entire download is skipped if the selected target
filename already exists when the opertion is about to begin.

Test 994, 995 and

curl: add --skip-existing

With this option, the entire download is skipped if the selected target
filename already exists when the opertion is about to begin.

Test 994, 995 and 996 verify.

Ref: #11012
Closes #13993

show more ...

curl: --help [option] displays documentation for given cmdline option

Since the documentation text blob might be gzipped, it needs to search
for what to output in a streaming manner. It

curl: --help [option] displays documentation for given cmdline option

Since the documentation text blob might be gzipped, it needs to search
for what to output in a streaming manner. It then first searches for
"\nALL OPTIONS".

Then, it looks for the start to display at "\n -[option]" and stops
again at "\n -". Except for the last option in the man page, which
ends at "\nFILES" - the subtitle for the section following all options
in the manpage.

Test 1707 to 1710 verify

Closes #13997

show more ...

curl: support embedding a CA bundle

Add the ability to embed a CA bundle into the curl binary. It is used
when no other runtime or build-time option set one.

This helps curl-for

curl: support embedding a CA bundle

Add the ability to embed a CA bundle into the curl binary. It is used
when no other runtime or build-time option set one.

This helps curl-for-win macOS and Linux builds to run standalone, and
also helps Windows builds to avoid picking up the CA bundle from an
arbitrary (possibly world-writable) location (though this behaviour is
not currently disablable).

Usage:
- cmake: `-DCURL_CA_EMBED=/path/to/curl-ca-bundle.crt`
- autotools: `--with-ca-embed=/path/to/curl-ca-bundle.crt`
- Makefile.mk: `CURL_CA_EMBED=/path/to/curl-ca-bundle.crt`

Also add new command-line option `--dump-ca-embed` to dump the embedded
CA bundle to standard output.

Closes #14059

show more ...

tool: make parser reject blank arguments if not supported

Already in the getstr() function that clones the input argument.

Closes #12620

tool: add "variable" support

Add support for command line variables. Set variables with --variable
name=content or --variable name@file (where "file" can be stdin if set
to a single

tool: add "variable" support

Add support for command line variables. Set variables with --variable
name=content or --variable name@file (where "file" can be stdin if set
to a single dash (-)).

Variable content is expanded in option parameters using "{{name}}"
(without the quotes) if the option name is prefixed with
"--expand-". This gets the contents of the variable "name" inserted, or
a blank if the name does not exist as a variable. Insert "{{" verbatim
in the string by prefixing it with a backslash, like "\\{{".

Import an environment variable with --variable %name. It makes curl exit
with an error if the environment variable is not set. It can also rather
get a default value if the variable does not exist, using =content or
@file like shown above.

Example: get the USER environment variable into the URL:

--variable %USER
--expand-url = "https://example.com/api/{{USER}}/method"

When expanding variables, curl supports a set of functions that can make
the variable contents more convenient to use. It can trim leading and
trailing white space with "trim", output the contents as a JSON quoted
string with "json", URL encode it with "url" and base 64 encode it with
"b64". To apply functions to a variable expansion, add them colon
separated to the right side of the variable. They are then performed in
a left to right order.

Example: get the contents of a file called $HOME/.secret into a variable
called "fix". Make sure that the content is trimmed and percent-encoded
sent as POST data:

--variable %HOME=/home/default
--expand-variable fix@{{HOME}}/.secret
--expand-data "{{fix:trim:url}}"
https://example.com/

Documented. Many new test cases.

Co-brainstormed-by: Emanuele Torre
Assisted-by: Jat Satiro
Closes #11346

show more ...

tool_getparam: fix hiding of command line secrets

Closes #10276

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- save

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205

show more ...

getparam: correctly clean args

Follow-up to bf7e887b2442783ab52

The previous fix for #9128 was incomplete and caused #9397.

Fixes #9397
Closes #9399

tool_getparam: fix cleanarg() for unicode builds

Use the correct type, and make cleanarg an empty macro if the cleaning
ability is absent.

Fixes #9195
Closes #9196

tool_getparam: fix cleanarg() for unicode builds

Use the correct type, and make cleanarg an empty macro if the cleaning
ability is absent.

Fixes #9195
Closes #9196

Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad

show more ...

tool_getparam: repair cleanarg

Regression since 9e5669f.

Make sure the "cleaning" of command line arguments is done on the
original argv[] pointers. As a bonus, it also exits be

tool_getparam: repair cleanarg

Regression since 9e5669f.

Make sure the "cleaning" of command line arguments is done on the
original argv[] pointers. As a bonus, it also exits better on out of
memory error.

Reported-by: Litter White
Fixes #9128
Closes #9130

show more ...

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869

show more ...

tool_paramhlp: use feof(3) to identify EOF correctly when using fread(3)

This loop was using the number of bytes read from the file as condition
to keep reading.

From Linux's fr

tool_paramhlp: use feof(3) to identify EOF correctly when using fread(3)

This loop was using the number of bytes read from the file as condition
to keep reading.

From Linux's fread(3) man page:
> On success, fread() and fwrite() return the number of items read or
> written. This number equals the number of bytes transferred only when
> size is 1. If an error occurs, or the end of the file is reached, the
> return value is a short item count (or zero).
>
> The file position indicator for the stream is advanced by the number
> of bytes successfully read or written.
>
> fread() does not distinguish between end-of-file and error, and
> callers must use feof(3) and ferror(3) to determine which occurred.

This means that nread!=0 doesn't make much sense as an end condition for
the loop: nread==0 doesn't necessarily mean that EOF has been reached or
an error has occured (but that is usually the case) and nread!=0 doesn't
necessarily mean that EOF has not been reached or that no read errors
have occured. feof(3) and ferror(3) should be uses when using fread(3).

Currently curl has to performs an extra fread(3) call to get a return
value equal to 0 to stop looping.

This usually "works" (even though nread==0 shouldn't be interpreted as
EOF) if stdin is a pipe because EOF usually marks the "real" end of the
stream, so the extra fread(3) call will return immediately and the extra
read syscall won't be noticeable:

bash-5.1$ strace -e read curl -s -F file=@- 0x0.st <<< a 2>&1 |
> tail -n 5
read(0, "a\n", 4096) = 2
read(0, "", 4096) = 0
read(0, "", 4096) = 0
http://0x0.st/oRs.txt
+++ exited with 0 +++
bash-5.1$

But this doesn't work if curl is reading from stdin, stdin is a
terminal, and the EOF is being emulated using a shell with ^D. Two
consecutive ^D will be required in this case to actually make curl stop
reading:

bash-5.1$ curl -F file=@- 0x0.st
a
^D^D
http://0x0.st/oRs.txt
bash-5.1$

A possible workaround to this issue is to use a program that handles EOF
correctly to indirectly send data to curl's stdin:

bash-5.1$ cat - | curl -F file=@- 0x0.st
a
^D
http://0x0.st/oRs.txt
bash-5.1$

This patch makes curl handle EOF properly when using fread(3) in
file2memory() so that the workaround is not necessary.

Since curl was previously ignoring read errors caused by this fread(3),
ferror(3) is also used in the condition of the loop: read errors and EOF
will have the same meaning; this is done to somewhat preserve the old
behaviour instead of making the command fail when a read error occurs.

Closes #8701

show more ...

parse_args: redo the warnings for --remote-header-name combos

... to avoid the memory leak risk pointed out by scan-build.

Follow-up from 7a3e981781d6c18a

Closes #7698

copyright: fix year ranges

Follow-up from 4d2f8006777

curl.se: new home

Closes #6172

curl: report error for "--no-" on non-boolean options

Reported-by: Olen Andoni
Fixes #3906
Closes #3907

whitespace fixes

- replace tabs with spaces where possible
- remove line ending spaces
- remove double/triple newlines at EOF
- fix a non-UTF-8 character
- cleanup a few inde

whitespace fixes

- replace tabs with spaces where possible
- remove line ending spaces
- remove double/triple newlines at EOF
- fix a non-UTF-8 character
- cleanup a few indentations/line continuations
in manual examples

Closes https://github.com/curl/curl/pull/3037

show more ...

curl: detect and bail out early on parameter integer overflows

Make the number parser aware of the maximum limit curl accepts for a
value and return an error immediately if larger, inste

curl: detect and bail out early on parameter integer overflows

Make the number parser aware of the maximum limit curl accepts for a
value and return an error immediately if larger, instead of running an
integer overflow later.

Fixes #1730
Closes #1736

show more ...

tool: fix remaining -Wcast-qual warnings

Avoid casting away low-level const.

curl: non-boolean command line args reject --no- prefixes

... and instead properly respond with an error message to the user
instead of silently ignoring.

Fixes #1453
Closes

curl: non-boolean command line args reject --no- prefixes

... and instead properly respond with an error message to the user
instead of silently ignoring.

Fixes #1453
Closes #1458

show more ...

URLs: change all http:// URLs to https://

CURLOPT_DEFAULT_PROTOCOL: added

- Add new option CURLOPT_DEFAULT_PROTOCOL to allow specifying a default
protocol for schemeless URLs.

- Add new tool option --proto-default to ex

CURLOPT_DEFAULT_PROTOCOL: added

- Add new option CURLOPT_DEFAULT_PROTOCOL to allow specifying a default
protocol for schemeless URLs.

- Add new tool option --proto-default to expose
CURLOPT_DEFAULT_PROTOCOL.

In the case of schemeless URLs libcurl will behave in this way:

When the option is used libcurl will use the supplied default.

When the option is not used, libcurl will follow its usual plan of
guessing from the hostname and falling back to 'http'.

show more ...

tool_getparam: Added initial support for --next/-:

Added initial support for --next/-: which will be used to replace the
rather confusing : command line operation what was used for the U

tool_getparam: Added initial support for --next/-:

Added initial support for --next/-: which will be used to replace the
rather confusing : command line operation what was used for the URL
specific options prototype.

show more ...

tool_getparam: Added global config to getparameter()

In preparation for parsing global options added the GlobalConfig
structure to the getparameter() function.

tool_getparam.h: Fixed compilation warning

warning: declaration of 'struct GlobalConfig' will not be visible
outside of this function