copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869

show more ...

test390: verify --parallel

Closes #8985

curl: add --rate to set max request rate per time unit

--rate "12/m" - for 12 per minute or
--rate "5/h" - for 5 per hour

Removed from TODO

Closes #8671

curl: deprecate --random-file and --egd-file

As libcurl no longer has any functionality for them, the tool now does
nothing with them.

Closes #8670

tool_operate: make sure --fail-with-body works with --retry

... in the same way --fail already does.

Reported-by: Jakub Bochenski
Fixes #8845
Closes #8847

post_per_transfer: remove the updated file name

When --remove-on-error is used with --no-clobber, it might have an
updated file name to remove.

Bug: https://curl.se/docs/CVE-202

post_per_transfer: remove the updated file name

When --remove-on-error is used with --no-clobber, it might have an
updated file name to remove.

Bug: https://curl.se/docs/CVE-2022-27778.html

CVE-2022-27778

Reported-by: Harry Sintonen

Closes #8824

show more ...

tool_paramhlp: use feof(3) to identify EOF correctly when using fread(3)

This loop was using the number of bytes read from the file as condition
to keep reading.

From Linux's fr

tool_paramhlp: use feof(3) to identify EOF correctly when using fread(3)

This loop was using the number of bytes read from the file as condition
to keep reading.

From Linux's fread(3) man page:
> On success, fread() and fwrite() return the number of items read or
> written. This number equals the number of bytes transferred only when
> size is 1. If an error occurs, or the end of the file is reached, the
> return value is a short item count (or zero).
>
> The file position indicator for the stream is advanced by the number
> of bytes successfully read or written.
>
> fread() does not distinguish between end-of-file and error, and
> callers must use feof(3) and ferror(3) to determine which occurred.

This means that nread!=0 doesn't make much sense as an end condition for
the loop: nread==0 doesn't necessarily mean that EOF has been reached or
an error has occured (but that is usually the case) and nread!=0 doesn't
necessarily mean that EOF has not been reached or that no read errors
have occured. feof(3) and ferror(3) should be uses when using fread(3).

Currently curl has to performs an extra fread(3) call to get a return
value equal to 0 to stop looping.

This usually "works" (even though nread==0 shouldn't be interpreted as
EOF) if stdin is a pipe because EOF usually marks the "real" end of the
stream, so the extra fread(3) call will return immediately and the extra
read syscall won't be noticeable:

bash-5.1$ strace -e read curl -s -F file=@- 0x0.st <<< a 2>&1 |
> tail -n 5
read(0, "a\n", 4096) = 2
read(0, "", 4096) = 0
read(0, "", 4096) = 0
http://0x0.st/oRs.txt
+++ exited with 0 +++
bash-5.1$

But this doesn't work if curl is reading from stdin, stdin is a
terminal, and the EOF is being emulated using a shell with ^D. Two
consecutive ^D will be required in this case to actually make curl stop
reading:

bash-5.1$ curl -F file=@- 0x0.st
a
^D^D
http://0x0.st/oRs.txt
bash-5.1$

A possible workaround to this issue is to use a program that handles EOF
correctly to indirectly send data to curl's stdin:

bash-5.1$ cat - | curl -F file=@- 0x0.st
a
^D
http://0x0.st/oRs.txt
bash-5.1$

This patch makes curl handle EOF properly when using fread(3) in
file2memory() so that the workaround is not necessary.

Since curl was previously ignoring read errors caused by this fread(3),
ferror(3) is also used in the condition of the loop: read errors and EOF
will have the same meaning; this is done to somewhat preserve the old
behaviour instead of making the command fail when a read error occurs.

Closes #8701

show more ...

curl: error out if -T and -d are used for the same URL

As one implies PUT and the other POST, both cannot be used
simultaneously.

Add test 378 to verify.

Reported-by: B

curl: error out if -T and -d are used for the same URL

As one implies PUT and the other POST, both cannot be used
simultaneously.

Add test 378 to verify.

Reported-by: Boris Verkhovskiy
Fixes #8704
Closes #8715

show more ...

curl: fix segmentation fault for empty output file names.

Function glob_match_url set *result to NULL when called with filename =
"", producing an indirect NULL pointer dereference.

curl: fix segmentation fault for empty output file names.

Function glob_match_url set *result to NULL when called with filename =
"", producing an indirect NULL pointer dereference.

Closes #8606

show more ...

curl: add --remove-on-error

If a transfer returns an error, using this option makes curl remove the
leftover downloded (partial) local file before exiting.

Added test 376 to ver

curl: add --remove-on-error

If a transfer returns an error, using this option makes curl remove the
leftover downloded (partial) local file before exiting.

Added test 376 to verify

Closes #8503

show more ...

tool_operate: fix a scan-build warning

... and avoid the temp storing of the return code in a diff variable.

Closes #8565

curl: error out when options need features not present in libcurl

Trying to use a proxy when libcurl was built with proxy support disabled
should make curl error out properly.

R

curl: error out when options need features not present in libcurl

Trying to use a proxy when libcurl was built with proxy support disabled
should make curl error out properly.

Remove knowledge of disabled features from the tool code and instead
make it properly respond to what libcurl returns. Update all tests to
properly require the necessary features to be present/absent so that the
test suite can still be run even with libcurl builds with disabled
features.

Ref: https://curl.se/mail/archive-2022-03/0013.html
Closes #8565

show more ...

curl: remove "separators" (when using globbed URLs)

Unless muted (with -s) When doing globbing, curl would output mime-like
separators between the separate transfers. This is not documen

curl: remove "separators" (when using globbed URLs)

Unless muted (with -s) When doing globbing, curl would output mime-like
separators between the separate transfers. This is not documented
anywhere, surprises users and clobbers the output. Gone now.

Updated test 18 and 1235

Reported-by: jonny112 on github
Bug: https://github.com/curl/curl/discussions/8257
Closes #8278

show more ...

tool_operate: warn if too many output arguments were found

More output instructions than URLs is likely a user error.

Add test case 371 to verify

Closes #8210

tool_operate: fix potential memory-leak

A 'CURLU *' would leak if url_proto() is called with no URL.

Detected by Coverity. CID 1494643.
Follow-up to 18270893abdb19
Closes #8

tool_operate: fix potential memory-leak

A 'CURLU *' would leak if url_proto() is called with no URL.

Detected by Coverity. CID 1494643.
Follow-up to 18270893abdb19
Closes #8098

show more ...

tool_findfile: search for a file in the homedir

The homedir() function is now renamed into findfile() and iterates over
all the environment variables trying to access the file in questio

tool_findfile: search for a file in the homedir

The homedir() function is now renamed into findfile() and iterates over
all the environment variables trying to access the file in question
until it finds it. Last resort is then getpwuid() if
available. Previously it would first try to find a home directory and if
that was set, insist on checking only that directory for the file. This
now returns the full file name it finds.

The Windows specific checks are now done differently too and in this
order:

1 - %USERPROFILE%
2 - %APPDATA%
3 - %USERPROFILE%\\Application Data

The windows order is modified to match how the Windows 10 ssh tool works
when it searches for .ssh/known_hosts.

Reported-by: jeffrson on github
Co-authored-by: Jay Satiro
Fixes #8033
Closes #8035

show more ...

tool_operate: only set SSH related libcurl options for SSH URLs

For example, this avoids trying to find and set the known_hosts file (or
warn for its absence) if SFTP or SCP are not used

tool_operate: only set SSH related libcurl options for SSH URLs

For example, this avoids trying to find and set the known_hosts file (or
warn for its absence) if SFTP or SCP are not used.

Closes #8040

show more ...

mime: use percent-escaping for multipart form field and file names

Until now, form field and file names where escaped using the
backslash-escaping algorithm defined for multipart mails.

mime: use percent-escaping for multipart form field and file names

Until now, form field and file names where escaped using the
backslash-escaping algorithm defined for multipart mails. This commit
replaces this with the percent-escaping method for URLs.

As this may introduce incompatibilities with server-side applications, a
new libcurl option CURLOPT_MIME_OPTIONS with bitmask
CURLMIMEOPT_FORMESCAPE is introduced to revert to legacy use of
backslash-escaping. This is controlled by new cli tool option
--form-escape.

New tests and documentation are provided for this feature.

Reported by: Ryan Sleevi
Fixes #7789
Closes #7805

show more ...

tool_operate: reorder code to avoid compiler warning

tool_operate.c(889) : warning C4701: potentially uninitialized local
variable 'per' use

Follow-up to cc71d352651a0d95
Re

tool_operate: reorder code to avoid compiler warning

tool_operate.c(889) : warning C4701: potentially uninitialized local
variable 'per' use

Follow-up to cc71d352651a0d95
Reported-by: Marc Hörsken
Bug: https://github.com/curl/curl/pull/7922#issuecomment-963042676
Closes #7971

show more ...

tool_operate: fclose stream only if fopened

Fixes torture test failures
Follow-up to cc71d352651

Closes #7972

tool_operate: fix torture leaks with etags

Spotted by torture testing 343 344 345 347.

Follow-up from cc71d352651a0
Pointed-out-by: Dan Fandrich

Closes #7969

tool_operate: a failed etag save now only fails that transfer

When failing to create the output file for saving an etag, only fail
that particular single transfer and allow others to fol

tool_operate: a failed etag save now only fails that transfer

When failing to create the output file for saving an etag, only fail
that particular single transfer and allow others to follow.

In a serial transfer setup, if no transfer at all is done due to them
all being skipped because of this error, curl will output an error
message and return exit code 26.

Added test 369 and 370 to verify.

Reported-by: Earnestly on github
Ref: #7942
Closes #7945

show more ...

libssh2: add SHA256 fingerprint support

Added support for SHA256 fingerprint in command line curl and in
libcurl.

Closes #7646

curl: stop retry if Retry-After: is longer than allowed

If Retry-After: specifies a period that is longer than what fits within
--retry-max-time, then stop retrying immediately.

curl: stop retry if Retry-After: is longer than allowed

If Retry-After: specifies a period that is longer than what fits within
--retry-max-time, then stop retrying immediately.

Added test 366 to verify.

Reported-by: Kari Pahula
Fixes #7675
Closes #7676

show more ...

curl: better error message when -O fails to get a good name

Due to how this currently works internally, it needs a working initial
file name to store contents in, so it may still fail ev

curl: better error message when -O fails to get a good name

Due to how this currently works internally, it needs a working initial
file name to store contents in, so it may still fail even with -J is
used (and thus accepting a name from content-disposition:) if the file
name part of the URL isn't "good enough".

Fixes #7628
Closes #7635

show more ...