RELEASE-NOTES: synced

vtls: initial implementation of rustls backend

This adds a new TLS backend, rustls. It uses the C-to-rustls bindings
from https://github.com/abetterinternet/crustls.

Rustls is a

vtls: initial implementation of rustls backend

This adds a new TLS backend, rustls. It uses the C-to-rustls bindings
from https://github.com/abetterinternet/crustls.

Rustls is at https://github.com/ctz/rustls/.

There is still a fair bit to be done, like sending CloseNotify on
connection shutdown, respecting CAPATH, and properly indicating features
like "supports TLS 1.3 ciphersuites." But it works well enough to make
requests and receive responses.

Blog post for context:
https://www.abetterinternet.org/post/memory-safe-curl/

Closes #6350

show more ...

sasl: support SCRAM-SHA-1 and SCRAM-SHA-256 via libgsasl

Closes #6372

lib: use int type for more port variables

This is a follow-up to 764c6bd. Prior to that change port variables
were usually type long.

Closes https://github.com/curl/curl/pull/65

lib: use int type for more port variables

This is a follow-up to 764c6bd. Prior to that change port variables
were usually type long.

Closes https://github.com/curl/curl/pull/6553

show more ...

tool_writeout: refactor write-out and write-out json

- Deduplicate the logic used by write-out and write-out json.

Rather than have separate writeLong, writeString, etc, logic for

tool_writeout: refactor write-out and write-out json

- Deduplicate the logic used by write-out and write-out json.

Rather than have separate writeLong, writeString, etc, logic for
each of write-out and write-out json instead have respective shared
functions that can output either format and a 'use_json' parameter to
indicate whether it is json that is output.

This will make it easier to maintain. Rather than have to go through
two sets of logic now we only have to go through one.

- Support write-out %{errormsg} and %{exitcode} in json.

- Clarify in the doc that %{exitcode} is the exit code of the transfer.

Prior to this change it just said "The numerical exitcode" which
implies it's the exit code of the tool, and it's not necessarily that.

Closes https://github.com/curl/curl/pull/6544

show more ...

lib: drop USE_SOCKETPAIR in favor of CURL_DISABLE_SOCKETPAIR

.. since the former is undocumented and they both do the same thing.

Closes https://github.com/curl/curl/pull/6517

curl_multibyte: fall back to local code page stat/access on Windows

If libcurl is built with Unicode support for Windows then it is assumed
the filename string is Unicode in UTF-8 encodi

curl_multibyte: fall back to local code page stat/access on Windows

If libcurl is built with Unicode support for Windows then it is assumed
the filename string is Unicode in UTF-8 encoding and it is converted to
UTF-16 to be passed to the wide character version of the respective
function (eg wstat). However the filename string may actually be in the
local encoding so, even if it successfully converted to UTF-16, if it
could not be stat/accessed then try again using the local code page
version of the function (eg wstat fails try stat).

We already do this with fopen (ie wfopen fails try fopen), so I think it
makes sense to extend it to stat and access functions.

Closes https://github.com/curl/curl/pull/6514

show more ...

file: Support unicode urls on windows

Closes https://github.com/curl/curl/pull/6501

cmake: fix import library name for non-MS compiler on Windows

- Use _imp.lib suffix only for Microsoft's compiler (MSVC).

Prior to this change library suffix _imp.lib was used for t

cmake: fix import library name for non-MS compiler on Windows

- Use _imp.lib suffix only for Microsoft's compiler (MSVC).

Prior to this change library suffix _imp.lib was used for the import
library on Windows regardless of compiler.

With this change the other compilers should now use their default
suffix which should be .dll.a.

This change is motivated by the usage of pkg-config on MSYS2.
Indeed, when 'pkg-config --libs libcurl' is used, -lcurl is
passed to ld. The documentation of ld on Windows :

https://sourceware.org/binutils/docs/ld/WIN32.html

lists, in the 'direct linking to a dll' section, the pattern
of the searched import library, and libcurl_imp.lib is not there.

Closes https://github.com/curl/curl/pull/6225

show more ...

urldata: move 'followlocation' to UrlState

As this is a state variable it does not belong in UserDefined which is
used to store values set by the user.

Closes #6582

http_proxy: fix typo in http_proxy.c

settting -> setting

Closes #6583

tests/server: Bump MAX_TAG_LEN to 200

This is useful for tests containing HTML inside of <data> sections.
For <img> tags it's not uncommon to be longer than the previous
limit of 79

tests/server: Bump MAX_TAG_LEN to 200

This is useful for tests containing HTML inside of <data> sections.
For <img> tags it's not uncommon to be longer than the previous
limit of 79 bytes.

An example of a previously problem-causing tag is:
<img src="http://config.privoxy.org/send-banner?type=auto" border="0" title="Killed-http://www.privoxy.org/images/privoxy.png-by-size" width="88" height="31">
which is needed for a Privoxy test for the banners-by-size filter.

Previously it caused server failures like:
12:29:05.786961 ====> Client connect
12:29:05.787116 accept_connection 3 returned 4
12:29:05.787194 accept_connection 3 returned 0
12:29:05.787285 Read 119 bytes
12:29:05.787345 Process 119 bytes request
12:29:05.787407 Got request: GET /banners-by-size/9 HTTP/1.1
12:29:05.787464 Requested test number 9 part 0
12:29:05.787686 getpart() failed with error: -2
12:29:05.787744 - request found to be complete (9)
12:29:05.787912 getpart() failed with error: -2
12:29:05.788048 Wrote request (119 bytes) input to log/server.input
12:29:05.788157 Send response test9 section <data>
12:29:05.788443 getpart() failed with error: -2
12:29:05.788498 instructed to close connection after server-reply
12:29:05.788550 ====> Client disconnect 0
12:29:05.871448 exit_signal_handler: 15
12:29:05.871714 signalled to die
12:29:05.872040 ========> IPv4 sws (port 21108 pid: 51758) exits with signal (15)

show more ...

tests/badsymbols.pl: when opening '$incdir' fails include it in the error message

runtests.1: document -o, -P, -L, and -E

runtests.pl: add %TESTNUMBER variable to make copying tests more convenient

runtests.pl: add an -o option to change internal variables

runtests.pl has lots of internal variables one might want to
change in certain situations, but adding a dedicated option
fo

runtests.pl: add an -o option to change internal variables

runtests.pl has lots of internal variables one might want to
change in certain situations, but adding a dedicated option
for every single one of them isn't practical.

Usage:
./runtests.pl -o TESTDIR=$privoxy_curl_test_dir -o HOSTIP=10.0.0.1 ...

show more ...

runtests.pl: cleanups

- show the summarized test result in the last line of the report
- do not use $_ after mapping it to a named variable
Doing that makes the code harder to foll

runtests.pl: cleanups

- show the summarized test result in the last line of the report
- do not use $_ after mapping it to a named variable
Doing that makes the code harder to follow.
- log the restraints sorted by the number of their occurrences
- fix language when logging restraints that only occured once
- let runhttpserver() use $TESTDIR instead of $srcdir
... so it works if a non-default $TESTDIR is being used.

show more ...

runtests.pl: add an -E option to specify an exclude file

It can contain additional restraints for test numbers,
keywords and tools.

The idea is to let third parties like the Pri

runtests.pl: add an -E option to specify an exclude file

It can contain additional restraints for test numbers,
keywords and tools.

The idea is to let third parties like the Privoxy project
distribute an exclude file with their tarballs that specifies
which curl tests are not expected to work when using Privoxy
as a proxy, without having to fork the whole curl test suite.

The syntax could be changed to be extendable and maybe
more closely reflect the "curl test" syntax. Currently
it's a bunch of lines like these:

test:$TESTNUMBER:Reason why this test with number $TESTNUMBER should be skipped
keyword:$KEYWORD:Reason why tests whose keywords contain the $KEYWORD should be skipped
tool:$TOOL:Reason why tests with tools that contain $TOOL should be skipped

To specify multiple $TESTNUMBERs, $KEYWORDs and $TOOLs
on a single line, split them with commas.

show more ...

runtests.pl: add -L parameter to require additional perl libraries

This is useful to change the behaviour of the script without
having to modify the file itself, for example to use a cus

runtests.pl: add -L parameter to require additional perl libraries

This is useful to change the behaviour of the script without
having to modify the file itself, for example to use a custom
compareparts() function that ignores header differences that
are expected to occur when an external proxy is being used.

Such differences are proxy-specific and thus the modifications
should be maintained together with the proxy.

show more ...

runtests.pl: add a -P option to specify an external proxy

... that should be used when executing the tests.

The assumption is that the proxy is an HTTP proxy.

This option s

runtests.pl: add a -P option to specify an external proxy

... that should be used when executing the tests.

The assumption is that the proxy is an HTTP proxy.

This option should be used together with -L to provide
a customized compareparts() version that knows which
proxy-specific header differences should be ignored.

This option doesn't work for all test types yet.

show more ...

tests: fixup several tests

missing CRs and modified %hostip

lib556/test556: use a real HTTP version to make test reuse more convenient

make sure the weekday in Date headers

tests: fixup several tests

missing CRs and modified %hostip

lib556/test556: use a real HTTP version to make test reuse more convenient

make sure the weekday in Date headers matches the date

test61: replace stray "^M" (5e 4d) at the end of a cookie with a '^M' (0d)

Gets the test working with external proxies like Privoxy again.

Closes #6463

show more ...

ftp: never set data->set.ftp_append outside setopt

Since the set value then risks getting used like that when the easy
handle is reused by the application.

Also: renamed the str

ftp: never set data->set.ftp_append outside setopt

Since the set value then risks getting used like that when the easy
handle is reused by the application.

Also: renamed the struct field from 'ftp_append' to 'remote_append'
since it is also used for SSH protocols.

Closes #6579

show more ...

urldata: remove the 'rtspversion' field

from struct connectdata and the corresponding code in http.c that set
it. It was never used for anything!

Closes #6581

CURLOPT_QUOTE.3: clarify that libcurl doesn't parse what's sent

... so passed in commands may confuse libcurl's knowledge of state.

Reported-by: Bodo Bergmann
Fixes #6577
Cl

CURLOPT_QUOTE.3: clarify that libcurl doesn't parse what's sent

... so passed in commands may confuse libcurl's knowledge of state.

Reported-by: Bodo Bergmann
Fixes #6577
Closes #6580

show more ...

vtls: factor out Curl_ssl_getsock to field of Curl_ssl

Closes #6558