http: add support to read and store the referrer header

- add CURLINFO_REFERER libcurl option
- add --write-out '%{referer}' command-line option
- extend --xattr command-line option

http: add support to read and store the referrer header

- add CURLINFO_REFERER libcurl option
- add --write-out '%{referer}' command-line option
- extend --xattr command-line option to fill user.xdg.referrer.url extended
attribute with the referrer (if there was any)

Closes #6591

show more ...

urldata: remove the _ORIG suffix from string names

It doesn't provide any useful info but only makes the names longer.

Closes #6624

url: fix memory leak if OOM in the HSTS handling

Reported-by: Viktor Szakats
Bug: https://github.com/curl/curl/pull/6627#issuecomment-781626205

Closes #6628

gnutls: assume nettle crypto support

nettle has been the default crypto library with GnuTLS since 2010. By
dropping support for the previous libcrypto, we simplify code.

Closes

gnutls: assume nettle crypto support

nettle has been the default crypto library with GnuTLS since 2010. By
dropping support for the previous libcrypto, we simplify code.

Closes #6625

show more ...

asyn-ares: use consistent resolve error message

... with the help of Curl_resolver_error() which now is moved from
asyn-thead.c and is provided globally for this purpose.

Follow

asyn-ares: use consistent resolve error message

... with the help of Curl_resolver_error() which now is moved from
asyn-thead.c and is provided globally for this purpose.

Follow-up to 35ca04ce1b77636

Makes test 1188 work for c-ares builds

Closes #6626

show more ...

ci: stop building on freebsd-12-1

An updated freebsd-12-2 image was added a few months ago, and this
older one is consistently failing to go past `pkginstall`:
```
Newer FreeBSD

ci: stop building on freebsd-12-1

An updated freebsd-12-2 image was added a few months ago, and this
older one is consistently failing to go past `pkginstall`:
```
Newer FreeBSD version for package py37-mlt:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1202000
- running kernel: 1201000
Ignore the mismatch and continue? [Y/n]: pkg: repository FreeBSD contains packages for wrong OS version: FreeBSD:12:amd64
```

FreeBSD thread suggests that 12.1 is EOL, and best to avoid.

Ref: https://forums.freebsd.org/threads/78856/

Reviewed-by: Daniel Stenberg
Closes #6622

show more ...

test1188: change error from connect to resolve error

Using the %NOLISTENPORT to trigger a connection failure is somewhat
"risky" (since it isn't guaranteed to not be listened to) and cau

test1188: change error from connect to resolve error

Using the %NOLISTENPORT to trigger a connection failure is somewhat
"risky" (since it isn't guaranteed to not be listened to) and caused
occasional CI problems. This fix changes the infused error to be a more
reliable one but still verifies the --write-out functionality properly -
which is the purpose of this test.

Reported-by: Jay Satiro
Fixes #6621
Closes #6623

show more ...

url.c: use consistent error message for failed resolve

BUGS: language polish

wolfssl: don't store a NULL sessionid

This caused a memory leak as the session id cache entry was still
erroneously stored with a NULL sessionid and that would later be treated
as no

wolfssl: don't store a NULL sessionid

This caused a memory leak as the session id cache entry was still
erroneously stored with a NULL sessionid and that would later be treated
as not needed to get freed.

Reported-by: Gisle Vanem
Fixes #6616
Closes #6617

show more ...

parse_proxy: fix a memory leak in the OOM path

Reported-by: Jay Satiro
Reviewed-by: Jay Satiro
Reviewed-by: Emil Engler

Closes #6614
Bug: https://github.com/curl/curl/pu

parse_proxy: fix a memory leak in the OOM path

Reported-by: Jay Satiro
Reviewed-by: Jay Satiro
Reviewed-by: Emil Engler

Closes #6614
Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541

show more ...

url: fix possible use-after-free in default protocol

Prior to this change if the user specified a default protocol and a
separately allocated non-absolute URL was used then it was freed

url: fix possible use-after-free in default protocol

Prior to this change if the user specified a default protocol and a
separately allocated non-absolute URL was used then it was freed
prematurely, before it was then used to make the replacement URL.

Bug: https://github.com/curl/curl/issues/6604#issuecomment-780138219
Reported-by: arvids-kokins-bidstack@users.noreply.github.com

Closes https://github.com/curl/curl/pull/6613

show more ...

multi: rename the multi transfer states

While working on documenting the states it dawned on me that step one is
to use more descriptive names on the states. This also changes prefix on

multi: rename the multi transfer states

While working on documenting the states it dawned on me that step one is
to use more descriptive names on the states. This also changes prefix on
the states to make them shorter in the source.

State names NOT ending with *ing are transitional ones.

Closes #6612

show more ...

http: do not add a referrer header with empty value

Previously an empty 'Referer:' header was added to the HTTP request when
passing `--referer ';auto'` or `--referer ''` on the command-

http: do not add a referrer header with empty value

Previously an empty 'Referer:' header was added to the HTTP request when
passing `--referer ';auto'` or `--referer ''` on the command-line. This
patch makes `--referer` work like `--header 'Referer:'` and will only add
the header if it has a non-zero length value.

Reviewed-by: Jay Satiro
Closes #6610

show more ...

lib: remove 'conn->data' completely

The Curl_easy pointer struct entry in connectdata is now gone. Just
before commit 215db086e0 landed on January 8, 2021 there were 919
references t

lib: remove 'conn->data' completely

The Curl_easy pointer struct entry in connectdata is now gone. Just
before commit 215db086e0 landed on January 8, 2021 there were 919
references to conn->data.

Closes #6608

show more ...

openldap: pass 'data' to the callbacks instead of 'conn'

doh: Fix sharing user's resolve list with DOH handles

- Share the shared object from the user's easy handle with the DOH
handles.

Prior to this change if the user had set a sh

doh: Fix sharing user's resolve list with DOH handles

- Share the shared object from the user's easy handle with the DOH
handles.

Prior to this change if the user had set a shared object with shared
cached DNS (CURL_LOCK_DATA_DNS) for their easy handle then that wasn't
used by any associated DOH handles, since they used the multi's default
hostcache.

This change means all the handles now use the same hostcache, which is
either the shared hostcache from the user created shared object if it
exists or if not then the multi's default hostcache.

Reported-by: Manuj Bhatia

Fixes https://github.com/curl/curl/issues/6589
Closes https://github.com/curl/curl/pull/6607

show more ...

http2: remove conn->data use

... but instead use a private alternative that points to the "driving
transfer" from the connection. We set the "user data" associated with
the connectio

http2: remove conn->data use

... but instead use a private alternative that points to the "driving
transfer" from the connection. We set the "user data" associated with
the connection to be the connectdata struct, but when we drive transfers
the code still needs to know the pointer to the transfer. We can change
the user data to become the Curl_easy handle, but with older nghttp2
version we cannot dynamically update that pointer properly when
different transfers are used over the same connection.

Closes #6520

show more ...

openssl: remove conn->data use

We still make the trace callback function get the connectdata struct
passed to it, since the callback is anchored on the connection.

Repeatedly up

openssl: remove conn->data use

We still make the trace callback function get the connectdata struct
passed to it, since the callback is anchored on the connection.

Repeatedly updating the callback pointer to set 'data' with
SSL_CTX_set_msg_callback_arg() doesn't seem to work, probably because
there might already be messages in the queue with the old pointer.

This code therefore makes sure to set the "logger" handle before using
OpenSSL calls so that the right easy handle gets used for tracing.

Closes #6522

show more ...

RELEASE-NOTES: synced

doh: add options to disable ssl verification

- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
same as their re

doh: add options to disable ssl verification

- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
same as their respective counterparts.

- New curl tool options --doh-insecure and --doh-cert-status do the same
as their respective counterparts.

Prior to this change DOH SSL certificate verification settings for
verifyhost and verifypeer were supposed to be inherited respectively
from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug
were not. As a result DOH verification remained at the default, ie
enabled, and it was not possible to disable. This commit changes
behavior so that the DOH verification settings are independent and not
inherited.

Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676

Fixes https://github.com/curl/curl/issues/4578
Closes https://github.com/curl/curl/pull/6597

show more ...

hostip: fix crash in sync resolver builds that use DOH

- Guard some Curl_async accesses with USE_CURL_ASYNC instead of
!CURLRES_SYNCH.

This is another follow-up to 8335c64 whi

hostip: fix crash in sync resolver builds that use DOH

- Guard some Curl_async accesses with USE_CURL_ASYNC instead of
!CURLRES_SYNCH.

This is another follow-up to 8335c64 which moved the async struct from
the connectdata struct into the Curl_easy struct. A previous follow-up
6cd167a fixed building for sync resolver by guarding some async struct
accesses with !CURLRES_SYNCH. The problem is since DOH (DNS-over-HTTPS)
is available as an asynchronous secondary resolver the async struct may
be used even when libcurl is built for the sync resolver. That means
that CURLRES_SYNCH and USE_CURL_ASYNC may be defined at the same time.

Closes https://github.com/curl/curl/pull/6603

show more ...

KNOWN_BUGS: cannot enable LDAPS on Windows with cmake

Reported-by: Jack Boos Yu
Closes #6284

KNOWN_BUGS: Excessive HTTP/2 packets with TCP_NODELAY

Reported-by: Alex Xu
Closes #6363

http: use credentials from transfer, not connection

HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credenti

http: use credentials from transfer, not connection

HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credentials from
the most recent transfer and since HTTP auth is typically done first
thing, this has not been an issue. It was still wrong and subject to
possible race conditions or future breakage if the sequence of functions
would change.

The data.set.str[] strings MUST remain unmodified exactly as set by the
user, and the credentials to use internally are instead set/updated in
state.aptr.*

Added test 675 to verify different credentials used in two requests done
over a reused HTTP connection, which previously behaved wrongly.

Fixes #6542
Closes #6545

show more ...