urlapi: remove superfluous host name check

... as it is checked later more proper.

Closes #11195

Revert "urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects"

This reverts commit df6c2f7b544f1f35f2a3e0be11f345affeb6fe9c.
(It only keep the test case that checks redi

Revert "urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects"

This reverts commit df6c2f7b544f1f35f2a3e0be11f345affeb6fe9c.
(It only keep the test case that checks redirection to an absolute URL
without hostname and CURLU_NO_AUTHORITY).

I originally wanted to make CURLU_ALLOW_SPACE accept spaces in the
hostname only because I thought
curl_url_set(CURLUPART_URL, CURLU_ALLOW_SPACE) was already accepting
them, and they were only not being accepted in the hostname when
curl_url_set(CURLUPART_URL) was used for a redirection.

That is not actually the case, urlapi never accepted hostnames with
spaces, and a hostname with a space in it never makes sense.
I probably misread the output of my original test when I they were
normally accepted when using CURLU_ALLOW_SPACE, and not redirecting.

Some other URL parsers seems to allow space in the host part of the URL,
e.g. both python3's urllib.parse module, and Chromium's javascript URL
object allow spaces (chromium percent escapes the spaces with %20),
(they also both ignore TABs, and other whitespace characters), but those
URLs with spaces in the hostname are useless, neither python3's requests
module nor Chromium's window.location can actually use them.

There is no reason to add support for URLs with spaces in the host,
since it was not a inconsistency bug; let's revert that patch before it
makes it into release. Sorry about that.

I also reverted the extra check for CURLU_NO_AUTHORITY since that does
not seem to be necessary, CURLU_NO_AUTHORITY already worked for
redirects.

Closes #11169

show more ...

urlapi: allow numerical parts in the host name

It can only be an IPv4 address if all parts are all digits and no more than
four parts, otherwise it is a host name. Even slightly wrong IP

urlapi: allow numerical parts in the host name

It can only be an IPv4 address if all parts are all digits and no more than
four parts, otherwise it is a host name. Even slightly wrong IPv4 will now be
passed through as a host name.

Regression from 17a15d88467 shipped in 8.1.0

Extended test 1560 accordingly.

Reported-by: Pavel Kalyugin
Fixes #11129
Closes #11131

show more ...

urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects

curl_url_set(uh, CURLUPART_URL, redirurl, flags) was not respecing
CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY in the

urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects

curl_url_set(uh, CURLUPART_URL, redirurl, flags) was not respecing
CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY in the host part of redirurl
when redirecting to an absolute URL.

Closes #11136

show more ...

urlapi: cleanups

- move host checks together
- simplify the scheme parser loop and the end of host name parser
- avoid itermediate buffer storing in multiple places
- reduce scop

urlapi: cleanups

- move host checks together
- simplify the scheme parser loop and the end of host name parser
- avoid itermediate buffer storing in multiple places
- reduce scope for several variables
- skip the Curl_dyn_tail() call for speed
- detect IPv6 earlier and skip extra checks for such hosts
- normalize directly in dynbuf instead of itermediate buffer
- split out the IPv6 parser into its own funciton
- call the IPv6 parser directly for ipv6 addresses
- remove (unused) special treatment of % in host names
- junkscan() once in the beginning instead of scattered
- make junkscan return error code
- remove unused query management from dedotdotify()
- make Curl_parse_login_details use memchr
- more use of memchr() instead of strchr() and less strlen() calls
- make junkscan check and return the URL length

An optimized build runs one of my benchmark URL parsing programs ~41%
faster using this branch. (compared against the shipped 7.88.1 library
in Debian)

Closes #10935

show more ...

lib1560: verify that more bad host names are rejected

when setting the hostname component of a URL

Closes #10922

urlapi: prevent setting invalid schemes with *url_set()

A typical mistake would be to try to set "https://" - including the
separator - this is now rejected as that would then lead to

urlapi: prevent setting invalid schemes with *url_set()

A typical mistake would be to try to set "https://" - including the
separator - this is now rejected as that would then lead to
url_get(... URL...) would get an invalid URL extracted.

Extended test 1560 to verify.

Closes #10911

show more ...

urlapi: detect and error on illegal IPv4 addresses

Using bad numbers in an IPv4 numerical address now returns
CURLUE_BAD_HOSTNAME.

I noticed while working on trurl and it was or

urlapi: detect and error on illegal IPv4 addresses

Using bad numbers in an IPv4 numerical address now returns
CURLUE_BAD_HOSTNAME.

I noticed while working on trurl and it was originally reported here:
https://github.com/curl/trurl/issues/78

Updated test 1560 accordingly.

Closes #10894

show more ...

urlapi: URL encoding for the URL missed the fragment

Meaning that it would wrongly still store the fragment using spaces
instead of %20 if allowing space while also asking for URL encodi

urlapi: URL encoding for the URL missed the fragment

Meaning that it would wrongly still store the fragment using spaces
instead of %20 if allowing space while also asking for URL encoding.

Discovered when playing with trurl.

Added test to lib1560 to verify the fix.

Closes #10887

show more ...

urlapi: '%' is illegal in host names

Update test 1560 to verify

Ref: #10708
Closes #10711

lib1560: fix enumerated type mixed with another type

Follow-up to c84c0f9aa3bb006

Closes #10684

lib1560: test parsing URLs with ridiculously large fields

In the order of 120K.

Closes #10665

lib1560: add a test using %25 in the userinfo in a URL

Closes #10578

lib1560: add IPv6 canonicalization tests

Closes #10552

urlapi: do the port number extraction without using sscanf()

- sscanf() is rather complex and slow, strchr() much simpler

- the port number function does not need to fully verify th

urlapi: do the port number extraction without using sscanf()

- sscanf() is rather complex and slow, strchr() much simpler

- the port number function does not need to fully verify the IPv6 address
anyway as it is done later in the hostname_check() function and doing
it twice is unnecessary.

Closes #10541

show more ...

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- save

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205

show more ...

urlapi: add CURLU_PUNYCODE

Allows curl_url_get() get the punycode version of host names for the
host name and URL parts.

Extend test 1560 to verify.

Closes #10109

lib1560: add some basic IDN host name tests

Closes #10094

urlapi: reject more bad letters from the host name: &+()

Follow-up from eb0167ff7d31d3a5

Extend test 1560 to verify

Closes #10096

urlapi: fix parsing URL without slash with CURLU_URLENCODE

When CURLU_URLENCODE is set, the parser would mistreat the path
component if the URL was specified without a slash like in

urlapi: fix parsing URL without slash with CURLU_URLENCODE

When CURLU_URLENCODE is set, the parser would mistreat the path
component if the URL was specified without a slash like in
http://local.test:80?-123

Extended test 1560 to reproduce and verify the fix.

Reported-by: Trail of Bits

Closes #9763

show more ...

urlapi: reject more bad characters from the host name field

Extended test 1560 to verify

Report from the ongoing source code audit by Trail of Bits.

Closes #9608

url: a zero-length userinfo part in the URL is still a (blank) user

Adjusted test 1560 to verify

Reported-by: Jay Satiro

Fixes #9088
Closes #9590

lib1560: extended to verify detect/reject of unknown schemes

... when no guessing is allowed.

libtest/lib1560: test basic websocket URL parsing

tests: several enumerated type cleanups

To please icc

Closes #9179