file: remove useless assignment

This code assigned the variable the same value it already had.

Spotted by CodeSonar

Closes #13425

file: add support for getting basic directory listings

Not supported on Windows (yet)

Closes #13137

lib: add Curl_xfer_write_resp_hd

Add method in protocol handlers to allow writing of a single,
0-terminated header line. Avoids parsing and copying these lines.

Closes #13165

TIMER_STARTTRANSFER: set the same for everyone

- set TIMER_STARTTRANSFER on seeing the first response bytes
in the download client writer, not coming from a CONNECT
- initialized t

TIMER_STARTTRANSFER: set the same for everyone

- set TIMER_STARTTRANSFER on seeing the first response bytes
in the download client writer, not coming from a CONNECT
- initialized the timer the same way for all protocols
- remove explicit setting of TIMER_STARTTRANSFER in file.c
and c-hyper.c

Closes #13052

show more ...

lib: further send/upload handling polish

- Move all the "upload_done" handling to request.c

- add possibility to abort sending of a request
- add `Curl_req_done_sending()` f

lib: further send/upload handling polish

- Move all the "upload_done" handling to request.c

- add possibility to abort sending of a request
- add `Curl_req_done_sending()` for checks
- transfer.c: readwrite_upload() now clean

- removing data->state.ulbuf and data->req.upload_fromhere

- as well as data->req.upload_present
- set data->req.upload_done on having read all from
the client and completely flushed the send buffer

- tftp, remove setting of data->req.upload_fromhere

- serves no purpose as `upload_present` is not set
and the data itself is directly `sendto()` anyway

- smtp, make upload EOB conversion a client reader
- xfer_ulbuf addition

- add xfer_ulbuf for borrowing, similar to xfer_buf
- use in file upload
- use in c-hyper body sending

- h1-proxy, remove init of data->state.uilbuf that is never used
- smb, add own send_buf instead of using data->state.ulbuf

Closes #13010

show more ...

lib: Curl_read/Curl_write clarifications

- replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to
clarify when and at what level they operate
- send/recv of transfer related

lib: Curl_read/Curl_write clarifications

- replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to
clarify when and at what level they operate
- send/recv of transfer related data is now done via
`Curl_xfer_send()/Curl_xfer_recv()` which no longer has
socket/socketindex as parameter. It decides on the transfer
setup of `conn->sockfd` and `conn->writesockfd` on which
connection filter chain to operate.
- send/recv on a specific connection filter chain is done via
`Curl_conn_send()/Curl_conn_recv()` which get the socket index
as parameter.
- rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for
naming consistency
- clarify that the special CURLE_AGAIN hangling to return
`CURLE_OK` with length 0 only applies to `Curl_xfer_send()`
and CURLE_AGAIN is returned by all other send() variants.
- fix a bug in websocket `curl_ws_recv()` that mixed up data
when it arrived in more than a single chunk (to be made
into a sperate PR, also)

Added as documented [in
CLIENT-READER.md](https://github.com/curl/curl/blob/5b1f31dfbab8aef467c419c68aa06dc738cb75d4/docs/CLIENT-READERS.md).

- old `Curl_buffer_send()` completely replaced by new `Curl_req_send()`
- old `Curl_fillreadbuffer()` replaced with `Curl_client_read()`
- HTTP chunked uploads are now formatted in a client reader added when
needed.
- FTP line-end conversions are done in a client reader added when
needed.
- when sending requests headers, remaining buffer space is filled with
body data for sending in "one go". This is independent of the request
body size. Resolves #12938 as now small and large requests have the
same code path.

Changes done to test cases:

- test513: now fails before sending request headers as this initial
"client read" triggers the setup fault. Behaves now the same as in
hyper build
- test547, test555, test1620: fix the length check in the lib code to
only fail for reads *smaller* than expected. This was a bug in the
test code that never triggered in the old implementation.

Closes #12969

show more ...

file: use xfer buf for file:// transfers

- For file:// transfers use the multi handle's transfer buffer for
up- and downloads.

Prior to this change a6c9a33 (precedes 8.6.0) ch

file: use xfer buf for file:// transfers

- For file:// transfers use the multi handle's transfer buffer for
up- and downloads.

Prior to this change a6c9a33 (precedes 8.6.0) changed the file://
transfers to use a smaller stack based buffer, and that caused a
significant performance decrease in Windows.

Bug: https://github.com/curl/curl/issues/12750#issuecomment-1920103086
Reported-by: edmcln@users.noreply.github.com

Closes https://github.com/curl/curl/pull/12932

show more ...

file+ftp: use stack buffers instead of data->state.buffer

Closes #12789

lib: replace readwrite with write_resp

This clarifies the handling of server responses by folding the code for
the complicated protocols into their protocol handlers. This concerns
m

lib: replace readwrite with write_resp

This clarifies the handling of server responses by folding the code for
the complicated protocols into their protocol handlers. This concerns
mainly HTTP and its bastard sibling RTSP.

The terms "read" and "write" are often used without clear context if
they refer to the connect or the client/application side of a
transfer. This PR uses "read/write" for operations on the client side
and "send/receive" for the connection, e.g. server side. If this is
considered useful, we can revisit renaming of further methods in another
PR.

Curl's protocol handler `readwrite()` method been changed:

```diff
- CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn,
- const char *buf, size_t blen,
- size_t *pconsumed, bool *readmore);
+ CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t blen,
+ bool is_eos, bool *done);
```

The name was changed to clarify that this writes reponse data to the
client side. The parameter changes are:

* `conn` removed as it always operates on `data->conn`
* `pconsumed` removed as the method needs to handle all data on success
* `readmore` removed as no longer necessary
* `is_eos` as indicator that this is the last call for the transfer
response (end-of-stream).
* `done` TRUE on return iff the transfer response is to be treated as
finished

This change affects many files only because of updated comments in
handlers that provide no implementation. The real change is that the
HTTP protocol handlers now provide an implementation.

The HTTP protocol handlers `write_resp()` implementation will get passed
**all** raw data of a server response for the transfer. The HTTP/1.x
formatted status and headers, as well as the undecoded response
body. `Curl_http_write_resp_hds()` is used internally to parse the
response headers and pass them on. This method is public as the RTSP
protocol handler also uses it.

HTTP/1.1 "chunked" transport encoding is now part of the general
*content encoding* writer stack, just like other encodings. A new flag
`CLIENTWRITE_EOS` was added for the last client write. This allows
writers to verify that they are in a valid end state. The chunked
decoder will check if it indeed has seen the last chunk.

The general response handling in `transfer.c:466` happens in function
`readwrite_data()`. This mainly operates now like:

```
static CURLcode readwrite_data(data, ...)
{
do {
Curl_xfer_recv_resp(data, buf)
...
Curl_xfer_write_resp(data, buf)
...
} while(interested);
...
}
```

All the response data handling is implemented in
`Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()`
implementation if available, or does the default behaviour.

All raw response data needs to pass through this function. Which also
means that anyone in possession of such data may call
`Curl_xfer_write_resp()`.

Closes #12480

show more ...

multi: remove total timer reset in file_do() while fetching file://

The total timer is properly reset in MSTATE_INIT. MSTATE_CONNECT starts
with resetting the timer that is a start poin

multi: remove total timer reset in file_do() while fetching file://

The total timer is properly reset in MSTATE_INIT. MSTATE_CONNECT starts
with resetting the timer that is a start point for further multi states.
If file://, MSTATE_DO calls file_do() that should not reset the total
timer. Otherwise, the total time is always less than the pre-transfer
and the start transfer times.

Closes #12682

show more ...

windows: use built-in `_WIN32` macro to detect Windows

Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. T

windows: use built-in `_WIN32` macro to detect Windows

Windows compilers define `_WIN32` automatically. Windows SDK headers
or build env defines `WIN32`, or we have to take care of it. The
agreement seems to be that `_WIN32` is the preferred practice here.
Make the source code rely on that to detect we're building for Windows.

Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for
Windows detection, next to the official `_WIN32`. After this patch it
only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`.

There is a slight chance these break compatibility with Windows
compilers that fail to define `_WIN32`. I'm not aware of any obsolete
or modern compiler affected, but in case there is one, one possible
solution is to define this macro manually.

grepping for `WIN32` remains useful to discover Windows-specific code.

Also:

- extend `checksrc` to ensure we're not using `WIN32` anymore.

- apply minor formatting here and there.

- delete unnecessary checks for `!MSDOS` when `_WIN32` is present.

Co-authored-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Closes #12376

show more ...

lib: client writer, part 2, accounting + logging

This PR has these changes:

Renaming of unencode_* to cwriter, e.g. client writers
- documentation of sendf.h functions
- mov

lib: client writer, part 2, accounting + logging

This PR has these changes:

Renaming of unencode_* to cwriter, e.g. client writers
- documentation of sendf.h functions
- move max decode stack checks back to content_encoding.c
- define writer phase which was used as order before
- introduce phases for monitoring inbetween decode phases
- offering default implementations for init/write/close

Add type paramter to client writer's do_write()
- always pass all writes through the writer stack
- writers who only care about BODY data will pass other writes unchanged

add RAW and PROTOCOL client writers
- RAW used for Curl_debug() logging of CURLINFO_DATA_IN
- PROTOCOL used for updates to data->req.bytecount, max_filesize checks and
Curl_pgrsSetDownloadCounter()
- remove all updates of data->req.bytecount and calls to
Curl_pgrsSetDownloadCounter() and Curl_debug() from other code
- adjust test457 expected output to no longer see the excess write

Closes #12184

show more ...

lib: let the max filesize option stop too big transfers too

Previously it would only stop them from getting started if the size is
known to be too big then.

Update the libcurl a

lib: let the max filesize option stop too big transfers too

Previously it would only stop them from getting started if the size is
known to be too big then.

Update the libcurl and curl docs accordingly.

Fixes #11810
Reported-by: Elliot Killick
Assisted-by: Jay Satiro
Closes #11820

show more ...

lib: unify the upload/method handling

By making sure we set state.upload based on the set.method value and not
independently as set.upload, we reduce confusion and mixup risks, both

lib: unify the upload/method handling

By making sure we set state.upload based on the set.method value and not
independently as set.upload, we reduce confusion and mixup risks, both
internally and externally.

Closes #11017

show more ...

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- save

copyright: update all copyright lines and remove year ranges

- they are mostly pointless in all major jurisdictions
- many big corporations and projects already don't use them
- saves us from pointless churn
- git keeps history for us
- the year range is kept in COPYING

checksrc is updated to allow non-year using copyright statements

Closes #10205

show more ...

lib: remove bad set.opt_no_body assignments

This struct field MUST remain what the application set it to, so that
handle reuse and handle duplication work.

Instead, the request

lib: remove bad set.opt_no_body assignments

This struct field MUST remain what the application set it to, so that
handle reuse and handle duplication work.

Instead, the request state bit 'no_body' is introduced for code flows
that need to change this in run-time.

Closes #9888

show more ...

lib: connection filters (cfilter) addition to curl:

- general construct/destroy in connectdata
- default implementations of callback functions
- connect: cfilters for connect and

lib: connection filters (cfilter) addition to curl:

- general construct/destroy in connectdata
- default implementations of callback functions
- connect: cfilters for connect and accept
- socks: cfilter for socks proxying
- http_proxy: cfilter for http proxy tunneling
- vtls: cfilters for primary and proxy ssl
- change in general handling of data/conn
- Curl_cfilter_setup() sets up filter chain based on data settings,
if none are installed by the protocol handler setup
- Curl_cfilter_connect() boot straps filters into `connected` status,
used by handlers and multi to reach further stages
- Curl_cfilter_is_connected() to check if a conn is connected,
e.g. all filters have done their work
- Curl_cfilter_get_select_socks() gets the sockets and READ/WRITE
indicators for multi select to work
- Curl_cfilter_data_pending() asks filters if the have incoming
data pending for recv
- Curl_cfilter_recv()/Curl_cfilter_send are the general callbacks
installed in conn->recv/conn->send for io handling
- Curl_cfilter_attach_data()/Curl_cfilter_detach_data() inform filters
and addition/removal of a `data` from their connection
- adding vtl functions to prevent use of Curl_ssl globals directly
in other parts of the code.

Reviewed-by: Daniel Stenberg
Closes #9855

show more ...

lib: fix some type mismatches and remove unneeded typecasts

Many of these castings are unneeded if we change the variables to work
better with each other.

Ref: https://github.co

lib: fix some type mismatches and remove unneeded typecasts

Many of these castings are unneeded if we change the variables to work
better with each other.

Ref: https://github.com/curl/curl/pull/9823

Closes https://github.com/curl/curl/pull/9835

show more ...

style: use space after comment start and before comment end

/* like this */

/*not this*/

checksrc is updated accordingly

Closes #9828

file: add handling of native AmigaOS paths

On AmigaOS 4.x, handle native absolute paths, whilst blocking relative
paths. Also allow unix style paths if feature enabled at link time.

file: add handling of native AmigaOS paths

On AmigaOS 4.x, handle native absolute paths, whilst blocking relative
paths. Also allow unix style paths if feature enabled at link time.

Inspiration-from: Michael Trebilcock

Closes #9259

show more ...

file: fix icc enumerated type mixed with another type warning

Ref: #9179
Closes #9181

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the

copyright: make repository REUSE compliant

Add licensing and copyright information for all files in this repository. This
either happens in the file itself as a comment header or in the file
`.reuse/dep5`.

This commit also adds a Github workflow to check pull requests and adapts
copyright.pl to the changes.

Closes #8869

show more ...

lib: remove support for CURL_DOES_CONVERSIONS

TPF was the only user and support for that was dropped.

Closes #8378

conn: add 'attach' to protocol handler, make libssh2 use it

The libssh2 backend has SSH session associated with the connection but
the callback context is the easy handle, so when a conn

conn: add 'attach' to protocol handler, make libssh2 use it

The libssh2 backend has SSH session associated with the connection but
the callback context is the easy handle, so when a connection gets
attached to a transfer, the protocol handler now allows for a custom
function to get used to set things up correctly.

Reported-by: Michael O'Farrell
Fixes #6898
Closes #7078

show more ...

lib: remove strlen call from Curl_client_write

At all call sites with an explicit 0 len, pass an appropriate nonzero
len.

Closes #6954