1QUIC API Overview 2================= 3 4This document sets out the objectives of the QUIC API design process, describes 5the new and changed APIs, and the design constraints motivating those API 6designs and the relevant design decisions. 7 8- [QUIC API Overview](#quic-api-overview) 9 * [Overview and Implementation Status](#overview-and-implementation-status) 10 * [Objectives](#objectives) 11 * [SSL Objects](#ssl-objects) 12 + [Structure of Documentation](#structure-of-documentation) 13 + [Existing APIs](#existing-apis) 14 - [`SSL_set_connect_state`](#-ssl-set-connect-state-) 15 - [`SSL_set_accept_state`](#-ssl-set-accept-state-) 16 - [`SSL_is_server`](#-ssl-is-server-) 17 - [`SSL_connect`](#-ssl-connect-) 18 - [`SSL_accept`](#-ssl-accept-) 19 - [`SSL_do_handshake`](#-ssl-do-handshake-) 20 - [`SSL_read`, `SSL_read_ex`, `SSL_peek`, `SSL_peek_ex`](#-ssl-read----ssl-read-ex----ssl-peek----ssl-peek-ex-) 21 - [`SSL_write`, `SSL_write_ex`](#-ssl-write----ssl-write-ex-) 22 - [`SSL_pending`](#-ssl-pending-) 23 - [`SSL_has_pending`](#-ssl-has-pending-) 24 - [`SSL_shutdown`](#-ssl-shutdown-) 25 - [`SSL_clear`](#-ssl-clear-) 26 - [`SSL_free`](#-ssl-free-) 27 - [`SSL_set0_rbio`, `SSL_set0_wbio`, `SSL_set_bio`](#-ssl-set0-rbio----ssl-set0-wbio----ssl-set-bio-) 28 - [`SSL_set_[rw]fd`](#-ssl-set--rw-fd-) 29 - [`SSL_get_[rw]fd`](#-ssl-get--rw-fd-) 30 - [`SSL_CTRL_MODE`, `SSL_CTRL_CLEAR_MODE`](#-ssl-ctrl-mode----ssl-ctrl-clear-mode-) 31 - [SSL Modes](#ssl-modes) 32 + [New APIs for Single-Stream Operation](#new-apis-for-single-stream-operation) 33 - [`SSL_handle_events`](#-ssl-handle-events-) 34 - [`SSL_get_event_timeout`](#-ssl-get-event-timeout-) 35 - [`SSL_set_blocking_mode`, `SSL_get_blocking_mode`](#-ssl-set-blocking-mode----ssl-get-blocking-mode-) 36 - [`SSL_get_rpoll_descriptor`, `SSL_get_wpoll_descriptor`](#-ssl-get-rpoll-descriptor----ssl-get-wpoll-descriptor-) 37 - [`SSL_net_read_desired`, `SSL_net_write_desired`](#-ssl-want-net-read----ssl-want-net-write-) 38 - [`SSL_want`, `SSL_want_read`, `SSL_want_write`](#-ssl-want----ssl-want-read----ssl-want-write-) 39 - [`SSL_set1_initial_peer_addr`](#-ssl-set-initial-peer-addr-) 40 - [`SSL_shutdown_ex`](#-ssl-shutdown-ex-) 41 - [`SSL_stream_conclude`](#-ssl-stream-conclude-) 42 - [`SSL_stream_reset`](#-ssl-stream-reset-) 43 - [`SSL_get_stream_state`](#-ssl-get-stream-state-) 44 - [`SSL_get_stream_read_error_code`, `SSL_get_stream_write_error_code`](#-ssl-get-stream-read-error-code----ssl-get-stream-write-error-code-) 45 - [`SSL_get_conn_close_info`](#-ssl-get-conn-close-info-) 46 + [New APIs for Multi-Stream Operation](#new-apis-for-multi-stream-operation) 47 - [Notes on Multi-Threaded Operation](#notes-on-multi-threaded-operation) 48 - [Notes on Blocking](#notes-on-blocking) 49 - [Notes on Application-Level Polling](#notes-on-application-level-polling) 50 - [`SSL_get0_connection`](#-ssl-get0-connection-) 51 - [`SSL_is_connection`](#-ssl-is-connection-) 52 - [`SSL_get_stream_type`](#-ssl-get-stream-type-) 53 - [`SSL_get_stream_id`](#-ssl-get-stream-id-) 54 - [`SSL_is_stream_local`](#-ssl-is-stream-local-) 55 - [`SSL_new_stream`](#-ssl-new-stream-) 56 - [`SSL_accept_stream`](#-ssl-accept-stream-) 57 - [`SSL_get_accept_stream_queue_len`](#-ssl-get-accept-stream-queue-len-) 58 - [`SSL_set_incoming_stream_policy`](#-ssl-set-incoming-stream-policy-) 59 - [`SSL_set_default_stream_mode`](#-ssl-set-default-stream-mode-) 60 + [Future APIs](#future-apis) 61 * [BIO Objects](#bio-objects) 62 + [Existing APIs](#existing-apis-1) 63 - [`BIO_s_connect`, `BIO_new_ssl_connect`, `BIO_set_conn_hostname`](#-bio-s-connect----bio-new-ssl-connect----bio-set-conn-hostname-) 64 - [`BIO_new_bio_pair`](#-bio-new-bio-pair-) 65 - [Interactions with `BIO_f_buffer`](#interactions-with--bio-f-buffer-) 66 - [MTU Signalling](#mtu-signalling) 67 + [New APIs](#new-apis) 68 - [`BIO_sendmmsg` and `BIO_recvmmsg`](#-bio-sendmmsg--and--bio-recvmmsg-) 69 - [Truncation Mode](#truncation-mode) 70 - [Capability Negotiation](#capability-negotiation) 71 - [Local Address Support](#local-address-support) 72 - [`BIO_s_dgram_pair`](#-bio-s-dgram-pair-) 73 - [`BIO_POLL_DESCRIPTOR`](#-bio-poll-descriptor-) 74 - [`BIO_s_dgram_mem`](#-bio-s-dgram-mem-) 75 - [`BIO_err_is_non_fatal`](#-bio-err-is-non-fatal-) 76 * [Q & A](#q---a) 77 78Overview and Implementation Status 79---------------------------------- 80 81A listing of all SSL object APIs and their implications for QUIC, including 82current implementation status, can be found in 83[quic-api-ssl-funcs.md](./quic-api-ssl-funcs.md). 84 85Non-SSL object APIs which are new or changed, or otherwise discussed in this 86document are listed below, along with their implementation status. SSL object 87APIs are not listed here; see [quic-api-ssl-funcs.md](./quic-api-ssl-funcs.md) 88for details on SSL object APIs. 89 90| Semantics | API | Status | 91|-----------|---------------------------------|--------| 92| Changed | `BIO_s_connect` | Done | 93| Unchanged | `BIO_set_conn_hostname` | Done | 94| N/A | `BIO_new_bio_pair` | N/A (see `BIO_new_bio_dgram_pair`) | 95| New | `BIO_s_dgram_pair` | Done | 96| Unchanged | `BIO_dgram_get_mtu` | Done | 97| Unchanged | `BIO_dgram_set_mtu` | Done | 98| New | `BIO_sendmmsg` | Done | 99| New | `BIO_recvmmsg` | Done | 100| New | `BIO_dgram_set_no_trunc` | Done | 101| New | `BIO_dgram_get_no_trunc` | Done | 102| New | `BIO_dgram_set_caps` | Done | 103| New | `BIO_dgram_get_caps` | Done | 104| New | `BIO_dgram_get_effective_caps` | Done | 105| New | `BIO_dgram_get_local_addr_cap` | Done | 106| New | `BIO_dgram_set_local_addr_enable` | Done | 107| New | `BIO_dgram_get_local_addr_enable` | Done | 108| New | `BIO_get_rpoll_descriptor` | Done | 109| New | `BIO_get_wpoll_descriptor` | Done | 110| New | `BIO_err_is_non_fatal` | Done | 111 112Objectives 113---------- 114 115The objectives of the QUIC API design are: 116 117- to provide an API suitable for use with QUIC, now and in the future; 118 119- to reuse the existing libssl APIs to the extent feasible; 120 121- to enable existing applications to adapt to using QUIC with only 122 minimal API changes. 123 124SSL Objects 125----------- 126 127### Structure of Documentation 128 129Each API listed below has an information table with the following fields: 130 131- **Semantics**: This can be one of: 132 133 - **Unchanged**: The semantics of this existing libssl API call are 134 unchanged. 135 - **Changed**: The semantics are changed for QUIC. 136 - **New**: The API is new for QUIC. 137 138- `SSL_get_error`: Can this API, when used with QUIC, change the 139 state returned by `SSL_get_error`? This can be any combination of: 140 141 - **Never**: Does not interact with `SSL_get_error`. 142 - **Error**: Non-`WANT_READ`/`WANT_WRITE` errors can be raised. 143 - **Want**: `WANT_READ`/`WANT_WRITE` can be raised. 144 145- **Can Tick?**: Whether this function is allowed to perform event processing 146 for the QUIC state machine and potentially perform network I/O. 147 148- **CSHL:** Connection/Stream/Handshake Layer classification. 149 This can be one of: 150 151 - **HL:** This is a handshake layer related call. It should be supported 152 on a QUIC connection SSL object, forwarding to the handshake layer 153 SSL object. QUIC stream SSL objects do not allow these calls to be 154 forwarded. 155 156 - **HL-Forbidden:** This is a handshake layer related call, but it is 157 inapplicable to QUIC, so it is not supported. 158 159 - **C:** Not handshake-layer related. QUIC connection SSL object usage only. 160 Fails on a QUIC stream SSL object. 161 162 - **CS:** Not handshake-layer related. Can be used on any QUIC SSL object. 163 164 - **S**: Requires a QUIC stream SSL object or a QUIC connection SSL object 165 with a default stream attached. 166 167### Existing APIs 168 169#### `SSL_set_connect_state` 170 171| Semantics | `SSL_get_error` | Can Tick? | CSHL | 172| --------- | ------------- | --------- | ------------- | 173| Unchanged | Never | No | HL | 174 175#### `SSL_set_accept_state` 176 177| Semantics | `SSL_get_error` | Can Tick? | CSHL | 178| --------- | ------------- | --------- | ------------- | 179| Unchanged | Never | No | HL | 180 181**Note:** Attempting to proceed in this state will not function for now because 182we do not implement server support at this time. However, the semantics of this 183function as such are unchanged. 184 185#### `SSL_is_server` 186 187| Semantics | `SSL_get_error` | Can Tick? | CSHL | 188| --------- | ------------- | --------- | ------------- | 189| Unchanged | Never | No | HL | 190 191#### `SSL_connect` 192 193| Semantics | `SSL_get_error` | Can Tick? | CSHL | 194| --------- | ------------- | --------- | ------------- | 195| Unchanged | Error/Want | Yes | HL | 196 197Simple composition of `SSL_set_connect_state` and `SSL_do_handshake`. 198 199#### `SSL_accept` 200 201| Semantics | `SSL_get_error` | Can Tick? | CSHL | 202| --------- | ------------- | --------- | ------------- | 203| Unchanged | Error/Want | Yes | HL | 204 205Simple composition of `SSL_set_accept_state` and `SSL_do_handshake`. 206 207#### `SSL_do_handshake` 208 209| Semantics | `SSL_get_error` | Can Tick? | CSHL | 210| --------- | ------------- | --------- | ------------- | 211| Unchanged | Error/Want | Yes | HL | 212 213**Note:** Idempotent if handshake already completed. 214 215**Blocking Considerations:** Blocks until handshake completed if in blocking 216mode. 217 218#### `SSL_read`, `SSL_read_ex`, `SSL_peek`, `SSL_peek_ex` 219 220| Semantics | `SSL_get_error` | Can Tick? | CSHL | 221| --------- | ------------- | --------- | ------------- | 222| Unchanged | Error/Want | Yes | CS | 223 224**Blocking Considerations:** Blocks until at least one byte is available or an 225error occurs if in blocking mode (including the peek functions). 226 227If the read part of the stream has been finished by the peer, calls to 228`SSL_read` will fail with `SSL_ERROR_ZERO_RETURN`. 229 230If a stream has terminated in a non-normal fashion (for example because the 231stream has been reset, or the connection has terminated), calls to `SSL_read` 232will fail with `SSL_ERROR_SSL`. 233 234`SSL_get_stream_read_state` can be used to clarify the stream state when an 235error occurs. 236 237#### `SSL_write`, `SSL_write_ex` 238 239| Semantics | `SSL_get_error` | Can Tick? | CSHL | 240| --------- | ------------- | --------- | ------------- | 241| Unchanged | Error/Want | Yes | CS | 242 243We have to implement all of the following modes: 244 245- `SSL_MODE_ENABLE_PARTIAL_WRITE` on or off 246- `SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER` on or off 247- Blocking mode on or off 248 249**Blocking Considerations:** Blocks until libssl has accepted responsibility for 250(i.e., copied) all data provided, or an error occurs, if in blocking mode. In 251other words, it blocks until it can buffer the data. This does not necessarily 252mean that the data has actually been sent. 253 254`SSL_get_stream_write_state` can be used to clarify the stream state when an 255error occurs. 256 257#### `SSL_pending` 258 259| Semantics | `SSL_get_error` | Can Tick? | CSHL | 260| --------- | ------------- | --------- | ------------- | 261| Unchanged | Never | No | CS | 262 263#### `SSL_has_pending` 264 265| Semantics | `SSL_get_error` | Can Tick? | CSHL | 266| --------- | ------------- | --------- | ------------- | 267| Unchanged | Never | No | CS | 268 269**TBD.** Options: 270 271 - Semantics unchanged or approximated (essentially, `SSL_pending() || any RXE 272 queued || any URXE queued`). 273 - Change semantics to only determine the return value based on if there is 274 data in the stream receive buffer. 275 276#### `SSL_shutdown` 277 278| Semantics | `SSL_get_error` | Can Tick? | CSHL | 279| --------- | ------------- | --------- | ------------- | 280| Unchanged | Error | Yes | CS | 281 282See `SSL_shutdown_ex` below for discussion of how this will work for QUIC. 283 284Calling `SSL_shutdown` is always exactly identical in function to calling 285`SSL_shutdown_ex` with `flags` set to 0 and `args` set to `NULL`. 286 287#### `SSL_clear` 288 289| Semantics | `SSL_get_error` | Can Tick? | CSHL | 290| --------- | ------------- | --------- | ------------- | 291| TBD | TBD | No | C | 292 293There are potential implementation hazards: 294 295>SSL_clear() resets the SSL object to allow for another connection. The reset 296>operation however keeps several settings of the last sessions (some of these 297>settings were made automatically during the last handshake). It only makes sense 298>for a new connection with the exact same peer that shares these settings, and 299>may fail if that peer changes its settings between connections. 300 301**TBD:** How should `SSL_clear` be implemented? Either: 302 303 - Modernised implementation which resets everything, handshake layer 304 re-instantiated (safer); 305 - Preserve `SSL_clear` semantics at the handshake layer, reset all QUIC state 306 (`QUIC_CHANNEL` torn down, CSM reset). 307 308**TBD:** Semantics of this on stream objects. 309 310#### `SSL_free` 311 312| Semantics | `SSL_get_error` | Can Tick? | CSHL | 313| --------- | ------------- | --------- | ------------- | 314| Changed | Never | No | CS | 315 316**QUIC stream SSL objects.** When used on a QUIC stream SSL object, parts of the 317stream state may continue to exist internally, managed inside the QUIC 318connection SSL object, until they can be correctly torn down, or until the QUIC 319connection SSL object is freed. 320 321If a QUIC stream SSL object is freed for a stream which has not reached a 322terminal state for all of its parts (both send and receive, as applicable), the 323stream is automatically reset (non-normal termination) with an application error 324code of 0. To explicitly reset a stream with a different application error code, 325call `SSL_stream_reset` before calling this function. 326 327If the peer continues to send data on the stream before it processes the 328notification of the stream's termination, that incoming data will be discarded. 329However, the peer will be reliably notified of the non-normal termination of the 330stream assuming that the connection remains healthy. 331 332When freeing a QUIC stream SSL object which was terminated in a non-normal 333fashion, or which was terminated automatically due to a call to this function, 334any data which was appended to the stream via `SSL_write` may or may not have 335already been transmitted, and even if already transmitted, may or may not be 336retransmitted in the event of loss. 337 338When freeing a QUIC stream SSL object which was terminated normally (for example 339via `SSL_stream_conclude`), data appended to the stream via `SSL_write` will 340still be transmitted or retransmitted as necessary, assuming that the QUIC 341connection SSL object is not freed and that the connection remains healthy. 342 343**QUIC connection SSL objects.** `SSL_free` is largely unchanged for QUIC 344connection SSL objects on the client side. When freeing a QUIC connection SSL 345object being used in client mode, there is immediate termination of any QUIC 346network I/O processing as the resources needed to handle the connection are 347immediately freed. This means that, if a QUIC connection SSL object which has 348not been shutdown properly is freed using this function: 349 350- Any data which was pending transmission or retransmission will not be 351 transmitted, including in streams which were terminated normally; 352 353- The connection closure process will not function correctly or in an 354 RFC-compliant manner. Connection closure will not be signalled to the peer 355 and the connection will simply disappear from the perspective of the peer. The 356 connection will appear to remain active until the connection's idle timeout 357 (if negotiated) takes effect. 358 359 For further discussion of this issue, see `SSL_shutdown_ex` and the Q&A. 360 361#### `SSL_set0_rbio`, `SSL_set0_wbio`, `SSL_set_bio` 362 363| Semantics | `SSL_get_error` | Can Tick? | CSHL | 364| --------- | ------------- | --------- | ------------- | 365| Changed | Never | No | C | 366 367Sets network-side BIO. 368 369The changes to the semantics of these calls are as follows: 370 371 - The BIO MUST be a BIO with datagram semantics (this is a change relative to 372 TLS, though not to DTLS). 373 374 - If the BIO is non-pollable (see below), application-level blocking mode will 375 be forced off. 376 377#### `SSL_set_[rw]fd` 378 379| Semantics | `SSL_get_error` | Can Tick? | CSHL | 380| --------- | ------------- | --------- | ------------- | 381| Changed | Never | No | C | 382 383Sets network-side socket FD. 384 385Existing behaviour: Instantiates a `BIO_s_socket`, sets an FD on it, and sets it 386as the BIO. 387 388New proposed behaviour: 389 390- Instantiate a `BIO_s_dgram` instead for a QUIC connection SSL object. 391- Fails (no-op) for a QUIC stream SSL object. 392 393#### `SSL_get_[rw]fd` 394 395| Semantics | `SSL_get_error` | Can Tick? | CSHL | 396| --------- | ------------- | --------- | ------------- | 397| Unchanged | Never | No | C | 398 399Should not require any changes. 400 401#### `SSL_CTRL_MODE`, `SSL_CTRL_CLEAR_MODE` 402 403| Semantics | `SSL_get_error` | Can Tick? | CSHL | 404| --------- | ------------- | --------- | ------------- | 405| Unchanged | Never | No | CS | 406 407#### SSL Modes 408 409- `SSL_MODE_ENABLE_PARTIAL_WRITE`: Implemented. If this mode is set during a 410 non-partial-write `SSL_write` operation spanning multiple `SSL_write` calls, 411 this mode does not take effect until the non-partial write operation is 412 completed. 413 414- `SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER`: Implemented. 415 416- `SSL_MODE_AUTO_RETRY`: TBD. 417 418- `SSL_MODE_RELEASE_BUFFERS`: Ignored. This is an optimization and if it has 419 any sensible semantic correspondence to QUIC, this can be considered later. 420 421- `SSL_MODE_SEND_FALLBACK_SCSV`: TBD: Either ignore or fail if the client 422 attempts to set this prior to handshake. The latter is probably safer. 423 424 Ignored if set after handshake (existing behaviour). 425 426- `SSL_MODE_ASYNC`: TBD. 427 428### New APIs for Single-Stream Operation 429 430#### `SSL_handle_events` 431 432| Semantics | `SSL_get_error` | Can Tick? | CSHL | 433| --------- | ------------- | --------- | ------------- | 434| New | Never | Yes | CS | 435 436Advances the QUIC state machine to the extent feasible, potentially performing 437network I/O. Also compatible with DTLSv1 and supersedes `DTLSv1_handle_timeout` 438for all use cases. 439 440#### `SSL_get_event_timeout` 441 442| Semantics | `SSL_get_error` | Can Tick? | CSHL | 443| --------- | ------------- | --------- | ------------- | 444| New | Never | No | CS | 445 446Gets the time until the QUIC state machine next wants to receive a timeout 447event, if any. 448 449This is similar to the existing `DTLSv1_get_timeout` function, but it is not 450specific to DTLSv1. It is also usable for DTLSv1 and can become a 451protocol-agnostic API for this purpose, superseding `DTLSv1_get_timeout` for all 452use cases. 453 454The design is similar to that of `DTLSv1_get_timeout` and uses a `struct 455timeval`. However, this function can also output an infinite timeout using the 456`is_infinite` argument, whereas whereas `DTLSv1_get_timeout` represents an 457infinite timeout using a 0 return value, which does not allow a failure 458condition to be distinguished. 459 460#### `SSL_set_blocking_mode`, `SSL_get_blocking_mode` 461 462| Semantics | `SSL_get_error` | Can Tick? | CSHL | 463| --------- | ------------- | --------- | ------------- | 464| New | Never | No | CS | 465 466Turns blocking mode on or off. This is necessary because up until now libssl has 467operated in blocking or non-blocking mode automatically as an emergent 468consequence of whether the underlying network socket is blocking. For QUIC, this 469is no longer viable, thus blocking semantics at the application level must be 470explicitly configured. 471 472Use on stream objects: It may be feasible to implement this such that different 473QUIC stream SSL objects can have different settings for this option. 474 475Not supported for non-QUIC SSL objects. 476 477#### `SSL_get_rpoll_descriptor`, `SSL_get_wpoll_descriptor` 478 479| Semantics | `SSL_get_error` | Can Tick? | CSHL | 480| --------- | ------------- | --------- | ------------- | 481| New | Never | No | CS | 482 483These functions output poll descriptors which can be used to determine when the 484QUIC state machine next needs to have events handled. `SSL_get_rpoll_descriptor` 485is relevant if `SSL_net_read_desired` returns 1, and `SSL_get_wpoll_descriptor` 486is relevant if `SSL_net_write_desired` returns 1. 487 488The implementation of these functions is a simple forward to 489`BIO_get_rpoll_descriptor` and `BIO_get_wpoll_descriptor` on the underlying 490network BIOs. 491 492TODO: Support these for non-QUIC SSL objects 493 494#### `SSL_net_read_desired`, `SSL_net_write_desired` 495 496| Semantics | `SSL_get_error` | Can Tick? | CSHL | 497| --------- | ------------- | --------- | ------------- | 498| New | Never | No | CS | 499 500These calls return 1 if the QUIC state machine is interested in receiving 501further data from the network, or writing to the network, respectively. The 502return values of these calls should be used to determine which wakeup events 503should cause an application to call `SSL_handle_events`. These functions do not 504mutate any state, and their return values may change after a call to any SSL 505function other than `SSL_net_read_desired`, `SSL_net_write_desired`, 506`SSL_get_rpoll_descriptor`, `SSL_get_wpoll_descriptor` and 507`SSL_get_event_timeout`. 508 509TODO: Support these for non-QUIC SSL objects, turning this into a unified 510replacement for `SSL_want` 511 512#### `SSL_want`, `SSL_want_read`, `SSL_want_write` 513 514The existing API `SSL_want`, and the macros defined in terms of it, are 515traditionally used to determine if the SSL state machine has exited in 516non-blocking mode due to a desire to read from or write to the underlying 517network BIO. However, this API is unsuitable for use with QUIC because the 518return value of `SSL_want` can only express one I/O direction at a time (read or 519write), not both. This call will not be implemented for QUIC (e.g. always 520returns `SSL_NOTHING`) and `SSL_net_read_desired` and `SSL_net_write_desired` 521will be used instead. 522 523#### `SSL_set1_initial_peer_addr` 524 525| Semantics | `SSL_get_error` | Can Tick? | CSHL | 526| --------- | ------------- | --------- | ------------- | 527| New | Never | No | CS | 528 529`SSL_set1_initial_peer_addr` sets the initial L4 UDP peer address for an outgoing 530QUIC connection. 531 532The initial peer address may be autodetected if no peer address has already been 533set explicitly and the QUIC connection SSL object is provided with a 534`BIO_s_dgram` with a peer set. 535 536`SSL_set1_initial_peer_addr` cannot be called after a connection is established. 537 538#### `SSL_shutdown_ex` 539 540| Semantics | `SSL_get_error` | Can Tick? | CSHL | 541| --------- | ------------- | --------- | ------------- | 542| New | Error | Yes | C | 543 544```c 545typedef struct ssl_shutdown_ex_args_st { 546 /* These arguments pertain only to QUIC connections. */ 547 uint64_t quic_error_code; /* [0, 2**62-1] */ 548 const char *quic_reason; 549} SSL_SHUTDOWN_EX_ARGS; 550 551#define SSL_SHUTDOWN_FLAG_RAPID (1U << 0) 552#define SSL_SHUTDOWN_FLAG_IMMEDIATE (1U << 1) 553 554int SSL_shutdown_ex(SSL *ssl, 555 uint64_t flags, 556 const SSL_SHUTDOWN_EX_ARGS *args, 557 size_t args_len); 558``` 559 560`SSL_shutdown_ex` is an extended version of `SSL_shutdown`. 561 562`args` specifies arguments which control how the SSL object is shut down. `args` 563are read only on the first call to `SSL_shutdown_ex` for a given SSL object and 564subsequent calls to `SSL_shutdown_ex` ignore the `args` argument. `args_len` 565should be set to `sizeof(*args)`. This function is idempotent; once the shutdown 566process for a SSL object is complete, further calls are a no-op and return 1. 567 568Calling `SSL_shutdown_ex` on a QUIC connection SSL object causes the immediate 569close of the QUIC connection. “Immediate close” is as defined by RFC 9000. 570 571If no QUIC connection attempt was ever initiated using the given SSL object, the 572QUIC connection transitions immediately to the Terminated state. Otherwise, the 573connection closure process is initiated if it has not already begun. 574 575Any application stream data on a non-terminated or normally terminated stream 576which has yet to be transmitted is flushed to the network before the termination 577process begins. This ensures that where an application which calls `SSL_write` 578and performs a connection closure in a way which is considered normal to the 579application protocol being used, all of the data written is delivered to the 580peer. This behaviour may be skipped by setting the `SSL_SHUTDOWN_FLAG_IMMEDIATE` 581flag, in which case any data appended to streams via `SSL_write` (or any 582end-of-stream conditions) may not be transmitted to the peer. This flag may be 583useful where a non-normal application condition has occurred and the delivery of 584data written to streams via `SSL_write` is no longer relevant. Application 585stream data on streams which were terminated non-normally (for example via 586`SSL_stream_reset`) is not transmitted by this function. 587 588A QUIC connection can be shut down using this function in two different ways: 589 590- **RFC compliant mode.** In this mode, which provides the most robust 591 operation, the shutdown process may take a period of time up to three times 592 the current estimated RTT to the peer. It is possible for the closure process 593 to complete much faster in some circumstances but this cannot be relied upon. 594 595 In blocking mode, the function will return once the closure process is 596 complete. In non-blocking mode, `SSL_shutdown_ex` should be called until it 597 returns 1, indicating the closure process is complete and the connection is 598 now terminated. 599 600- **Rapid mode.** In this mode, a `CONNECTION_CLOSE` frame is sent in a 601 best-effort manner and the connection is terminated immediately. If the 602 `CONNECTION_CLOSE` frame sent is lost, the peer will not know that the 603 connection has terminated until the negotiated idle timeout (if any) expires. 604 605 This will generally return 0 on success, indicating that the connection has 606 not yet reached the Terminating state (unless it has already done so, in which 607 case it will return 1). 608 609 In blocking mode, this blocks until at least one `CONNECTION_CLOSE` frame is 610 sent but does not otherwise block. In non-blocking mode, this should be called 611 until it returns a non-negative value. A negative value indicates failure or 612 an I/O would-block condition. 613 614It is permissible for an application to implement a hybrid approach, for example 615by initiating a rapid or non-blocking shutdown and continuing to call 616`SSL_handle_events` for a duration it chooses. 617 618If `SSL_SHUTDOWN_FLAG_RAPID` is specified in `flags`, a rapid shutdown is 619performed, otherwise an RFC-compliant shutdown is performed. The principal 620effect of this flag is to partially disable blocking behaviour in blocking mode, 621and the QUIC implementation will still attempt to implement the Terminating 622state semantics if the application happens to call `SSL_handle_events`, until it 623reaches the Terminated state or is freed. An application can change its mind 624about performing a rapid shutdown by making a subsequent call to 625`SSL_shutdown_ex` without the flag set. 626 627Calling `SSL_shutdown_ex` on a QUIC stream SSL object is not valid; such a call 628will fail and has no effect. The rationale for this is that an application may 629well want to pass around SSL objects for individual QUIC streams to existing 630parts of its own code which expect something which behaves like a typical SSL 631object (i.e., a single bytestream); those components may well already call 632`SSL_shutdown` and it is not desired for such calls to affect the whole 633connection. 634 635The `args->quic_error_code` and `args->reason` fields allow the application 636error code and reason string for the closure of a QUIC connection to be 637specified. If `args` or `args->reason` is `NULL`, a zero-length string is used 638for the reason. If `args` is `NULL`, an error code of 0 is used. 639`args->quic_error_code` must be in the range `[0, 2**62-1]`, else this function 640fails. These fields are ignored for SSL objects which do not represent QUIC 641connections. 642 643#### `SSL_stream_conclude` 644 645| Semantics | `SSL_get_error` | Can Tick? | CSHL | 646| --------- | ------------- | --------- | ------------- | 647| New | Error | Yes | S | 648 649```c 650int SSL_stream_conclude(SSL *ssl, uint64_t flags); 651``` 652 653`SSL_stream_conclude` signals the normal end-of-stream condition to the send 654part of a QUIC stream. If called on a QUIC connection SSL object with a default 655stream, it signals the end of that stream to the peer. If called on a QUIC 656stream SSL object, it signals the end of that stream to the peer. 657 658This function may only be called for bidirectional streams and for outgoing 659unidirectional streams. It is a no-op if it has already been called for a given 660stream, or if either the stream or connection have entered an error state. 661 662Any data already queued for transmission via a call to `SSL_write()` will still 663be written in a reliable manner before the end-of-stream is signalled, assuming 664the connection remains healthy. This function can be thought of as appending a 665logical end-of-stream marker after any data which has previously been written to 666the stream via calls to `SSL_write`. Further attempts to call `SSL_write` after 667calling this function will fail. 668 669When calling this on a bidirectional stream, the receive part of the stream 670remains unaffected, and the peer may continue to send data via it until the peer 671also signals the end of the stream. Thus, `SSL_read()` can still be used. 672 673This function is used to conclude the send part of a stream in a normal manner. 674To perform non-normal termination of both the sending and receiving parts of a 675stream, see `SSL_stream_reset`. 676 677`flags` is reserved and should be set to 0. 678 679#### `SSL_stream_reset` 680 681| Semantics | `SSL_get_error` | Can Tick? | CSHL | 682| --------- | ------------- | --------- | ------------- | 683| New | Error | Yes | S | 684 685```c 686typedef struct ssl_stream_reset_args_st { 687 uint64_t quic_error_code; /* [0, 2**62-1] */ 688} SSL_STREAM_RESET_ARGS; 689 690int SSL_stream_reset(SSL *ssl, 691 const SSL_STREAM_RESET_ARGS *args, 692 size_t args_len); 693``` 694 695Conducts a non-normal termination of a bidirectional or outgoing unidirectional 696stream. For QUIC, this corresponds to a stream reset using a `RESET_STREAM` 697frame. 698 699It may be called on either a QUIC stream SSL object or a QUIC connection SSL 700object with a default stream; the given stream is reset. The QUIC connection is 701not affected. 702 703For bidirectional streams, this terminates both sending and receiving parts of 704the stream. It may not be called on an incoming unidirectional stream. 705 706If `args` is `NULL`, an application error code of 0 is used. Otherwise, the 707application error code to use is specified in `args->quic_error_code`, which 708must be in the range `[0, 2**62-1]`. `args_len` must be set to `sizeof(*args)` 709if `args` is non-NULL. 710 711Only the first call to this function has any effect; subsequent calls are 712no-ops. This is considered a success case. 713 714#### `SSL_get_stream_state` 715 716| Semantics | `SSL_get_error` | Can Tick? | CSHL | 717| --------- | ------------- | --------- | ------------- | 718| New | Never | No | S | 719 720```c 721/* 722 * e.g. Non-QUIC SSL object, or QUIC connection SSL object without a default 723 * stream. 724 */ 725#define SSL_STREAM_STATE_NONE 0 726 727/* 728 * The read or write part of the stream is still available and has not been 729 * terminated in a normal or non-normal manner. 730 */ 731#define SSL_STREAM_STATE_OK 1 732 733/* 734 * The stream is a unidirectional stream and this direction cannot be used; for 735 * example, a remotely initiated unidirectional stream where 736 * SSL_get_stream_write_state is called, or a locally initiated unidirectional 737 * stream where SSL_get_stream_read_state is 738 called. 739 */ 740#define SSL_STREAM_STATE_WRONG_DIR 2 741 742/* 743 * The read or write part of the stream has been finished in a normal manner. 744 * 745 * For SSL_get_stream_read_state, this means that there is no more data to read, 746 * and that any future SSL_read calls will return any residual data waiting to 747 * be read followed by a SSL_ERROR_ZERO_RETURN condition. 748 * 749 * For SSL_get_stream_write_state, this means that the local application has 750 * already indicated the end of the stream by calling SSL_stream_conclude, 751 * and that future calls to SSL_write will fail. 752 */ 753#define SSL_STREAM_STATE_FINISHED 3 754 755/* 756 * The stream was reset by the local party. 757 * 758 * For SSL_get_stream_read_state, this means that the stream was aborted using a 759 * locally transmitted STOP_SENDING frame. Attempts to read from the stream via 760 * SSL_read will fail, though SSL_read may allow any residual data waiting to 761 * be read to be read first. 762 * 763 * For SSL_get_stream_write_state, this means that the stream was aborted 764 * using a locally transmitted RESET_STREAM frame. Attempts to write to 765 * the stream will fail. 766 */ 767#define SSL_STREAM_STATE_RESET_LOCAL 4 768 769/* 770 * The stream was reset by the remote party. 771 * 772 * For SSL_get_stream_read_state, this means the peer sent a STREAM_RESET 773 * frame for the stream. 774 * 775 * For SSL_get_stream_write_state, this means the peer sent a STOP_SENDING 776 * frame for the stream. 777 */ 778#define SSL_STREAM_STATE_RESET_REMOTE 5 779 780/* 781 * The underlying connection supporting the stream has closed or otherwise 782 * failed. 783 * 784 * For SSL_get_stream_read_state, this means that attempts to read from the 785 * stream via SSL_read will fail, though SSL_read may allow any residual 786 * data waiting to be read to be read first. 787 * 788 * For SSL_get_stream_write_state, this means that attempts to write to the 789 * stream will fail. 790 */ 791#define SSL_STREAM_STATE_CONN_CLOSED 6 792 793int SSL_get_stream_read_state(SSL *ssl); 794int SSL_get_stream_write_state(SSL *ssl); 795``` 796 797This API allows the current state of a stream to be queried. This allows an 798application to determine whether a stream is still usable and why a stream has 799reached an error state. 800 801#### `SSL_get_stream_read_error_code`, `SSL_get_stream_write_error_code` 802 803| Semantics | `SSL_get_error` | Can Tick? | CSHL | 804| --------- | ------------- | --------- | ------------- | 805| New | Never | No | S | 806 807```c 808int SSL_get_stream_read_error_code(SSL *ssl, uint64_t *app_error_code); 809int SSL_get_stream_write_error_code(SSL *ssl, uint64_t *app_error_code); 810``` 811 812`SSL_get_stream_read_error_code` gets the error code for the read part of the 813stream. 814 815`SSL_get_stream_write_error_code` gets the error code for the write part of 816the stream. 817 818If a stream has been terminated normally, returns 0. 819 820If a stream has been terminated non-normally, returns 1 and writes the 821applicable application error code to `*app_error_code`. 822 823If a stream is still healthy, or was healthy at the time the connection was 824closed, or the respective part of the stream does not exist (e.g. for a 825unidirectional stream), returns -1. 826 827#### `SSL_get_conn_close_info` 828 829| Semantics | `SSL_get_error` | Can Tick? | CSHL | 830| --------- | ------------- | --------- | ------------- | 831| New | Never | No | C | 832 833```c 834#define SSL_CONN_CLOSE_FLAG_LOCAL 835#define SSL_CONN_CLOSE_FLAG_TRANSPORT 836 837typedef struct ssl_conn_close_info_st { 838 uint64_t error_code; 839 char *reason; 840 size_t reason_len; 841 uint32_t flags; 842} SSL_CONN_CLOSE_INFO; 843 844int SSL_get_conn_close_info(SSL *ssl, 845 SSL_CONN_CLOSE_INFO *info, 846 size_t info_len); 847``` 848 849If a connection is still healthy, returns 0. Otherwise, fills `*info` with 850information about the error causing connection termination and returns 1. 851`info_len` must be set to `sizeof(*info)`. Returns -1 if called on a non-QUIC 852SSL object or if the connection status cannot be determined. 853 854`info->reason` is set to point to a buffer containing a reason string. The 855buffer is valid for the lifetime of the SSL object. The reason string will 856always be zero terminated, but since it is received from a potentially untrusted 857peer, may also contain zero bytes. `info->reason_len` is the true length of the 858reason string in bytes. 859 860`info->flags` has `SSL_CONN_CLOSE_FLAG_LOCAL` set if the connection closure was 861locally initiated. 862 863`info->flags` has `SSL_CONN_CLOSE_FLAG_TRANSPORT` if the connection closure was 864initiated by QUIC, and 0 if it was initiated by the application. The namespace 865of `info->error_code` is determined by this parameter. 866 867### New APIs for Multi-Stream Operation 868 869The above new APIs are built on constructively to facilitate multi-stream 870operation. 871 872The concept of a QUIC stream SSL object is introduced. A QUIC SSL object is 873either a QUIC connection SSL object or a QUIC stream SSL object. A QUIC stream 874SSL object belongs to a QUIC connection SSL object. A QUIC connection SSL object 875may or may not have an associated default stream. There may only be at most one 876default stream for a QUIC connection SSL object. Reading or writing application 877data to a QUIC connection SSL object with a default stream is equivalent to 878reading or writing to that stream. It is an error to attempt to read or write 879application data, or perform other stream-specific operations, on a QUIC 880connection SSL object without a default stream associated. 881 882#### Notes on Multi-Threaded Operation 883 884Initially these APIs will not be thread safe over the same connection, but in 885the longer term we intend to support multiple threads using different QUIC 886stream SSL objects on different threads over the same connection without the 887application having to do any locking. This is referred to as multi-stream 888multi-thread (MSMT) operation. Only APIs explicitly denoted below will 889eventually be MSMT-safe. 890 891#### Notes on Blocking 892 893The blocking mode can be configured on each SSL object individually. When a QUIC 894stream SSL object is created it inherits its blocking state from the currently 895configured blocking state of the QUIC connection SSL object at the time the 896stream is created. This can be changed independently. For example, a QUIC 897connection SSL object can be in blocking mode to allow for blocking 898`SSL_accept_stream` calls, yet have some or all QUIC stream SSL objects be in 899non-blocking mode concurrently. 900 901#### Notes on Application-Level Polling 902 903An API may be added in the future to allow applications to poll multiple QUIC 904connection SSL objects efficiently for new stream and stream readability events. 905This is not yet urgent but will be more relevant for concurrent server 906applications. 907 908#### `SSL_get0_connection` 909 910| Semantics | `SSL_get_error` | Can Tick? | CSHL | 911| --------- | ------------- | --------- | ------------- | 912| New | Never | No | CS | 913 914```c 915/* 916 * Get the SSL object representing the connection associated with this object. 917 * 918 * If the SSL object represents a non-QUIC method or a QUIC connection, this 919 * returns the same object passed. 920 * 921 * If the SSL object represents a QUIC stream returns the QUIC connection 922 * object. 923 */ 924SSL *SSL_get0_connection(SSL *ssl); 925``` 926 927#### `SSL_is_connection` 928 929| Semantics | `SSL_get_error` | Can Tick? | CSHL | 930| --------- | ------------- | --------- | ------------- | 931| New | Never | No | CS | 932 933```c 934/* 935 * Returns 1 if the object represents a connection. This always returns 1 for 936 * non-QUIC methods, but returns 0 for SSL objects for QUIC streams which are 937 * not also the QUIC connection object. 938 * 939 * This is exactly equivalent to (SSL_get0_connection(ssl) == ssl). 940 */ 941int SSL_is_connection(SSL *ssl); 942``` 943 944#### `SSL_get_stream_type` 945 946| Semantics | `SSL_get_error` | Can Tick? | CSHL | 947| --------- | ------------- | --------- | ------------- | 948| New | Never | No | S | 949 950```c 951/* 952 * If the object represents a stream, returns a SSL_STREAM_TYPE value 953 * designating whether the stream can be used for transmission, reception, 954 * or both. 955 * 956 * This always returns SSL_STREAM_TYPE_BIDI for non-QUIC methods. 957 * 958 * It returns SSL_STREAM_TYPE_NONE for a QUIC connection object if it 959 * does not have a default stream. 960 */ 961#define SSL_STREAM_TYPE_NONE 0 962#define SSL_STREAM_TYPE_READ 1 963#define SSL_STREAM_TYPE_WRITE 2 964#define SSL_STREAM_TYPE_BIDI (SSL_STREAM_TYPE_READ | SSL_STREAM_TYPE_WRITE) 965__owur int SSL_get_stream_type(SSL *ssl); 966``` 967 968#### `SSL_get_stream_id` 969 970| Semantics | `SSL_get_error` | Can Tick? | CSHL | 971| --------- | ------------- | --------- | ------------- | 972| New | Never | No | S | 973 974```c 975/* 976 * QUIC: Returns the unique stream ID for the stream, an integer in range [0, 2**62-1], 977 * or UINT64_MAX if the stream ID is not available. If called on a QUIC 978 * connection, returns the unique stream ID for the default stream if there is 979 * one, and otherwise returns UINT64_MAX. 980 * 981 * TLS, DTLS: Returns UINT64_MAX. 982 */ 983__owur uint64_t SSL_get_stream_id(SSL *ssl); 984``` 985 986#### `SSL_is_stream_local` 987 988| Semantics | `SSL_get_error` | Can Tick? | CSHL | 989| --------- | ------------- | --------- | ------------- | 990| New | Never | No | S | 991 992```c 993/* 994 * QUIC: Returns 1 if the stream was locally initiated, or 0 otherwise. 995 * 996 * TLS, DTLS: Returns -1. 997 */ 998__owur int SSL_is_stream_local(SSL *ssl); 999``` 1000 1001#### `SSL_new_stream` 1002 1003| Semantics | `SSL_get_error` | Can Tick? | CSHL | 1004| --------- | ------------- | --------- | ------------- | 1005| New | Never | No | C | 1006 1007```c 1008/* 1009 * Create a new SSL object representing a single additional stream. 1010 * 1011 * There is no need to call SSL_connect on the resulting object, and 1012 * any such call is a no-op. 1013 * 1014 * For QUIC: 1015 * Creates a new stream. Must be called only on a QUIC connection SSL object. 1016 * Can be used on client or server. If the SSL_STREAM_FLAG_UNI flag is set, 1017 * the created stream is unidirectional, otherwise it is bidirectional. 1018 * 1019 * To be MSMT-safe. 1020 * 1021 * For TLS and DTLS SSL objects: 1022 * Always fails. 1023 */ 1024#define SSL_STREAM_FLAG_UNI 1 1025 1026SSL *SSL_new_stream(SSL *ssl, uint64_t flags); 1027``` 1028 1029#### `SSL_accept_stream` 1030 1031| Semantics | `SSL_get_error` | Can Tick? | CSHL | 1032| --------- | ------------- | --------- | ------------- | 1033| New | Never | Yes | C | 1034 1035```c 1036/* 1037 * Create a new SSL object representing an additional stream which was created 1038 * by the peer. 1039 * 1040 * There is no need to call SSL_accept on the resulting object, and 1041 * any such call is a no-op. 1042 * 1043 * For QUIC: 1044 * Must be called only on a QUIC connection SSL object. Fails if called on a 1045 * stream object. Checks if a new stream has been created by the peer. If it 1046 * has, creates a new SSL object to represent it and returns it. Otherwise, 1047 * returns NULL. If multiple streams are available to be accepted, the oldest 1048 * stream (that is, the stream with the lowest stream ID) is accepted. 1049 * 1050 * For all other methods: 1051 * Returns NULL. 1052 * 1053 * The flags argument is unused and should be set to zero. 1054 * 1055 * To be MSMT-safe (i.e., can be called from multiple threads). 1056 * 1057 * If the QUIC connection SSL object is configured in blocking mode, this 1058 * function will block unless the SSL_ACCEPT_STREAM_NO_BLOCK flag is passed. 1059 * 1060 * This function returns NULL if the effective incoming stream reject policy is 1061 * `REJECT`. 1062 */ 1063#define SSL_ACCEPT_STREAM_NO_BLOCK 1 1064 1065SSL *SSL_accept_stream(SSL *ssl, uint64_t flags); 1066``` 1067 1068#### `SSL_get_accept_stream_queue_len` 1069 1070| Semantics | `SSL_get_error` | Can Tick? | CSHL | 1071| --------- | ------------- | --------- | ------------- | 1072| New | Never | No | C | 1073 1074```c 1075/* 1076 * Determine the number of streams waiting to be returned on a subsequent call 1077 * to SSL_accept_stream. If this returns a non-zero value, the next call to 1078 * SSL_accept_stream (on any thread) is guaranteed to work. Returns 0 for 1079 * non-QUIC objects, or for QUIC stream objects. 1080 * 1081 * To be MSMT-safe. 1082 */ 1083size_t SSL_get_accept_stream_queue_len(SSL *ssl); 1084``` 1085 1086#### `SSL_set_incoming_stream_policy` 1087 1088| Semantics | `SSL_get_error` | Can Tick? | CSHL | 1089| --------- | ------------- | --------- | ------------- | 1090| New | Never | No | C | 1091 1092```c 1093/* 1094 * Sets the policy for incoming streams. If `policy` is `AUTO` (the default): 1095 * 1096 * - if the default stream mode is 1097 * `SSL_DEFAULT_STREAM_MODE_AUTO_BIDI` or 1098 * `SSL_DEFAULT_STREAM_MODE_AUTO_UNI`, this is equivalent to `REJECT`; 1099 * 1100 * - otherwise, this is equivalent to `ACCEPT`. 1101 * 1102 * If configured to `ACCEPT`, incoming streams are placed on the accept queue 1103 * for application consumption. `aec` is ignored in this case. 1104 * 1105 * If configured to `REJECT`, incoming streams automatically have both their 1106 * receiving and sending parts handled via non-normal termination. `aec` is an 1107 * application error code used for the `STOP_SENDING` and `RESET_STREAM` frames 1108 * used for the purposes of this termination. The default AEC value used if this 1109 * function is never called is 0. 1110 */ 1111#define SSL_INCOMING_STREAM_POLICY_AUTO 0 1112#define SSL_INCOMING_STREAM_POLICY_ACCEPT 1 1113#define SSL_INCOMING_STREAM_POLICY_REJECT 2 1114 1115int SSL_set_incoming_stream_policy(SSL *ssl, int policy, uint64_t aec); 1116``` 1117 1118#### `SSL_set_default_stream_mode` 1119 1120| Semantics | `SSL_get_error` | Can Tick? | CSHL | 1121| --------- | ------------- | --------- | ------------- | 1122| New | Never | No | C | 1123 1124```c 1125/* 1126 * Used to control single stream operation. Calling this function determines the 1127 * nature of the default stream which will automatically be created on the QUIC 1128 * connection SSL object. 1129 * 1130 * The default mode is `SSL_DEFAULT_STREAM_MODE_AUTO_BIDI`. 1131 * 1132 * The modes are as follows: 1133 * 1134 * - `SSL_DEFAULT_STREAM_MODE_NONE`: No default stream will ever be created. 1135 * The application is assumed to understand multi-stream operation. 1136 * Remotely-initiated streams are placed in the accept queue for application 1137 * consumption. `SSL_read` and `SSL_write` calls must be made on a QUIC 1138 * stream SSL object, not the QUIC connection SSL object, as no default 1139 * stream will be associated with it. 1140 * 1141 * - `SSL_DEFAULT_STREAM_MODE_AUTO_BIDI`: "First stream wins" mode of 1142 * operation for single-stream usage. If `SSL_write` is called before the 1143 * peer opens a remotely-initiated stream, a locally-initiated bidirectional 1144 * stream is created and bound as the default stream. If the peer opens a 1145 * remotely-initiated stream before the local application calls `SSL_write` 1146 * (with `len > 0`) for the first time, that stream is bound as the default 1147 * stream, which may be bidirectional or unidirectional; if it is 1148 * unidirectional, calls to `SSL_write` will fail. Attempts to create 1149 * additional streams by the peer are automatically rejected unless 1150 * the application opts in (API TBD). 1151 * 1152 * - `SSL_DEFAULT_STREAM_MODE_AUTO_UNI`: "First stream wins" mode of 1153 * operation for single-stream usage, with a unidirectional stream. This 1154 * functions identically to `SSL_DEFAULT_STREAM_MODE_AUTO_BIDI`, but if the 1155 * local application calls `SSL_write` prior to the peer creating a 1156 * remotely-initiated stream, a unidirectional TX-only stream is created and 1157 * bound as the default stream. Thereafter, calls to `SSL_read` will fail. 1158 * If the peer creates a remotely-initiated stream prior to the first call 1159 * to `SSL_write` (with `len > 0`), that stream will be bound as the default 1160 * stream; note that a bidirectional stream may be bound in this case. 1161 * Attempts to create additional streams by the peer are automatically 1162 * rejected unless the application opts in (API TBD). 1163 * 1164 * This function must be called before a default stream object is created, for 1165 * example before initiating a connection. If the function is too late to have 1166 * an effect, this function fails and returns 0. 1167 */ 1168#define SSL_DEFAULT_STREAM_MODE_NONE 0 1169#define SSL_DEFAULT_STREAM_MODE_AUTO_BIDI 1 1170#define SSL_DEFAULT_STREAM_MODE_AUTO_UNI 2 1171 1172__owur int SSL_set_default_stream_mode(SSL *ssl, uint32_t mode); 1173``` 1174 1175### Future APIs 1176 1177A custom poller interface may be provided in the future. For more information, 1178see the QUIC I/O Architecture design document. 1179 1180BIO Objects 1181----------- 1182 1183### Existing APIs 1184 1185#### `BIO_s_connect`, `BIO_new_ssl_connect`, `BIO_set_conn_hostname` 1186 1187We are aiming to support use of the existing `BIO_new_ssl_connect` API with only 1188minimal changes. This will require internal changes to `BIO_s_connect`, which 1189should automatically detect when it is being used with a QUIC `SSL_CTX` and act 1190accordingly. 1191 1192#### `BIO_new_bio_pair` 1193 1194Unsuitable for use with QUIC on the network side; instead, applications can 1195make use of the new `BIO_s_dgram_pair` which provides equivalent functionality 1196with datagram semantics. 1197 1198#### Interactions with `BIO_f_buffer` 1199 1200Existing applications sometimes combine a network socket BIO with a 1201`BIO_f_buffer`. This is problematic because the datagram semantics of writes are 1202not preserved, therefore the BIO provided to libssl is, as provided, unusable 1203for the purposes of implementing QUIC. Moreover, output buffering is not a 1204relevant or desirable performance optimisation for the transmission of UDP 1205datagrams and will actually undermine QUIC performance by causing incorrect 1206calculation of ACK delays and consequently inaccurate RTT calculation. 1207 1208Options: 1209 1210 - Require applications to be changed to not use QUIC with a `BIO_f_buffer`. 1211 - Detect when a `BIO_f_buffer` is part of a BIO stack and bypass it 1212 (yucky and surprising). 1213 1214#### MTU Signalling 1215 1216**See also:** 1217[BIO_s_dgram_pair(3)](https://www.openssl.org/docs/manmaster/man3/BIO_s_dgram_pair.html) 1218 1219`BIO_dgram_get_mtu` (`BIO_CTRL_DGRAM_GET_MTU`) and `BIO_dgram_set_mtu` 1220(`BIO_CTRL_DGRAM_SET_MTU`) already exist for `BIO_s_dgram` and are implemented 1221on a `BIO_s_dgram_pair` to allow the MTU to be determined and configured. One 1222side of a pair can configure the MTU to allow the other side to detect it. 1223 1224`BIO_s_dgram` also has pre-existing support for getting the correct MTU value 1225from the OS using `BIO_CTRL_DGRAM_QUERY_MTU`. 1226 1227### New APIs 1228 1229#### `BIO_sendmmsg` and `BIO_recvmmsg` 1230 1231**See also:** 1232[BIO_sendmmsg(3)](https://www.openssl.org/docs/manmaster/man3/BIO_sendmmsg.html) 1233 1234The BIO interface features a new high-performance API for the execution of 1235multiple read or write operations in a single system call, on supported OSes. On 1236other OSes, a compatible fallback implementation is used. 1237 1238Unlike all other BIO APIs, this API is intended for concurrent threaded use and 1239as such operates in a stateless fashion with regards to a BIO. This means, for 1240example, that retry indications are made using explicit API inputs and outputs 1241rather than setting an internal flag on the BIO. 1242 1243This new BIO API includes: 1244 1245- Local address support (getting the destination address of an incoming 1246 packet; setting the source address of an outgoing packet), where support 1247 for this is available; 1248- Peer address support (setting the destination address of an outgoing 1249 packet; getting the source address of an incoming packet), where support 1250 for this is available. 1251 1252The following functionality was intentionally left out of this design because 1253not all OSes can provide support: 1254 1255- Iovecs (which have also been determined not to be necessary for a 1256 performant QUIC implementation); 1257- Features such as `MSG_DONTWAIT`, etc. 1258 1259This BIO API is intended to be extensible. For more information on this API, see 1260BIO_sendmmsg(3) and BIO_recvmmsg(3). 1261 1262Custom BIO implementers may set their own implementation of these APIs via 1263corresponding `BIO_meth` getter/setter functions. 1264 1265#### Truncation Mode 1266 1267**See also:** 1268[BIO_s_dgram_pair(3)](https://www.openssl.org/docs/manmaster/man3/BIO_s_dgram_pair.html) 1269 1270The controls `BIO_dgram_set_no_trunc` (`BIO_CTRL_DGRAM_SET_NO_TRUNC`) and 1271`BIO_dgram_get_no_trunc` (`BIO_CTRL_DGRAM_GET_NO_TRUNC`) are introduced. This is 1272a boolean value which may be implemented by BIOs with datagram semantics. When 1273enabled, attempting to receive a datagram such that the datagram would 1274ordinarily be truncated (as per the design of the Berkeley sockets API) instead 1275results in a failure. This is intended for implementation by `BIO_s_dgram_pair`. 1276For compatibility, the default behaviour is off. 1277 1278#### Capability Negotiation 1279 1280**See also:** 1281[BIO_s_dgram_pair(3)](https://www.openssl.org/docs/manmaster/man3/BIO_s_dgram_pair.html) 1282 1283Where a `BIO_s_dgram_pair` is used, there is the potential for such a memory BIO 1284to be used by existing application code which is being adapted for use with 1285QUIC. A problem arises whereby one end of a `BIO_s_dgram_pair` (for example, the 1286side being used by OpenSSL's QUIC implementation) may assume that the other end 1287supports certain capabilities (for example, specifying a peer address), when in 1288actual fact the opposite end of the `BIO_s_dgram_pair` does not. 1289 1290A capability signalling mechanism is introduced which allows one end of a 1291`BIO_s_dgram_pair` to indicate to the user of the opposite BIO the following 1292capabilities and related information: 1293 1294- Whether source addresses the peer specifies will be processed. 1295- Whether destination addresses the peer specifies will be processed. 1296- Whether source addresses will be provided to the opposite BIO when it 1297 receives datagrams. 1298- Whether destination addresses will be provided to the opposite BIO 1299 when it receives datagrams. 1300 1301The usage is as follows: 1302 1303- One side of a BIO pair calls `BIO_dgram_set_caps` with zero or 1304 more of the following flags to advertise its capabilities: 1305 - `BIO_DGRAM_CAP_HANDLES_SRC_ADDR` 1306 - `BIO_DGRAM_CAP_HANDLES_DST_ADDR` 1307 - `BIO_DGRAM_CAP_PROVIDES_SRC_ADDR` 1308 - `BIO_DGRAM_CAP_PROVIDES_DST_ADDR` 1309- The other side of the BIO pair calls `BIO_dgram_get_effective_caps` 1310 to learn the effective capabilities of the BIO. These are the capabilities set 1311 by the opposite BIO. 1312- The above process can also be repeated in the opposite direction. 1313 1314#### Local Address Support 1315 1316**See also:** 1317[BIO_s_dgram_pair(3)](https://www.openssl.org/docs/manmaster/man3/BIO_s_dgram_pair.html) 1318 1319Support for local addressing (the reception of destination addresses for 1320incoming packets, and the specification of source addresses for outgoing 1321packets) varies by OS. Thus, it may not be available in all circumstances. A 1322feature negotiation mechanism is introduced to facilitate this. 1323 1324`BIO_dgram_get_local_addr_cap` (`BIO_CTRL_DGRAM_GET_LOCAL_ADDR_CAP`) determines 1325if a BIO is potentially capable of supporting local addressing on the current 1326platform. If it determines that support is available, local addressing support 1327must then be explicitly enabled via `BIO_dgram_set_local_addr_enable` 1328(`BIO_CTRL_DGRAM_SET_LOCAL_ADDR_ENABLE`). If local addressing support has not 1329been enabled, attempts to use local addressing (for example via `BIO_sendmmsg` 1330or `BIO_recvmmsg` with a `BIO_MSG` with a non-NULL `local` field) fails. 1331 1332An explicit enablement call is required because setting up local addressing 1333support requires system calls on most operating systems prior to sending or 1334receiving packets and we do not wish to do this automatically inside the 1335`BIO_sendmmsg`/`BIO_recvmmsg` fastpaths, particularly since the process of 1336enabling support could fail due to lack of OS support, etc. 1337 1338`BIO_dgram_get_local_addr_enable` (`BIO_CTRL_DGRAM_GET_LOCAL_ADDR_ENABLE`) is 1339also available. 1340 1341It is important to note that `BIO_dgram_get_local_addr_cap` is entirely distinct 1342from the application capability negotiation mechanism discussed above. Whereas 1343the capability negotiation mechanism discussed above allows *applications* to 1344signal what they are capable of handling in their usage of a given BIO, 1345`BIO_dgram_local_addr_cap` allows a *BIO implementation* to indicate to the 1346users of that BIO whether it is able to support local addressing (where 1347enabled). 1348 1349#### `BIO_s_dgram_pair` 1350 1351**See also:** 1352[BIO_s_dgram_pair(3)](https://www.openssl.org/docs/manmaster/man3/BIO_s_dgram_pair.html) 1353 1354A new BIO implementation, `BIO_s_dgram_pair`, is provided. This is similar to 1355the existing BIO pair but provides datagram semantics. It provides full support 1356for the new APIs `BIO_sendmmsg`, `BIO_recvmmsg`, the capability negotiation 1357mechanism described above, local address support and the MTU signalling 1358mechanism described above. 1359 1360It can be instantiated using the new API `BIO_new_dgram_pair`. 1361 1362#### `BIO_POLL_DESCRIPTOR` 1363 1364The concept of *poll descriptors* are introduced. A poll descriptor is a tagged 1365union structure which represents an abstraction over some unspecified kind of OS 1366descriptor which can be used for synchronization and waiting. 1367 1368The most commonly used kind of poll descriptor is one which describes a network 1369socket (i.e., on POSIX-like platforms, a file descriptor), however other kinds 1370of poll descriptor may be defined. 1371 1372A BIO may be queried for whether it has a poll descriptor for read or write 1373operations respectively: 1374 1375- Where `BIO_get_rpoll_descriptor` (`BIO_CTRL_GET_RPOLL_DESCRIPTOR`) is called, 1376 the BIO should output a poll descriptor which describes a resource which can 1377 be used to determine when the BIO will next become readable via a call to 1378 `BIO_read` or, if supported by the BIO, `BIO_recvmmsg`. 1379- Where 1380 `BIO_get_wpoll_descriptor` (`BIO_CTRL_GET_WPOLL_DESCRIPTOR`) is called, the 1381 BIO should output a poll descriptor which describes a resource which can be 1382 used to determine when the BIO will next become writeable via a call to 1383 `BIO_write` or, if supported by the BIO, `BIO_sendmmsg`. 1384 1385A BIO may not necessarily be able to provide a poll descriptor. For example, 1386memory-based BIOs such as `BIO_s_dgram_pair` do not correspond to any OS 1387synchronisation resource, and thus the `BIO_get_rpoll_descriptor` and 1388`BIO_get_wpoll_descriptor` calls are not supported for such BIOs. 1389 1390A BIO which supports these functions is known as pollable, and a BIO which does 1391not is known as non-pollable. `BIO_s_dgram` supports these functions. 1392 1393The implementation of these functions for a `BIO_f_ssl` forwards to 1394`SSL_get_rpoll_descriptor` and `SSL_get_wpoll_descriptor` respectively. The 1395 1396#### `BIO_s_dgram_mem` 1397 1398This is a basic memory buffer BIO with datagram semantics. Unlike 1399`BIO_s_dgram_pair`, it is unidirectional and does not support peer addressing or 1400local addressing. 1401 1402#### `BIO_err_is_non_fatal` 1403 1404A new predicate function `BIO_err_is_non_fatal` is defined which determines if 1405an error code represents a non-fatal or transient error. For details, see 1406[BIO_sendmmsg(3)](https://www.openssl.org/docs/manmaster/man3/BIO_sendmmsg.html). 1407 1408Q & A 1409----- 1410 1411To assist in understanding, when a “TBD” listed above is removed, or when a 1412relevant question is raised, the resolution to the question will be placed here. 1413 1414**Q. Should `SSL_do_handshake` wait until the handshake is completed, or until it 1415is confirmed?** 1416 1417**Note:** [The terms *handshake complete* and *handshake confirmed* are defined 1418in RFC 9001 and have specific 1419meanings.](https://www.rfc-editor.org/rfc/rfc9001.html#name-handshake-complete) 1420 1421A. `SSL_do_handshake` should wait until the handshake is completed, because 1422handshake completion represents the completion of the cryptographic 1423authentication of the connection. When a connection's handshake is completed, 1424TLS 1.3 Finished messages have been exchanged by both parties, even if the 1425handshake has not yet been *confirmed*. Moreover, RFC 9001 s. 4.1.2 states: 1426 1427>Additionally, a client MAY consider the handshake to be confirmed when it 1428>receives an acknowledgment for a 1-RTT packet. 1429 1430This logically implies that it is OK for a client to start transmitting 1-RTT 1431packets prior to handshake confirmation, otherwise there would be no in-flight 14321-RTT packets for the client to receive ACKs for. 1433 1434**Q. Does `ENABLE_PARTIAL_WRITE` interact with blocking mode?** 1435 1436A. No; this mode is only relevant to non-blocking mode. In blocking mode, 1437`SSL_write` always waits until all data is written unless an error occurs. The 1438semantics of `SSL_write` are preserved unchanged. 1439 1440**Q. Does `SSL_write` block until data is written to the network, or simply 1441until it is buffered?** 1442 1443A. `SSL_write` blocks until it has accepted responsibility for the data passed 1444to it, just like `write(2)` or `send(2)`. In other words, it blocks until it can 1445buffer the data. This does not necessarily mean that the data has actually been 1446sent. 1447 1448**Q. How should connection closure work?** 1449 1450A. **RFC requirements.** After we begin terminating the connection by sending a 1451`CONNECTION_CLOSE` frame, QUIC requires that we continue to process network I/O 1452for a certain period of time so that any further traffic from the peer results 1453in generation of a further `CONNECTION_CLOSE` frame. This is necessary to handle 1454the possibility that the `CONNECTION_CLOSE` frame which was initially sent may 1455be lost. 1456 1457**API issues.** This creates a complication because it implies that the 1458connection closure process may take a fair amount of time, whereas existing API 1459users will generally expect to be able to call `SSL_shutdown` and then 1460immediately free the SSL object. 1461 1462However, if the caller immediately frees the SSL object, this precludes 1463our implementing the applicable logic, at least on the client side. Moreover, 1464existing API users are likely to tear down underlying network BIOs immediately 1465after calling `SSL_free` anyway. In other words, any implementation based on 1466secretly keeping QUIC state around after a call to `SSL_free` does not seem 1467particularly workable on the client side. 1468 1469**Server side considerations.** There is more of a prospect here on the server 1470side, since multiple connections will share the same socket, which will 1471presumably be associated with some kind of enduring listener object. Thus when 1472server support is implemented in the future connection teardown could be handled 1473internally by maintaining the state of connections undergoing termination inside 1474the listener object. However, similar caveats to those discussed here arise when 1475the listener object itself is to be town down. (It is also possible we could 1476optionally allow use of the server-style API to make multiple outgoing client 1477connections with a non-zero-length client-side CID on the same underlying 1478network BIO.) 1479 1480There are only really two ways to handle this: 1481 1482- **RFC conformant mode.** `SSL_shutdown` only indicates that shutdown is 1483 complete once the entire connection closure process is complete. 1484 1485 This process consists of the Closing and Draining states. In some cases the 1486 Closing state may last only briefly, namely if the peer chooses to respond to 1487 our `CONNECTION_CLOSE` frame with a `CONNECTION_CLOSE` frame of its own. This 1488 allows immediate progression to the Draining state. However, a peer is *not* 1489 required to respond with such a frame. Thus in the worst case, this state can 1490 be as long as `3*PTO`; for example a peer with a high estimated RTT of 300ms 1491 would have us wait for 900ms. 1492 1493 In the Draining state we simply ignore all incoming traffic and do not 1494 generate outgoing traffic. The purpose of this state is to simply tie up the 1495 socket and ensure any data still in flight is discarded. However, RFC 9000 1496 states: 1497 1498 Disposing of connection state prior to exiting the closing or draining state 1499 could result in an endpoint generating a Stateless Reset unnecessarily when 1500 it receives a late-arriving packet. Endpoints that have some alternative 1501 means to ensure that late-arriving packets do not induce a response, such as 1502 those that are able to close the UDP socket, MAY end these states earlier to 1503 allow for faster resource recovery. Servers that retain an open socket for 1504 accepting new connections SHOULD NOT end the closing or draining state early 1505 1506 Because our client mode implementation uses one socket per connection, it 1507 appears to be reasonable based on the above text to omit the implementation of 1508 the draining state (the same may not be the case for the server role when 1509 implemented in the future). 1510 1511 Thus, in general, `SSL_shutdown` can be expected to take about one round 1512 trip's time to complete when dealing with a peer whose QUIC implementation 1513 happens to respond to a `CONNECTION_CLOSE` frame with a `CONNECTION_CLOSE` 1514 frame of its own, and about three round trips otherwise. 1515 1516- **Rapid shutdown mode.** `SSL_shutdown` sends a `CONNECTION_CLOSE` frame once 1517 and completes immediately. The Closing and Draining states are not used, and 1518 if the `CONNECTION_CLOSE` frame was lost, the peer will have to wait for idle 1519 timeout to determine that the connection is gone (there is also the 1520 possibility that, if the socket is closed by the application after teardown, a 1521 peer will make something of ICMP Port Unreachable messages, but this is 1522 unlikely to be reliable and since this message is not authenticated, QUIC 1523 implementations probably shouldn't pay much attention to it anyway.) 1524 1525There is little problem with `SSL_shutdown` taking as long as it needs to for 1526some long-running applications, but for others it poses a real issue. For 1527example, a command-line tool which makes one connection, performs one 1528application-specific transaction, and then tears down the connection. In this 1529case an RFC-conformant connection termination would essentially require the 1530process to hang around for a substantial amount of time after the work of the 1531process is done. 1532 1533For this reason, it is concluded that both of these shutdown modes need to be 1534offered. 1535 1536Where connection closure is initiated remotely rather than locally, only the 1537draining state is relevant. Since we conclude above that we do not need to 1538implement the draining state on the client side, this means that connection 1539closure can be completed immediately in the case of a remote closure. 1540 1541**Q. Should we just map `SSL_handle_events` to `DTLS_CTRL_HANDLE_TIMEOUT` 1542internally?** 1543 1544A. No, since the infinite time representation is different between the two 1545calls. 1546 1547**Q. How should `STOP_SENDING` be supported?** 1548 1549We trigger `STOP_SENDING` automatically if an application frees the associated 1550QUIC stream SSL object. 1551 1552**Q. Can data be received on a locally initiated bidirectional stream before any 1553data is sent on that stream?** 1554 1555This is an interesting question without a clear answer to be found in the QUIC 1556RFCs. A close reading of RFC 9000 suggests that the answer is, in principle, 1557yes; however the RFC also grants explicit permission to make design choices in 1558implementations which would preclude this: 1559 1560>An implementation might choose to defer allocating a stream ID to a stream until 1561>it sends the first STREAM frame and enters this state, which can allow for 1562>better stream prioritization. 1563 1564If an ID has not been allocated to a stream, obviously incoming data cannot be 1565addressed to it. However, supposing that an implementation does not do this, 1566RFC 9000 seems basically clear that it is valid for an application to create a 1567stream locally, then receive data on it before sending anything: 1568 1569>The sending part of a stream that the endpoint initiates (types 0 and 2 for 1570>clients, 1 and 3 for servers) is opened by the application. The "Ready" 1571>state represents a newly created stream that is able to accept data from the 1572>application. 1573> 1574>[...] 1575> 1576>For a bidirectional stream, the receiving part enters the "Recv" state when 1577>the sending part initiated by the endpoint (type 0 for a client, type 1 for 1578>a server) enters the "Ready" state. 1579 1580A peer is not generally notified of the creation of a stream which has not sent 1581any data yet, since the creation of a stream is signalled only implicitly via 1582the transmission of data in `STREAM` frames. However, a zero-length STREAM frame 1583could presumably be used to effect such a notification. RFC 9000 contains no 1584specific discussion of this possibility but does not preclude it. As such, in 1585order to receive data on a locally-initiated bidirectional stream before sending 1586any data on that stream, it would be necessary to either 1587 1588- Use a QUIC implementation which signals a bidirectional stream which has 1589 not yet sent any data via a zero-length stream frame, or 1590 1591- Use an application protocol which can inform the peer of the stream ID 1592 of the created stream in some application protocol-specific way. 1593 This is somewhat less plausible because it would require an API between 1594 the application and its QUIC library to inform the QUIC library 1595 that the peer has in fact created a stream with a given ID and to 1596 take its word for it. This is unlikely to be commonly available, especially as 1597 application errors in usage of such an API would lead to internal 1598 inconsistencies in QUIC connection state. 1599 1600Of course this discussion is somewhat esoteric as it is unclear why an 1601application would want to create a locally-initiated stream and then have the 1602peer transmit on it first, rather than simply use a remotely-initiated stream. 1603Thus this discussion of this edge case is more of a curiosity, however for 1604completeness it needs to be thought about in the API design. 1605 1606**Q. How should single-stream operation support locally and remotely-initiated 1607streams?** 1608 1609Note that the ID of a stream depends on whether it is bidirectional and whether 1610it is initiated by the client or server. Therefore, in single stream operation, 1611it is necessary to know whether single-stream QUIC is being used with 1612client-initiated or server-initiated stream initiation, and whether a 1613bidirectional or unidirectional stream is being used; otherwise, we do not know 1614which stream ID to bind to. 1615 1616The object of single stream operation is to support simple uses cases for simple 1617applications. There seems no need to support esoteric usage of streams such as 1618receiving first on a locally initiated stream here, thus we avoid supporting 1619this to simplify the API. 1620 1621As such, an application which calls `SSL_write` on a QUIC connection SSL object 1622before it calls `SSL_read` by definition is using a locally-initiated stream, 1623and an application which does the opposite is using a remotely-initiated stream. 1624We can use the ordering of initial calls to `SSL_read` and `SSL_write` to infer 1625the desired stream type. 1626 1627Supporting locally-initiated streams (`SSL_write` called first) is simple; 1628we automatically create the stream and queue data for transmission. 1629 1630Supporting remotely-initiated streams (`SSL_read` called first) is a little 1631stranger. We could create the stream with the correct ID when cued to by the 1632initial call to `SSL_read` implying use of a remotely-initiated stream. However, 1633this would mean we are creating state tracking a remotely-initiated stream 1634before the peer has signalled it. This would work in the happy case where the 1635client is connected to a compatible server but may result in strange 1636inconsistencies of QUIC internal state if a client is accidentally connected to 1637an incompatible peer. Since the peer ought to be the authority on the streams it 1638creates, this seems like an undesirable approach. 1639 1640Ergo, creation of a default remotely-initiated stream needs to be deferred 1641until the *peer* signals such a stream. 1642 1643This leads naturally to a "first stream wins" model of implementation: 1644 1645- When a QUIC connection SSL object is created, default stream mode is 1646 enabled, meaning that a default stream will be bound to the QUIC connection 1647 SSL object at the earliest available opportunity. However, no default 1648 stream is bound yet. 1649 1650- One of the following events happened — whichever happens first wins: 1651 1652 - The local application calls `SSL_write()` (`len > 0`). A locally-initiated 1653 stream with ordinal 0 is created. The stream is bidirectional by default but 1654 this can be changed. This stream is bound as the default stream. 1655 1656 - The peer creates a stream. This stream is bound as the default stream. 1657 1658If the local application calls `SSL_read()` before either of the above 1659occur, `SSL_read()` fails as though no data is available until one 1660of the above events occurs. 1661 1662Once one of the above events occurs, any additional stream created by the peer 1663is automatically terminated using both `STOP_SENDING` and `STREAM_RESET` frames 1664(to terminate both the receiving and sending parts respectively) and there is no 1665API-visible effect to the local application (unless the application explicitly 1666opts into supporting additional streams). 1667