1QUIC Thread Assisted Mode Synchronisation Requirements
2======================================================
3
4In thread assisted mode, we create a background thread to ensure that periodic
5QUIC processing is handled in a timely fashion regardless of whether an
6application is frequently calling (or blocked in) SSL API I/O functions.
7
8Part of the QUIC state comprises the TLS handshake layer. However, synchronising
9access to this is extremely difficult.
10
11At first glance, one could synchronise handshake layer public APIs by locking a
12per-connection mutex for the duration of any public API call which we forward to
13the handshake layer. Since we forward a very large number of APIs to the
14handshake layer, this would require a very large number of code changes to add
15the locking to every single public HL-related API call.
16
17However, on second glance, this does not even solve the problem, as
18applications existing usage of the HL APIs assumes exclusive access, and thus
19consistency over multiple API calls. For example:
20
21    x = SSL_get_foo(s);
22    /* application mutates x */
23    SSL_set_foo(s, x);
24
25For locking of API calls the lock would only be held for the separate get and
26set calls, but the combination of the two would not be safe if the assist thread
27can process some event which causes mutation of `foo`.
28
29As such, there are really only three possible solutions:
30
31- **1. Application-controlled explicit locking.**
32
33  We would offer something like `SSL_lock()` and `SSL_unlock()`.
34  An application performing a single HL API call, or a sequence of related HL
35  calls, would be required to take the lock. As a special exemption, an
36  application is not required to take the lock prior to connection
37  (specifically, prior to the instantiation of a QUIC channel and consequent
38  assist thread creation).
39
40  The key disadvantage here is that it requires more API changes on the
41  application side, although since most HL API calls made by an application
42  probably happen prior to initiating a connection, things may not be that bad.
43  It would also only be required for applications which want to use thread
44  assisted mode.
45
46  Pro: Most “robust” solution in terms of HL evolution.
47
48  Con: API changes.
49
50- **2. Handshake layer always belongs to the application thread.**
51
52  In this model, the handshake layer “belongs” to the application thread
53  and the assist thread is never allowed to touch it:
54
55  - `SSL_tick()` (or another I/O function) called by the application fully
56    services the connection.
57
58  - The assist thread performs a reduced tick operation which does everything
59    except servicing the crypto stream, or any other events we may define in
60    future which would be processed by the handshake layer.
61
62  - This is rather hacky but should work adequately. When using TLS 1.3
63    as the handshake layer, the only thing we actually need to worry about
64    servicing after handshake completion is the New Session Ticket message,
65    which doesn't need to be acknowledged and isn't “urgent”. The other
66    post-handshake messages used by TLS 1.3 aren't relevant to QUIC TLS:
67
68    - Post-handshake authentication is not allowed;
69
70    - Key update uses a separate, QUIC-specific method;
71
72    - TLS alerts are signalled via `CONNECTION_CLOSE` frames rather than the TLS
73      1.3 Alert message; thus if a peer's HL does raise an alert after
74      handshake completion (which would in itself be highly unusual), we simply
75      receive a `CONNECTION_CLOSE` frame and process it normally.
76
77  Thus so long as we don't expect our own TLS implementation to spontaneously
78  generate alerts or New Session Ticket messages after handshake completion,
79  this should work.
80
81  Pro: No API changes.
82
83  Con: Somewhat hacky solution.
84
85- **3. Handshake layer belongs to the assist thread after connection begins.**
86
87  In this model, the application may make handshake layer calls freely prior to
88  connecting, but after that, ownership of the HL is transferred to the assist
89  thread and may not be touched further. We would need to block all API calls
90  which would forward to the HL after connection commences (specifically, after
91  the QUIC channel is instantiated).
92
93  Con: Many applications probably expect to be able to query the HL after
94  connection. We could selectively enable some important post-handshake HL calls
95  by specially implementing synchronised forwarders, but doing this in the
96  general case runs into the same issues as option 1 above. We could only enable
97  APIs we think have safe semantics here; e.g. implement only getters and not
98  setters, focus on APIs which return data which doesn't change after
99  connection. The work required is proportional to the number of APIs to be
100  enabled. Some APIs may not have ways to indicate failure; for such APIs which
101  we don't implement for thread assisted post-handshake QUIC, we would
102  essentially return incorrect data here.
103
104Option 2 has been chosen as the basis for implementation.
105