1=pod 2 3=head1 NAME 4 5OSSL_HTTP_open, 6OSSL_HTTP_bio_cb_t, 7OSSL_HTTP_proxy_connect, 8OSSL_HTTP_set1_request, 9OSSL_HTTP_exchange, 10OSSL_HTTP_get, 11OSSL_HTTP_transfer, 12OSSL_HTTP_close 13- HTTP client high-level functions 14 15=head1 SYNOPSIS 16 17 #include <openssl/http.h> 18 19 typedef BIO *(*OSSL_HTTP_bio_cb_t)(BIO *bio, void *arg, 20 int connect, int detail); 21 OSSL_HTTP_REQ_CTX *OSSL_HTTP_open(const char *server, const char *port, 22 const char *proxy, const char *no_proxy, 23 int use_ssl, BIO *bio, BIO *rbio, 24 OSSL_HTTP_bio_cb_t bio_update_fn, void *arg, 25 int buf_size, int overall_timeout); 26 int OSSL_HTTP_proxy_connect(BIO *bio, const char *server, const char *port, 27 const char *proxyuser, const char *proxypass, 28 int timeout, BIO *bio_err, const char *prog); 29 int OSSL_HTTP_set1_request(OSSL_HTTP_REQ_CTX *rctx, const char *path, 30 const STACK_OF(CONF_VALUE) *headers, 31 const char *content_type, BIO *req, 32 const char *expected_content_type, int expect_asn1, 33 size_t max_resp_len, int timeout, int keep_alive); 34 BIO *OSSL_HTTP_exchange(OSSL_HTTP_REQ_CTX *rctx, char **redirection_url); 35 BIO *OSSL_HTTP_get(const char *url, const char *proxy, const char *no_proxy, 36 BIO *bio, BIO *rbio, 37 OSSL_HTTP_bio_cb_t bio_update_fn, void *arg, 38 int buf_size, const STACK_OF(CONF_VALUE) *headers, 39 const char *expected_content_type, int expect_asn1, 40 size_t max_resp_len, int timeout); 41 BIO *OSSL_HTTP_transfer(OSSL_HTTP_REQ_CTX **prctx, 42 const char *server, const char *port, 43 const char *path, int use_ssl, 44 const char *proxy, const char *no_proxy, 45 BIO *bio, BIO *rbio, 46 OSSL_HTTP_bio_cb_t bio_update_fn, void *arg, 47 int buf_size, const STACK_OF(CONF_VALUE) *headers, 48 const char *content_type, BIO *req, 49 const char *expected_content_type, int expect_asn1, 50 size_t max_resp_len, int timeout, int keep_alive); 51 int OSSL_HTTP_close(OSSL_HTTP_REQ_CTX *rctx, int ok); 52 53=head1 DESCRIPTION 54 55OSSL_HTTP_open() initiates an HTTP session using the I<bio> argument if not 56NULL, else by connecting to a given I<server> optionally via a I<proxy>. 57 58Typically the OpenSSL build supports sockets and the I<bio> parameter is NULL. 59In this case I<rbio> must be NULL as well and the I<server> must be non-NULL. 60The function creates a network BIO internally using L<BIO_new_connect(3)> 61for connecting to the given server and the optionally given I<port>, 62defaulting to 80 for HTTP or 443 for HTTPS. 63Then this internal BIO is used for setting up a connection 64and for exchanging one or more request and response. 65If I<bio> is given and I<rbio> is NULL then this I<bio> is used instead. 66If both I<bio> and I<rbio> are given (which may be memory BIOs for instance) 67then no explicit connection is set up, but 68I<bio> is used for writing requests and I<rbio> for reading responses. 69As soon as the client has flushed I<bio> the server must be ready to provide 70a response or indicate a waiting condition via I<rbio>. 71 72If I<bio> is given, it is an error to provide I<proxy> or I<no_proxy> arguments, 73while I<server> and I<port> arguments may be given to support diagnostic output. 74If I<bio> is NULL the optional I<proxy> parameter can be used to set an 75HTTP(S) proxy to use (unless overridden by "no_proxy" settings). 76If TLS is not used this defaults to the environment variable C<http_proxy> 77if set, else C<HTTP_PROXY>. 78If I<use_ssl> != 0 it defaults to C<https_proxy> if set, else C<HTTPS_PROXY>. 79An empty proxy string C<""> forbids using a proxy. 80Otherwise, the format is 81C<[http[s]://][userinfo@]host[:port][/path][?query][#fragment]>, 82where any userinfo, path, query, and fragment given is ignored. 83If the host string is an IPv6 address, it must be enclosed in C<[> and C<]>. 84The default proxy port number is 80, or 443 in case "https:" is given. 85The HTTP client functions connect via the given proxy unless the I<server> 86is found in the optional list I<no_proxy> of proxy hostnames or IP addresses 87separated by C<,> and/or whitespace (if not NULL; 88default is the environment variable C<no_proxy> if set, else C<NO_PROXY>). 89Proxying plain HTTP is supported directly, 90while using a proxy for HTTPS connections requires a suitable callback function 91such as OSSL_HTTP_proxy_connect(), described below. 92 93If I<use_ssl> is nonzero a TLS connection is requested 94and the I<bio_update_fn> parameter must be provided. 95 96The parameter I<bio_update_fn>, which is optional if I<use_ssl> is 0, 97may be used to modify the connection BIO used by the HTTP client, 98but cannot be used when both I<bio> and I<rbio> are given. 99I<bio_update_fn> is a BIO connect/disconnect callback function with prototype 100 101 BIO *(*OSSL_HTTP_bio_cb_t)(BIO *bio, void *arg, int connect, int detail) 102 103The callback function may modify the BIO provided in the I<bio> argument, 104whereby it may use an optional custom defined argument I<arg>, 105which can for instance point to an B<SSL_CTX> structure. 106During connection establishment, just after calling BIO_do_connect_retry(), the 107callback function is invoked with the I<connect> argument being 1 and 108I<detail> being 1 if I<use_ssl> is nonzero (i.e., HTTPS is requested), else 0. 109On disconnect I<connect> is 0 and I<detail> is 1 if no error occurred, else 0. 110For instance, on connect the callback may push an SSL BIO to implement HTTPS; 111after disconnect it may do some diagnostic output and pop and free the SSL BIO. 112 113The callback function must return either the potentially modified BIO I<bio> 114or NULL to indicate failure, in which case it should not modify the BIO. 115 116Here is a simple example that supports TLS connections (but not via a proxy): 117 118 BIO *http_tls_cb(BIO *bio, void *arg, int connect, int detail) 119 { 120 if (connect && detail) { /* connecting with TLS */ 121 SSL_CTX *ctx = (SSL_CTX *)arg; 122 BIO *sbio = BIO_new_ssl(ctx, 1); 123 124 bio = sbio != NULL ? BIO_push(sbio, bio) : NULL; 125 } else if (!connect) { /* disconnecting */ 126 BIO *hbio; 127 128 if (!detail) { /* an error has occurred */ 129 /* optionally add diagnostics here */ 130 } 131 BIO_ssl_shutdown(bio); 132 hbio = BIO_pop(bio); 133 BIO_free(bio); /* SSL BIO */ 134 bio = hbio; 135 } 136 return bio; 137 } 138 139After disconnect the modified BIO will be deallocated using BIO_free_all(). 140The optional callback function argument I<arg> is not consumed, 141so must be freed by the caller when not needed any more. 142 143The I<buf_size> parameter specifies the response header maximum line length. 144A value <= 0 means that the B<OSSL_HTTP_DEFAULT_MAX_LINE_LEN> (4KiB) is used. 145I<buf_size> is also used as the number of content bytes that are read at a time. 146 147If the I<overall_timeout> parameter is > 0 this indicates the maximum number of 148seconds the overall HTTP transfer (i.e., connection setup if needed, 149sending requests, and receiving responses) is allowed to take until completion. 150A value <= 0 enables waiting indefinitely, i.e., no timeout. 151 152OSSL_HTTP_proxy_connect() may be used by an above BIO connect callback function 153to set up an SSL/TLS connection via an HTTPS proxy. 154It promotes the given BIO I<bio> representing a connection 155pre-established with a TLS proxy using the HTTP CONNECT method, 156optionally using proxy client credentials I<proxyuser> and I<proxypass>, 157to connect with TLS protection ultimately to I<server> and I<port>. 158If the I<port> argument is NULL or the empty string it defaults to "443". 159If the I<timeout> parameter is > 0 this indicates the maximum number of 160seconds the connection setup is allowed to take. 161A value <= 0 enables waiting indefinitely, i.e., no timeout. 162Since this function is typically called by applications such as 163L<openssl-s_client(1)> it uses the I<bio_err> and I<prog> parameters (unless 164NULL) to print additional diagnostic information in a user-oriented way. 165 166OSSL_HTTP_set1_request() sets up in I<rctx> the request header and content data 167and expectations on the response using the following parameters. 168If <rctx> indicates using a proxy for HTTP (but not HTTPS), the server host 169(and optionally port) needs to be placed in the header; thus it must be present 170in I<rctx>. 171For backward compatibility, the server (and optional port) may also be given in 172the I<path> argument beginning with C<http://> (thus giving an absoluteURI). 173If I<path> is NULL it defaults to "/". 174If I<req> is NULL the HTTP GET method will be used to send the request 175else HTTP POST with the contents of I<req> and optional I<content_type>, where 176the length of the data in I<req> does not need to be determined in advance: the 177BIO will be read on-the-fly while sending the request, which supports streaming. 178The optional list I<headers> may contain additional custom HTTP header lines. 179 180If the I<expected_content_type> argument is not NULL, 181the client will check that the specified content-type string 182is included in the HTTP header of the response and return an error if not. 183In the content-type header line the specified string should be present either 184as a whole, or in case the specified string does not include a C<;> character, 185it is sufficient that the specified string appears as a prefix 186in the header line, followed by a C<;> character and any further text. 187For instance, if I<expected_content_type> specifies C<text/html>, 188this is matched by C<text/html>, C<text/html; charset=UTF-8>, etc. 189 190If the I<expect_asn1> parameter is nonzero, 191a structure in ASN.1 encoding will be expected as response content. 192The I<max_resp_len> parameter specifies the maximum allowed 193response content length, where the value 0 indicates no limit. 194If the I<timeout> parameter is > 0 this indicates the maximum number of seconds 195the subsequent HTTP transfer (sending the request and receiving a response) 196is allowed to take. 197A value of 0 enables waiting indefinitely, i.e., no timeout. 198A value < 0 indicates that the I<overall_timeout> parameter value given 199when opening the HTTP transfer will be used instead. 200If I<keep_alive> is 0 the connection is not kept open 201after receiving a response, which is the default behavior for HTTP 1.0. 202If the value is 1 or 2 then a persistent connection is requested. 203If the value is 2 then a persistent connection is required, 204i.e., an error occurs in case the server does not grant it. 205 206OSSL_HTTP_exchange() exchanges any form of HTTP request and response 207as specified by I<rctx>, which must include both connection and request data, 208typically set up using OSSL_HTTP_open() and OSSL_HTTP_set1_request(). 209It implements the core of the functions described below. 210If the HTTP method is GET and I<redirection_url> 211is not NULL the latter pointer is used to provide any new location that 212the server may return with HTTP code 301 (MOVED_PERMANENTLY) or 302 (FOUND). 213In this case the function returns NULL and the caller is 214responsible for deallocating the URL with L<OPENSSL_free(3)>. 215If the response header contains one or more "Content-Length" header lines and/or 216an ASN.1-encoded response is expected, which should include a total length, 217the length indications received are checked for consistency 218and for not exceeding any given maximum response length. 219If an ASN.1-encoded response is expected, the function returns on success 220the contents buffered in a memory BIO, which does not support streaming. 221Otherwise it returns directly the read BIO that holds the response contents, 222which allows a response of indefinite length and may support streaming. 223The caller is responsible for freeing the BIO pointer obtained. 224 225OSSL_HTTP_get() uses HTTP GET to obtain data from I<bio> if non-NULL, 226else from the server contained in the I<url>, and returns it as a BIO. 227It supports redirection via HTTP status code 301 or 302. It is meant for 228transfers with a single round trip, so does not support persistent connections. 229If I<bio> is non-NULL, any host and port components in the I<url> are not used 230for connecting but the hostname is used, as usual, for the C<Host> header. 231Any userinfo and fragment components in the I<url> are ignored. 232Any query component is handled as part of the path component. 233If the scheme component of the I<url> is C<https> a TLS connection is requested 234and the I<bio_update_fn>, as described for OSSL_HTTP_open(), must be provided. 235Also the remaining parameters are interpreted as described for OSSL_HTTP_open() 236and OSSL_HTTP_set1_request(), respectively. 237The caller is responsible for freeing the BIO pointer obtained. 238 239OSSL_HTTP_transfer() exchanges an HTTP request and response 240over a connection managed via I<prctx> without supporting redirection. 241It combines OSSL_HTTP_open(), OSSL_HTTP_set1_request(), OSSL_HTTP_exchange(), 242and OSSL_HTTP_close(). 243If I<prctx> is not NULL it reuses any open connection represented by a non-NULL 244I<*prctx>. It keeps the connection open if a persistent connection is requested 245or required and this was granted by the server, else it closes the connection 246and assigns NULL to I<*prctx>. 247The remaining parameters are interpreted as described for OSSL_HTTP_open() 248and OSSL_HTTP_set1_request(), respectively. 249The caller is responsible for freeing the BIO pointer obtained. 250 251OSSL_HTTP_close() closes the connection and releases I<rctx>. 252The I<ok> parameter is passed to any BIO update function 253given during setup as described above for OSSL_HTTP_open(). 254It must be 1 if no error occurred during the HTTP transfer and 0 otherwise. 255 256=head1 NOTES 257 258The names of the environment variables used by this implementation: 259C<http_proxy>, C<HTTP_PROXY>, C<https_proxy>, C<HTTPS_PROXY>, C<no_proxy>, and 260C<NO_PROXY>, have been chosen for maximal compatibility with 261other HTTP client implementations such as wget, curl, and git. 262 263When built with tracing enabled, OSSL_HTTP_transfer() and all functions using it 264may be traced using B<OSSL_TRACE_CATEGORY_HTTP>. 265See also L<OSSL_trace_enabled(3)> and L<openssl(1)/ENVIRONMENT>. 266 267=head1 RETURN VALUES 268 269OSSL_HTTP_open() returns on success a B<OSSL_HTTP_REQ_CTX>, else NULL. 270 271OSSL_HTTP_proxy_connect() and OSSL_HTTP_set1_request() 272return 1 on success, 0 on error. 273 274On success, OSSL_HTTP_exchange(), OSSL_HTTP_get(), and OSSL_HTTP_transfer() 275return a memory BIO that buffers all the data received if an ASN.1-encoded 276response is expected, otherwise a BIO that may support streaming. 277The BIO must be freed by the caller. 278On failure, they return NULL. 279Failure conditions include connection/transfer timeout, parse errors, etc. 280The caller is responsible for freeing the BIO pointer obtained. 281 282OSSL_HTTP_close() returns 0 if anything went wrong while disconnecting, else 1. 283 284=head1 SEE ALSO 285 286L<OSSL_HTTP_parse_url(3)>, L<BIO_new_connect(3)>, 287L<ASN1_item_i2d_mem_bio(3)>, L<ASN1_item_d2i_bio(3)>, 288L<OSSL_HTTP_is_alive(3)>, 289L<OSSL_trace_enabled(3)> 290 291=head1 HISTORY 292 293All the functions described here were added in OpenSSL 3.0. 294 295=head1 COPYRIGHT 296 297Copyright 2019-2023 The OpenSSL Project Authors. All Rights Reserved. 298 299Licensed under the Apache License 2.0 (the "License"). You may not use 300this file except in compliance with the License. You can obtain a copy 301in the file LICENSE in the source distribution or at 302L<https://www.openssl.org/source/license.html>. 303 304=cut 305
