1--- 2c: Copyright (C) Daniel Stenberg, <daniel@haxx.se>, et al. 3SPDX-License-Identifier: curl 4Title: libcurl-multi 5Section: 3 6Source: libcurl 7See-also: 8 - libcurl (3) 9 - libcurl-easy (3) 10 - libcurl-errors (3) 11Protocol: 12 - All 13--- 14 15# NAME 16 17libcurl-multi - how to use the multi interface 18 19# DESCRIPTION 20 21This is an overview on how to use the libcurl multi interface in your C 22programs. There are specific man pages for each function mentioned in 23here. There is also the libcurl-tutorial(3) man page for a complete 24tutorial to programming with libcurl and the libcurl-easy(3) man page 25for an overview of the libcurl easy interface. 26 27All functions in the multi interface are prefixed with curl_multi. 28 29# OBJECTIVES 30 31The multi interface offers several abilities that the easy interface does not. 32They are mainly: 33 341. Enable a "pull" interface. The application that uses libcurl decides where 35and when to ask libcurl to get/send data. 36 372. Enable multiple simultaneous transfers in the same thread without making it 38complicated for the application. 39 403. Enable the application to wait for action on its own file descriptors and 41curl's file descriptors simultaneously. 42 434. Enable event-based handling and scaling transfers up to and beyond 44thousands of parallel connections. 45 46# ONE MULTI HANDLE MANY EASY HANDLES 47 48To use the multi interface, you must first create a 'multi handle' with 49curl_multi_init(3). This handle is then used as input to all further 50curl_multi_* functions. 51 52With a multi handle and the multi interface you can do several simultaneous 53transfers in parallel. Each single transfer is built up around an easy 54handle. You create all the easy handles you need, and setup the appropriate 55options for each easy handle using curl_easy_setopt(3). 56 57There are two flavors of the multi interface, the select() oriented one and 58the event based one we call multi_socket. You benefit from reading through the 59description of both versions to fully understand how they work and 60differentiate. We start out with the select() oriented version. 61 62When an easy handle is setup and ready for transfer, then instead of using 63curl_easy_perform(3) like when using the easy interface for transfers, 64you should add the easy handle to the multi handle with 65curl_multi_add_handle(3). You can add more easy handles to a multi 66handle at any point, even if other transfers are already running. 67 68Should you change your mind, the easy handle is again removed from the multi 69stack using curl_multi_remove_handle(3). Once removed from the multi 70handle, you can again use other easy interface functions like 71curl_easy_perform(3) on the handle or whatever you think is 72necessary. You can remove handles at any point during transfers. 73 74Adding the easy handle to the multi handle does not start the transfer. 75Remember that one of the main ideas with this interface is to let your 76application drive. You drive the transfers by invoking 77curl_multi_perform(3). libcurl then transfers data if there is anything 78available to transfer. It uses the callbacks and everything else you have 79setup in the individual easy handles. It transfers data on all current 80transfers in the multi stack that are ready to transfer anything. It may be 81all, it may be none. When there is nothing more to do for now, it returns back 82to the calling application. 83 84Your application extracts info from libcurl about when it would like to get 85invoked to transfer data or do other work. The most convenient way is to use 86curl_multi_poll(3) that helps you wait until the application should call 87libcurl again. The older API to accomplish the same thing is 88curl_multi_fdset(3) that extracts *fd_sets* from libcurl to use in 89select() or poll() calls in order to get to know when the transfers in the 90multi stack might need attention. Both these APIs allow for your program to 91wait for input on your own private file descriptors at the same time. 92curl_multi_timeout(3) also helps you with providing a suitable timeout 93period for your select() calls. 94 95curl_multi_perform(3) stores the number of still running transfers in 96one of its input arguments, and by reading that you can figure out when all 97the transfers in the multi handles are done. 'done' does not mean 98successful. One or more of the transfers may have failed. 99 100To get information about completed transfers, to figure out success or not and 101similar, curl_multi_info_read(3) should be called. It can return a 102message about a current or previous transfer. Repeated invokes of the function 103get more messages until the message queue is empty. The information you 104receive there includes an easy handle pointer which you may use to identify 105which easy handle the information regards. 106 107When a single transfer is completed, the easy handle is still left added to 108the multi stack. You need to first remove the easy handle with 109curl_multi_remove_handle(3) and then close it with 110curl_easy_cleanup(3), or possibly set new options to it and add it again 111with curl_multi_add_handle(3) to start another transfer. 112 113When all transfers in the multi stack are done, close the multi handle with 114curl_multi_cleanup(3). Be careful and please note that you **MUST** 115invoke separate curl_easy_cleanup(3) calls for every single easy handle 116to clean them up properly. 117 118If you want to reuse an easy handle that was added to the multi handle for 119transfer, you must first remove it from the multi stack and then re-add it 120again (possibly after having altered some options at your own choice). 121 122# MULTI_SOCKET 123 124curl_multi_socket_action(3) function offers a way for applications to 125not only avoid being forced to use select(), but it also offers a much more 126high-performance API that makes a significant difference for applications 127using large numbers of simultaneous connections. 128 129curl_multi_socket_action(3) is then used instead of 130curl_multi_perform(3). 131 132When using this API, you add easy handles to the multi handle just as with the 133normal multi interface. Then you also set two callbacks with the 134CURLMOPT_SOCKETFUNCTION(3) and CURLMOPT_TIMERFUNCTION(3) options 135to curl_multi_setopt(3). They are two callback functions that libcurl 136calls with information about what sockets to wait for, and for what activity, 137and what the current timeout time is - if that expires libcurl should be 138notified. 139 140The multi_socket API is designed to inform your application about which 141sockets libcurl is currently using and for what activities (read and/or write) 142on those sockets your application is expected to wait for. 143 144Your application must make sure to receive all sockets informed about in the 145CURLMOPT_SOCKETFUNCTION(3) callback and make sure it reacts on the given 146activity on them. When a socket has the given activity, you call 147curl_multi_socket_action(3) specifying which socket and action there 148are. 149 150The CURLMOPT_TIMERFUNCTION(3) callback is called to set a timeout. When 151that timeout expires, your application should call the 152curl_multi_socket_action(3) function saying it was due to a timeout. 153 154This API is typically used with an event-driven underlying functionality (like 155libevent, libev, kqueue, epoll or similar) with which the application 156"subscribes" on socket changes. This allows applications and libcurl to much 157better scale upward and beyond thousands of simultaneous transfers without 158losing performance. 159 160When you have added your initial set of handles, you call 161curl_multi_socket_action(3) with CURL_SOCKET_TIMEOUT set in the 162*sockfd* argument, and you get callbacks invoked that set you up and you 163then continue to call curl_multi_socket_action(3) accordingly when you 164get activity on the sockets you have been asked to wait on, or if the timeout 165timer expires. 166 167You can poll curl_multi_info_read(3) to see if any transfer has 168completed, as it then has a message saying so. 169 170# BLOCKING 171 172A few areas in the code are still using blocking code, even when used from the 173multi interface. While we certainly want and intend for these to get fixed in 174the future, you should be aware of the following current restrictions: 175 176~~~c 177 - Name resolves unless the c-ares or threaded-resolver backends are used 178 - file:// transfers 179 - TELNET transfers 180~~~ 181