23 This document attempts to describe the general principles and some basic
24 approaches to consider when programming with libcurl. The text focuses on the
28 This document refers to 'the user' as the person writing the source code that
30 generally referred to as 'the program' is the collected source code that you
35 refer to their respective man pages.
39 There are many different ways to build C programs. This chapter assumes a Unix
41 this to get general information that may apply to your environment as well.
45 Your compiler needs to know where the libcurl headers are located. Therefore
46 you must set your compiler's include path to point to the directory where you
47 installed them. The 'curl-config'[3] tool can be used to get this information:
54 When having compiled the program, you need to link your object files to create
55 a single executable. For that to succeed, you need to link with libcurl and
58 command line. To figure out which flags to use, once again the 'curl-config'
59 tool comes to the rescue:
77 If SSL is supported, the keyword *SSL* is written to stdout, possibly together
85 When you write your configure script to detect libcurl and setup variables
87 area. See docs/libcurl/libcurl.m4 file - it includes docs on how to use it.
91 The people behind libcurl have put a considerable effort to make libcurl work
95 are only a few minor details that differ. If you just make sure to write your
102 means it should be done exactly once, no matter how many times you intend to
107 and it takes one parameter which is a bit pattern that tells libcurl what to
118 or of another library in use does it, you should not tell libcurl to do this
125 application. This only needs to be done once for each application so if your
136 performs the reversed operations to cleanup the resources the
139 Repeated calls to curl_global_init(3) and curl_global_cleanup(3)
144 It is considered best-practice to determine libcurl features at runtime rather
158 interface is detailed in a separate chapter further down. You still need to
165 need one handle for each easy session you want to perform. Basically, you
166 should use one handle for every thread you plan to use for transferring. You
173 It returns an easy handle. Using that you proceed to the next step: setting
180 set again to something different. They are sticky. Multiple requests using the
183 If you at any point would like to blank all previously set options for a
188 Many of the options you set in libcurl are "strings", pointers to data
191 to be kept around in your application after being set[4].
193 One of the most basic properties to set in the handle is the URL. You set your
194 preferred URL to transfer with CURLOPT_URL(3) in a manner similar to:
200 Let's assume for a while that you want to receive data as the URL identifies a
201 remote resource you want to get here. Since you write a sort of application
202 that needs this transfer, I assume that you would like to get the data passed
203 to you directly instead of simply getting it passed to stdout. So, you write
208 You tell libcurl to pass all data to this function by issuing a function
209 similar to this:
224 outputs the received data to stdout. You can have the default callback write
225 the data to a different file handle by passing a 'FILE *' to a file opened for
228 Now, we need to take a step back and take a deep breath. Here is one of those
230 libcurl is not able to operate on file handles opened by the
233 to make your program run fine virtually everywhere.
242 There are of course many more options you can set, and we get back to a few of
243 them later. Let's instead continue to the actual transfer:
249 curl_easy_perform(3) connects to the remote site, does the necessary commands
254 care of". If that is not the same amount of bytes that was passed to it,
259 you, you can use the CURLOPT_ERRORBUFFER(3) to point libcurl to a buffer of
262 If you then want to transfer another file, the handle is ready to be used
264 if you intend to make another transfer. libcurl then attempts to reuse a
270 complication for you. Given simply the URL to a file, libcurl takes care of
271 all the details needed to get the file moved from one machine to another.
275 libcurl is thread safe but there are a few exceptions. Refer to
286 CURLOPT_VERBOSE(3) option to 1. it causes the library to spew out the
289 adding the headers in the received output to study is also a clever way to get
293 Of course, there are bugs left. We need to know about them to be able to fix
304 and if you are trying to do funny things, you might understand libcurl and how
305 to use it better if you study the appropriate RFC documents at least briefly.
307 # Upload Data to a Remote Site
309 libcurl tries to keep a protocol independent approach to most transfers, thus
310 uploading to a remote FTP site is similar to uploading data to an HTTP server
314 one. Then you set the URL to operate on just like before. This is the remote
317 Since we write an application, we most likely want libcurl to get the upload
319 custom pointer libcurl passes to our read callback. The read callback should
320 have a prototype similar to:
324 Where *bufptr* is the pointer to a buffer we fill in with data to upload
326 amount of data we can return to libcurl in this call. The *userp* pointer
327 is the custom pointer we set to point to a struct of ours to pass private data
334 Tell libcurl that we want to upload:
349 callback to get the data to upload. The program should return as much data as
350 possible in every invoke, as that is likely to make the upload perform as fast
357 to be able to download or upload the data of your choice. libcurl offers
358 several ways to specify them.
369 libcurl also provides options to set various passwords. The username and
371 CURLOPT_USERPWD(3) option. The argument passed to libcurl should be a
372 char * to a string in the format "user:password". In a manner like this:
379 users who need to authenticate themselves to a proxy they use. libcurl offers
381 to the CURLOPT_USERPWD(3) option like this:
392 the password in plain text. libcurl has the ability to use this file to figure
393 out what set of username and password to use for a particular host. As an
394 extension to the normal functionality, libcurl also supports this file for
411 at least you could leave it out and have libcurl attempt to do its job
415 To pass the known private key password to libcurl:
422 The previous chapter showed how to set username and password for getting URLs
424 different ways a client can provide those credentials to the server and you
429 At the time of this writing, libcurl can be built to use: Basic, Digest, NTLM,
430 Negotiate (SPNEGO). You can tell libcurl which one to use with
438 When you send authentication to a proxy, you can also set authentication type
445 Both these options allow you to set multiple types (by ORing them together),
446 to make libcurl pick the most secure one out of the types the server/proxy
447 claims to support. This method does however add a round-trip since libcurl
455 specific types) which allows libcurl to use whatever method it wants.
462 We get many questions regarding how to issue HTTP POSTs with libcurl the
467 pages using the \<form\> tag uses. We provide a pointer to the data and tell
468 libcurl to post it all to the remote site:
479 CURLOPT_POSTFIELDS(3), this automatically switches the handle to use
482 What if you want to post binary data that also requires you to set the
484 able to do strlen() on the data to figure out the size, so therefore we must
487 that list to libcurl.
509 formposts were introduced as a better way to post (possibly large) binary data
515 yourself and provide to libcurl.
520 to a multi-part body using curl_mime_addpart(3).
568 ought to be converted to the MIME API. It is however described here as an
569 aid to conversion.
571 Using *curl_formadd*, you add parts to the form. When you are done adding
601 application to handicraft this formpost even more, libcurl allows you to
602 supply your own set of custom headers to such an individual form part. You can
603 of course supply headers to as many parts as you like, but this little example
604 shows how you set headers to one specific part when you add that to the post
624 changed even if you do call curl_easy_perform(3), you may need to tell
625 curl to go back to a plain GET request if you intend to do one as your next
626 request. You force an easy handle to go back to GET by using the
631 Just setting CURLOPT_POSTFIELDS(3) to "" or NULL does *not* stop libcurl
632 from doing a POST. It just makes it POST without any data to send!
634 # Converting from deprecated form API to MIME API
636 Four rules have to be respected in building the multi-part:
640 - The multi-part is always created by a call to curl_mime_init(handle).
642 - Each part is created by a call to curl_mime_addpart(multipart).
644 - When complete, the multi-part must be bound to the easy handle using
647 Here are some example of *curl_formadd* calls to MIME API sequences:
664 Setting the last curl_mime_headers(3) argument to TRUE would have caused
665 the headers to be automatically released upon destroyed the multi-part, thus
666 saving a clean-up call to curl_slist_free_all(3).
682 not supported by curl_mime_filename(3): to read an open file, use a callback
704 translated to two distinct parts with the same name.
763 therefore necessary to clear it for *CURLFORM_FILECONTENT* emulation.
772 CURLOPT_NOPROGRESS(3) to zero. This option is set to 1 by default.
775 instead is interesting is the ability to specify a progress callback. The
776 function pointer you pass to libcurl is then called on irregular intervals
780 to a function that matches this prototype:
791 the 'clientp' is the pointer you pass to libcurl with
796 There is basically only one thing to keep in mind when using C++ instead of C
815 What "proxy" means according to Merriam-Webster: "a person authorized to act
820 access to employees through their proxies. Network clients or user-agents ask
825 asks the proxy for it instead of trying to connect to the actual remote host
833 HTTP URL is passed to the HTTP proxy to deliver back to libcurl. This happens
834 transparently, and an application may not need to know. I say "may", because
835 at times it is important to understand that all operations over an HTTP proxy
841 To tell libcurl to use a proxy at a given port number:
846 pass that information similar to this:
850 If you want to, you can specify the hostname only in the
855 it defaults to assuming an HTTP proxy):
862 libcurl automatically checks and uses a set of environment variables to know
863 what proxies to use for certain protocols. The names of the variables are
866 proxy to use when the input URL is HTTP. Following the same rule, the variable
869 proxies to be used.
875 number is used and that is most likely not the one you would like it to be.
883 variables, set the proxy name to "" - an empty string - with
888 SSL is for secure point-to-point connections. This involves strong encryption
889 and similar things, which effectively makes it impossible for a proxy to
891 discussed. Instead, the only way to have SSL work over an HTTP proxy is to ask
892 the proxy to tunnel everything through without being able to check or fiddle
896 proxy for a straight connection to the target host on a specified port. This
897 is made with the HTTP request CONNECT. ("please dear proxy, connect me to that
903 organizations prevent this kind of tunneling to other destination port numbers
908 As explained above, tunneling is required for SSL to work and often even
909 restricted to the operation intended for SSL; HTTPS.
911 This is however not the only time proxy-tunneling might offer benefits to
914 As tunneling opens a direct connection from your application to the remote
915 machine, it suddenly also re-introduces the ability to do non-HTTP
922 Tell libcurl to use proxy tunneling like this:
926 In fact, there might even be times when you want to do plain HTTP operations
927 using a tunnel like this, as it then enables you to operate on the remote
928 server instead of asking the proxy to do so. libcurl does not stand in the way
935 requested URL as input, returns information to the browser on how to connect
936 to the URL. The returned information might be "DIRECT" (which means no proxy
937 should be used), "PROXY host:port" (to tell the browser where the proxy for
938 this particular URL is) or "SOCKS host:port" (to direct the browser to a SOCKS
941 libcurl has no means to interpret or evaluate JavaScript and thus it does not
946 to another language and execute that.
953 - Ask your admins to stop this, for a static proxy setup or similar.
955 # Persistence Is The Way to Happiness
958 the way to go.
961 connection alive and open. A subsequent request using the same easy handle to
962 the same host might just be able to use the already open connection! This
965 Even if the connection is dropped, all connections involving SSL to the same
971 without permission to login again like on many FTP servers only allowing N
972 persons to be logged in at the same time.
974 libcurl caches DNS name resolving results, to make lookups of a previously
980 Each easy handle attempts to keep the last few connections alive for a while
981 in case they are to be used again. You can set the size of this "cache" with
986 To force your upcoming request to not use an already existing connection, you
987 can do that by setting CURLOPT_FRESH_CONNECT(3) to 1. In a similar
988 spirit, you can also forbid the upcoming request to be "lying" around and
990 CURLOPT_FORBID_REUSE(3) to 1.
994 When you use libcurl to do HTTP requests, it passes along a series of headers
995 automatically. It might be good for you to know and understand these. You can
1001 the name of the server we want to talk to. This includes the port number if
1010 When doing POST requests, libcurl sets this header to "100-continue" to ask
1022 programming you may need to change the traditional HTTP (or FTP or...)
1023 manners. You may need to change words, headers or various data.
1031 is there for you. It is simple to use:
1040 keyword if you want to. You are the boss.
1044 HTTP-like protocols pass a series of headers to the server when doing the
1045 request, and you are free to pass any amount of extra headers that you
1049 struct curl_slist *headers=NULL; /* init to NULL is important */
1063 or Host: do not contain the data you want them to contain, you can replace
1074 header from being sent. For instance, if you want to completely prevent the
1075 "Accept:" header from being sent, you can disable it with code similar to
1087 when doing a non-GET HTTP operation, libcurl switches over to "chunked"
1088 upload, even though the size of the data to upload might be known. By default,
1089 libcurl usually switches over to chunked upload automatically if the upload
1094 All HTTP requests includes the version number to tell the server which version
1097 can tell libcurl to use 1.0 instead by doing something like this:
1104 you want to make, for example, your FTP transfers to behave differently.
1106 Sending custom commands to an FTP server means that you need to send the
1110 data-connection must be left to libcurl's own judgment. Also be aware that
1111 libcurl does its best to change directory to the target directory before doing
1113 confuse libcurl and then it might not attempt to transfer the file in the
1119  headers = curl_slist_append(headers, "DELE file-to-remove");
1121  /* pass the list of custom commands to the handle */
1129 If you would instead want this operation (or chain of operations) to happen
1130 _after_ the data transfer took place the option to curl_easy_setopt(3)
1134 The custom FTP commands are issued to the server in the same order they are
1135 added to the list, and if a command gets an error code returned back from the
1137 (CURLE_QUOTE_ERROR). Note that if you use CURLOPT_QUOTE(3) to send
1141 If you set the CURLOPT_HEADER(3) to 1, you tell libcurl to get
1145 The option to enable headers or to run custom FTP commands may be useful to
1151 If you do want to list the contents of an FTP directory using your own defined
1153 one for listing directories but you are free to pass in your idea of a good
1159 the name and value to the client, and expects it to get sent back on every
1160 subsequent request to the server that matches the particular conditions set.
1164 In real-world cases, servers send new cookies to replace existing ones to
1165 update them. Server use cookies to "track" users and to keep "sessions".
1167 Cookies are sent from server to clients with the header Set-Cookie: and
1168 they are sent from clients to servers with the Cookie: header.
1170 To just send whatever cookie you want to a server, you can use
1171 CURLOPT_COOKIE(3) to set a cookie string like this:
1177 In many cases, that is not enough. You might want to dynamically save whatever
1178 cookies the remote server passes to you, and make sure those cookies are then
1181 One way to do this, is to save all headers you receive in a plain file and
1182 when you make a request, you tell libcurl to read the previous headers to
1183 figure out which cookies to use. Set the header file to read cookies from with
1191 used. Many times this is enough, and you may not have to save the cookies to
1192 disk at all. Note that the file you specify to CURLOPT_COOKIEFILE(3)
1193 does not have to exist to enable the parser, so a common way to just enable
1194 the parser and not read any cookies is to use the name of a file you know does
1206 stored in it when curl_easy_cleanup(3) is called. This enables cookies to get
1214 to haunt you. libcurl offers several different ways to customize how the
1217 libcurl can either connect to the server a second time or tell the server to
1218 connect back to it. The first option is the default and it is also what works
1220 libcurl then tells the server to open up a new port and wait for a second
1222 work it tries PASV instead. (EPSV is an extension to the original FTP spec
1226 CURLOPT_FTP_USE_EPSV(3) to zero.
1228 In some cases, you want to have the server connect back to you for the second
1231 the remote server which IP address and port number to connect to. This is made
1232 with the CURLOPT_FTPPORT(3) option. If you set it to "-", libcurl uses your
1233 system's "default IP address". If you want to use a particular IP, you can set
1234 the full IP address, a hostname to resolve to an IP address or even a local
1237 When doing the "PORT" approach, libcurl attempts to use the EPRT and the LPRT
1239 behavior by setting CURLOPT_FTP_USE_EPRT(3) to zero.
1243 In addition to support HTTP multi-part form fields, the MIME API can be used
1244 to build structured email messages and send them via SMTP or append such
1245 messages to IMAP directories.
1249 multi-part, for example to include another email message or to offer several
1250 text formats alternatives. This can be nested to any level.
1253 it as a source to the parent multi-part using function
1255 bound to its parent multi-part, a nth-level multi-part belongs to it and
1258 Email messages data is not supposed to be non-ASCII and line length is
1259 limited: fortunately, some transfer encodings are defined by the standards to
1263 If the part data you want to send is already encoded in such a scheme, do not
1313 It should be noted that appending a message to an IMAP directory requires
1314 the message size to be known prior upload. It is therefore not possible to
1322 to 1.
1324 What might be even more useful, is libcurl's ability to separate the headers
1326 different pointer to pass to the ordinary write callback by setting
1329 Or, you can set an entirely separate function to receive the headers, by using
1332 The headers are passed to the callback function one by one, and you can
1333 depend on that fact. It makes it easier for you to add custom header parsers
1349 The multi interface, on the other hand, allows your program to transfer
1350 multiple files in both directions at the same time, without forcing you to use
1353 interface allows a single-threaded application to perform the same kinds of
1365 of how to use the easy interface. The multi interface is simply a way to make
1371 multi handle with curl_multi_init(3) and add all those easy handles to
1379 now and then return control to your program. It is designed to never
1380 block. You need to keep calling the function until all transfers are
1384 file descriptors or sockets to know when to call libcurl again. This also
1385 makes it easy for you to wait and respond to actions on your own application's
1386 sockets/handles. You figure out what to select() for by using
1391 action and you then call curl_multi_perform(3) to allow libcurl to do
1392 what it wants to do. Take note that libcurl does also feature some time-out
1393 code so we advise you to never use long timeouts on select() before you call
1394 curl_multi_perform(3) again. curl_multi_timeout(3) is provided to
1401 If you want to stop the transfer of one of the easy handles in the stack, you
1402 can use curl_multi_remove_handle(3) to remove individual easy
1409 curl_multi_info_read(3) can be used to get information about completed
1410 transfers. It then returns the CURLcode for each easy transfer, to allow you
1411 to figure out success on each individual transfer.
1422 When you add easy handles to a multi handle, these easy handles automatically
1427 subsequent name resolving faster, and the connection pool that is kept to
1433 example cookies so the only way to share that is with the share interface.
1439 libcurl 7.10.3 and later have the ability to switch over to chunked
1457 This behavior was different in versions before 7.17.0, where strings had to

Last Index update Fri Nov 22 14:03:13 UTC 2024

Dedicated to & Maintained by Room-11