xref: /curl/packages/OS400/README.OS400 (revision ad8c9a58) 
1
2Implementation notes:
3
4  This is a true OS/400 ILE implementation, not a PASE implementation (for
5PASE, use AIX implementation).
6
7  The biggest problem with OS/400 is EBCDIC. Libcurl implements an internal
8conversion mechanism, but it has been designed for computers that have a
9single native character set. OS/400 default native character set varies
10depending on the country for which it has been localized. And more, a job
11may dynamically alter its "native" character set.
12  Several characters that do not have fixed code in EBCDIC variants are
13used in libcurl strings. As a consequence, using the existing conversion
14mechanism would have lead in a localized binary library - not portable across
15countries.
16  For this reason, and because libcurl was originally designed for ASCII based
17operating systems, the current OS/400 implementation uses ASCII as internal
18character set. This has been accomplished using the QADRT library and
19include files, a C and system procedures ASCII wrapper library. See IBM QADRT
20description for more information.
21  This then results in libcurl being an ASCII library: any function string
22argument is taken/returned in ASCII and a C/C++ calling program built around
23QADRT may use libcurl functions as on any other platform.
24  QADRT does not define ASCII wrappers for all C/system procedures: the
25OS/400 configuration header file and an additional module (os400sys.c) define
26some more of them, that are used by libcurl and that QADRT left out.
27  To support all the different variants of EBCDIC, non-standard wrapper
28procedures have been added to libcurl on OS/400: they provide an additional
29CCSID (numeric Coded Character Set ID specific to OS/400) parameter for each
30string argument. Callback procedures arguments giving access to strings are
31NOT converted, so text gathered this way is (probably !) ASCII.
32
33  Another OS/400 problem comes from the fact that the last fixed argument of a
34vararg procedure may not be of type char, unsigned char, short or unsigned
35short. Enums that are internally implemented by the C compiler as one of these
36types are also forbidden. Libcurl uses enums as vararg procedure tagfields...
37Happily, there is a pragma forcing enums to type "int". The original libcurl
38header files are thus altered during build process to use this pragma, in
39order to force libcurl enums of being type int (the pragma disposition in use
40before inclusion is restored before resuming the including unit compilation).
41
42  Non-standard EBCDIC wrapper prototypes are defined in an additional header
43file: ccsidcurl.h. These should be self-explanatory to an OS/400-aware
44designer. CCSID 0 can be used to select the current job's CCSID.
45  Wrapper procedures with variable arguments are described below:
46
47_ curl_easy_setopt_ccsid()
48  Variable arguments are a string pointer and a CCSID (unsigned int) for
49options:
50        CURLOPT_ABSTRACT_UNIX_SOCKET
51        CURLOPT_ACCEPT_ENCODING
52        CURLOPT_ALTSVC
53        CURLOPT_AWS_SIGV4
54        CURLOPT_CAINFO
55        CURLOPT_CAPATH
56        CURLOPT_COOKIE
57        CURLOPT_COOKIEFILE
58        CURLOPT_COOKIEJAR
59        CURLOPT_COOKIELIST
60        CURLOPT_CRLFILE
61        CURLOPT_CUSTOMREQUEST
62        CURLOPT_DEFAULT_PROTOCOL
63        CURLOPT_DNS_INTERFACE
64        CURLOPT_DNS_LOCAL_IP4
65        CURLOPT_DNS_LOCAL_IP6
66        CURLOPT_DNS_SERVERS
67        CURLOPT_DOH_URL
68        CURLOPT_EGDSOCKET
69        CURLOPT_FTPPORT
70        CURLOPT_FTP_ACCOUNT
71        CURLOPT_FTP_ALTERNATIVE_TO_USER
72        CURLOPT_HAPROXY_CLIENT_IP
73        CURLOPT_HSTS
74        CURLOPT_INTERFACE
75        CURLOPT_ISSUERCERT
76        CURLOPT_KEYPASSWD
77        CURLOPT_KRBLEVEL
78        CURLOPT_LOGIN_OPTIONS
79        CURLOPT_MAIL_AUTH
80        CURLOPT_MAIL_FROM
81        CURLOPT_NETRC_FILE
82        CURLOPT_NOPROXY
83        CURLOPT_PASSWORD
84        CURLOPT_PINNEDPUBLICKEY
85        CURLOPT_PRE_PROXY
86        CURLOPT_PROTOCOLS_STR
87        CURLOPT_PROXY
88        CURLOPT_PROXYPASSWORD
89        CURLOPT_PROXYUSERNAME
90        CURLOPT_PROXYUSERPWD
91        CURLOPT_PROXY_CAINFO
92        CURLOPT_PROXY_CAPATH
93        CURLOPT_PROXY_CRLFILE
94        CURLOPT_PROXY_ISSUERCERT
95        CURLOPT_PROXY_KEYPASSWD
96        CURLOPT_PROXY_PINNEDPUBLICKEY
97        CURLOPT_PROXY_SERVICE_NAME
98        CURLOPT_PROXY_SSLCERT
99        CURLOPT_PROXY_SSLCERTTYPE
100        CURLOPT_PROXY_SSLKEY
101        CURLOPT_PROXY_SSLKEYTYPE
102        CURLOPT_PROXY_SSL_CIPHER_LIST
103        CURLOPT_PROXY_TLS13_CIPHERS
104        CURLOPT_PROXY_TLSAUTH_PASSWORD
105        CURLOPT_PROXY_TLSAUTH_TYPE
106        CURLOPT_PROXY_TLSAUTH_USERNAME
107        CURLOPT_RANDOM_FILE
108        CURLOPT_RANGE
109        CURLOPT_REDIR_PROTOCOLS_STR
110        CURLOPT_REFERER
111        CURLOPT_REQUEST_TARGET
112        CURLOPT_RTSP_SESSION_ID
113        CURLOPT_RTSP_STREAM_URI
114        CURLOPT_RTSP_TRANSPORT
115        CURLOPT_SASL_AUTHZID
116        CURLOPT_SERVICE_NAME
117        CURLOPT_SOCKS5_GSSAPI_SERVICE
118        CURLOPT_SSH_HOST_PUBLIC_KEY_MD5
119        CURLOPT_SSH_HOST_PUBLIC_KEY_SHA256
120        CURLOPT_SSH_KNOWNHOSTS
121        CURLOPT_SSH_PRIVATE_KEYFILE
122        CURLOPT_SSH_PUBLIC_KEYFILE
123        CURLOPT_SSLCERT
124        CURLOPT_SSLCERTTYPE
125        CURLOPT_SSLENGINE
126        CURLOPT_SSLKEY
127        CURLOPT_SSLKEYTYPE
128        CURLOPT_SSL_CIPHER_LIST
129        CURLOPT_SSL_EC_CURVES
130        CURLOPT_TLS13_CIPHERS
131        CURLOPT_TLSAUTH_PASSWORD
132        CURLOPT_TLSAUTH_TYPE
133        CURLOPT_TLSAUTH_USERNAME
134        CURLOPT_UNIX_SOCKET_PATH
135        CURLOPT_URL
136        CURLOPT_USERAGENT
137        CURLOPT_USERNAME
138        CURLOPT_USERPWD
139        CURLOPT_XOAUTH2_BEARER
140  All blob options are also supported.
141  In all other cases, it ignores the ccsid parameter and behaves as
142curl_easy_setopt().
143  Note that CURLOPT_ERRORBUFFER is not in the list above, since it gives the
144address of an (empty) character buffer, not the address of a string.
145CURLOPT_POSTFIELDS stores the address of static binary data (of type void *)
146and thus is not converted. If CURLOPT_COPYPOSTFIELDS is issued after
147CURLOPT_POSTFIELDSIZE != -1, the data size is adjusted according to the
148CCSID conversion result length.
149
150_ curl_formadd_ccsid()
151  In the variable argument list, string pointers should be followed by a (long)
152CCSID for the following options:
153        CURLFORM_BUFFER
154        CURLFORM_CONTENTTYPE
155        CURLFORM_COPYCONTENTS
156        CURLFORM_COPYNAME
157        CURLFORM_FILE
158        CURLFORM_FILECONTENT
159        CURLFORM_FILENAME
160        CURLFORM_PTRNAME
161  If taken from an argument array, an additional array entry must follow each
162entry containing one of the above option. This additional entry holds the CCSID
163in its value field, and the option field is meaningless.
164  It is not possible to have a string pointer and its CCSID across a function
165parameter/array boundary.
166  Please note that CURLFORM_PTRCONTENTS and CURLFORM_BUFFERPTR are considered
167unconvertible strings and thus are NOT followed by a CCSID.
168
169_ curl_easy_getinfo_ccsid()
170  The following options are followed by a 'char * *' and a CCSID. Unlike
171curl_easy_getinfo(), the value returned in the pointer should be released with
172curl_free() after use:
173        CURLINFO_CONTENT_TYPE
174        CURLINFO_EFFECTIVE_URL
175        CURLINFO_FTP_ENTRY_PATH
176        CURLINFO_LOCAL_IP
177        CURLINFO_PRIMARY_IP
178        CURLINFO_REDIRECT_URL
179        CURLINFO_REFERER
180        CURLINFO_RTSP_SESSION_ID
181        CURLINFO_SCHEME
182  Likewise, the following options are followed by a struct curl_slist * * and a
183CCSID.
184        CURLINFO_COOKIELIST
185        CURLINFO_SSL_ENGINES
186Lists returned should be released with curl_slist_free_all() after use.
187  Option CURLINFO_CERTINFO is followed by a struct curl_certinfo * * and a
188CCSID. Returned structures should be freed with curl_certinfo_free_all()
189after use.
190  Other options are processed like in curl_easy_getinfo().
191
192_ curl_easy_strerror_ccsid(), curl_multi_strerror_ccsid(),
193curl_share_strerror_ccsid() and curl_url_strerror_ccsid() work as their
194non-ccsid version and return a string encoded in the additional ccsid
195parameter. These strings belong to libcurl and may not be freed by the caller.
196A subsequent call to the same procedure in the same thread invalidates the
197previous result.
198
199_ curl_pushheader_bynum_cssid() and curl_pushheader_byname_ccsid()
200  Although the prototypes are self-explanatory, the returned string pointer
201should be released with curl_free() after use, as opposite to the non-ccsid
202versions of these procedures.
203  Please note that HTTP2 is not (yet) implemented on OS/400, thus these
204functions will always return NULL.
205
206_ curl_easy_option_by_name_ccsid() returns a pointer to an untranslated option
207metadata structure. As each curl_easyoption structure holds the option name in
208ASCII, the curl_easy_option_get_name_ccsid() function allows getting it in any
209supported ccsid. However the caller should release the returned pointer with
210curl_free() after use.
211
212_ curl_easy_header_ccsid() works as its non-CCSID counterpart but requires an
213additional ccsid parameter specifying the name parameter encoding. The output
214hout parameter is kept in libcurl's encoding and should not be altered.
215
216_ curl_from_ccsid() and curl_to_ccsid() are string encoding conversion
217functions between ASCII (latin1) and the given CCSID. The first parameter is
218the source string, the second is the CCSID and the returned value is a pointer
219to the dynamically allocated string. These functions do not impact on Curl's
220behavior and are only provided for user convenience. After use, returned values
221must be released with curl_free().
222
223
224  Standard compilation environment does support neither autotools nor make;
225in fact, very few common utilities are available. As a consequence, the
226config-os400.h has been coded manually and the compilation scripts are
227a set of shell scripts stored in subdirectory packages/OS400.
228
229  The test environment is currently not supported on OS/400.
230
231
232Protocols currently implemented on OS/400:
233_ DICT
234_ FILE
235_ FTP
236_ FTPS
237_ FTP with secure transmission
238_ GOPHER
239_ HTTP
240_ HTTPS
241_ IMAP
242_ IMAPS
243_ IMAP with secure transmission
244_ LDAP
245_ POP3
246_ POP3S
247_ POP3 with secure transmission
248_ RTSP
249_ SCP if libssh2 is enabled
250_ SFTP if libssh2 is enabled
251_ SMTP
252_ SMTPS
253_ SMTP with secure transmission
254_ TELNET
255_ TFTP
256
257
258
259Compiling on OS/400:
260
261  These instructions targets people who knows about OS/400, compiling, IFS and
262archive extraction. Do not ask questions about these subjects if you're not
263familiar with.
264
265_ As a prerequisite, QADRT development environment must be installed.
266  For more information on downloading and installing the QADRT development kit,
267  please see https://www.ibm.com/support/pages/node/6258183
268_ If data compression has to be supported, ZLIB development environment must
269  be installed.
270_ Likewise, if SCP and SFTP protocols have to be compiled in, LIBSSH2
271  developent environment must be installed.
272_ Install the curl source directory in IFS. Do NOT install it in the
273  installation target directory (which defaults to /curl).
274_ Enter Qshell (QSH, not PASE)
275_ Change current directory to the curl installation directory
276_ Change current directory to ./packages/OS400
277- If you want to change the default configuration parameters like debug info
278  generation, optimization level, listing option, target library, ZLIB/LIBSSH2
279  availability and location, etc., copy file config400.default to
280  config400.override and edit the latter. Do not edit the original default file
281  as it might be overwritten by a subsequent source installation.
282_ Copy any file in the current directory to makelog (i.e.:
283  cp initscript.sh makelog): this is intended to create the makelog file with
284  an ASCII CCSID!
285_ Enter the command "sh makefile.sh > makelog 2>&1"
286_ Examine the makelog file to check for compilation errors. CZM0383 warnings on
287  C or system standard API come from QADRT inlining and can safely be ignored.
288
289  Without configuration parameters override, this will produce the following
290OS/400 objects:
291_ Library CURL. All other objects will be stored in this library.
292_ Modules for all libcurl units.
293_ Binding directory CURL_A, to be used at calling program link time for
294  statically binding the modules (specify BNDSRVPGM(QADRTTS QGLDCLNT QGLDBRDR)
295  when creating a program using CURL_A).
296_ Service program CURL.<soname>, where <soname> is extracted from the
297  lib/Makefile.am VERSION variable. To be used at calling program run-time
298  when this program has dynamically bound curl at link time.
299_ Binding directory CURL. To be used to dynamically bind libcurl when linking a
300  calling program.
301- CLI tool bound program CURL.
302- CLI command CURL.
303_ Source file H. It contains all the include members needed to compile a C/C++
304  module using libcurl, and an ILE/RPG /copy member for support in this
305  language.
306_ Standard C/C++ libcurl include members in file H.
307_ CCSIDCURL member in file H. This defines the non-standard EBCDIC wrappers for
308  C and C++.
309_ CURL.INC member in file H. This defines everything needed by an ILE/RPG
310  program using libcurl.
311_ IFS directory /curl/include/curl containing the C header files for IFS source
312  C/C++ compilation and curl.inc.rpgle for IFS source ILE/RPG compilation.
313- IFS link /curl/bin/curl to CLI tool program.
314
315
316Special programming consideration:
317
318QADRT being used, the following points must be considered:
319_ If static binding is used, service program QADRTTS must be linked too.
320_ The EBCDIC CCSID used by QADRT is 37 by default, NOT THE JOB'S CCSID. If
321  another EBCDIC CCSID is required, it must be set via a locale through a call
322  to setlocale_a (QADRT's setlocale() ASCII wrapper) with category LC_ALL or
323  LC_CTYPE, or by setting environment variable QADRT_ENV_LOCALE to the locale
324  object path before executing the program.
325_ Do not use original source include files unless you know what you are doing.
326  Use the installed members instead (in /QSYS.LIB/CURL.LIB/H.FILE and
327  /curl/include/curl).
328
329
330
331ILE/RPG support:
332
333  Since most of the ILE OS/400 programmers use ILE/RPG exclusively, a
334definition /INCLUDE member is provided for this language. To include all
335libcurl definitions in an ILE/RPG module, line
336
337     h bnddir('CURL/CURL')
338
339must figure in the program header, and line
340
341     d/include curl/h,curl.inc
342
343in the global data section of the module's source code.
344
345  No vararg procedure support exists in ILE/RPG: for this reason, the following
346considerations apply:
347_ Procedures curl_easy_setopt_long(), curl_easy_setopt_object(),
348  curl_easy_setopt_function(), curl_easy_setopt_offset() and
349  curl_easy_setopt_blob() are all alias prototypes to curl_easy_setopt(), but
350  with different parameter lists.
351_ Procedures curl_easy_getinfo_string(), curl_easy_getinfo_long(),
352  curl_easy_getinfo_double(), curl_easy_getinfo_slist(),
353  curl_easy_getinfo_ptr(), curl_easy_getinfo_socket() and
354  curl_easy_getinfo_off_t() are all alias prototypes to curl_easy_getinfo(),
355  but with different parameter lists.
356_ Procedures curl_multi_setopt_long(), curl_multi_setopt_object(),
357  curl_multi_setopt_function() and curl_multi_setopt_offset() are all alias
358  prototypes to curl_multi_setopt(), but with different parameter lists.
359_ Procedures curl_share_setopt_int(), curl_share_setopt_ptr() and
360  curl_share_setopt_proc() are all alias prototypes to curl_share_setopt,
361  but with different parameter lists.
362_ Procedure curl_easy_setopt_blob_ccsid() is an alias of
363  curl_easy_setopt_ccsid() supporting blob encoding conversion.
364_ The prototype of procedure curl_formadd() allows specifying a pointer option
365  and the CURLFORM_END option. This makes possible to use an option array
366  without any additional definition. If some specific incompatible argument
367  list is used in the ILE/RPG program, the latter must define a specialised
368  alias. The same applies to curl_formadd_ccsid() too.
369_ Since V7R4M0, procedure overloading is used to emulate limited "vararg-like"
370  definitions of curl_easy_setopt(), curl_multi_setopt(), curl_share_setopt()
371  and curl_easy_getinfo(). Blob and CCSID alternatives are NOT included in
372  overloading.
373
374  Since RPG cannot cast a long to a pointer, procedure curl_form_long_value()
375is provided for that purpose: this allows storing a long value in the
376curl_forms array. Please note the form API is deprecated and the MIME API
377should be used instead.
378
379
380CLI tool:
381
382  The build system provides it as a bound program, an IFS link to it and a
383simple CL command. The latter however is not able to provide a different
384parameter for each option since there are too many of those; instead,
385parameters are entered in a single field subject to quoting and escaping, in
386the same form as expected by the standard CLI program.
387  Care must be taken about the program output encoding: by default, it is sent
388to the standard output and is thus subject to transcoding. It is therefore
389recommended to use option "--output" to redirect output to a specific IFS file.
390Similar problems may occur about the standard input encoding.
391