xref: /PHP-8.0/docs/streams.md (revision 1c94aac8)
1# An overview of the PHP streams abstraction
2
3WARNING: some prototypes in this file are out of date.
4
5## Why streams?
6
7You may have noticed a shed-load of issock parameters flying around the PHP
8code; we don't want them - they are ugly and cumbersome and force you to special
9case sockets and files every time you need to work with a "user-level" PHP file
10pointer.
11
12Streams take care of that and present the PHP extension coder with an ANSI
13stdio-alike API that looks much nicer and can be extended to support non file
14based data sources.
15
16## Using streams
17
18Streams use a `php_stream*` parameter just as ANSI stdio (fread etc.) use a
19`FILE*` parameter.
20
21The main functions are:
22
23```c
24PHPAPI size_t php_stream_read(php_stream * stream, char * buf, size_t count);
25PHPAPI size_t php_stream_write(php_stream * stream, const char * buf, size_t
26        count);
27PHPAPI size_t php_stream_printf(php_stream * stream,
28        const char * fmt, ...);
29PHPAPI int php_stream_eof(php_stream * stream);
30PHPAPI int php_stream_getc(php_stream * stream);
31PHPAPI char *php_stream_gets(php_stream * stream, char *buf, size_t maxlen);
32PHPAPI int php_stream_close(php_stream * stream);
33PHPAPI int php_stream_flush(php_stream * stream);
34PHPAPI int php_stream_seek(php_stream * stream, off_t offset, int whence);
35PHPAPI off_t php_stream_tell(php_stream * stream);
36PHPAPI int php_stream_lock(php_stream * stream, int mode);
37```
38
39These (should) behave in the same way as the ANSI stdio functions with similar
40names: fread, fwrite, fprintf, feof, fgetc, fgets, fclose, fflush, fseek, ftell,
41flock.
42
43## Opening streams
44
45In most cases, you should use this API:
46
47```c
48PHPAPI php_stream *php_stream_open_wrapper(const char *path, const char *mode,
49    int options, char **opened_path);
50```
51
52Where:
53
54* `path` is the file or resource to open.
55* `mode` is the stdio compatible mode eg: "wb", "rb" etc.
56* `options` is a combination of the following values:
57  * `IGNORE_PATH` (default) - don't use include path to search for the file
58  * `USE_PATH` - use include path to search for the file
59  * `IGNORE_URL` - do not use plugin wrappers
60  * `REPORT_ERRORS` - show errors in a standard format if something goes wrong.
61  * `STREAM_MUST_SEEK` - If you really need to be able to seek the stream and
62    don't need to be able to write to the original file/URL, use this option to
63    arrange for the stream to be copied (if needed) into a stream that can be
64    seek()ed.
65* `opened_path` is used to return the path of the actual file opened, but if you
66  used `STREAM_MUST_SEEK`, may not be valid. You are responsible for
67  `efree()ing` `opened_path`.
68* `opened_path` may be (and usually is) `NULL`.
69
70If you need to open a specific stream, or convert standard resources into
71streams there are a range of functions to do this defined in `php_streams.h`. A
72brief list of the most commonly used functions:
73
74```c
75PHPAPI php_stream *php_stream_fopen_from_file(FILE *file, const char *mode);
76    /* Convert a FILE * into a stream. */
77
78PHPAPI php_stream *php_stream_fopen_tmpfile(void);
79    /* Open a FILE * with tmpfile() and convert into a stream. */
80
81PHPAPI php_stream *php_stream_fopen_temporary_file(const char *dir,
82    const char *pfx, char **opened_path);
83    /* Generate a temporary file name and open it. */
84```
85
86There are some network enabled relatives in `php_network.h`:
87
88```c
89PHPAPI php_stream *php_stream_sock_open_from_socket(int socket, int persistent);
90    /* Convert a socket into a stream. */
91
92PHPAPI php_stream *php_stream_sock_open_host(const char *host, unsigned short port,
93        int socktype, int timeout, int persistent);
94    /* Open a connection to a host and return a stream. */
95
96PHPAPI php_stream *php_stream_sock_open_unix(const char *path, int persistent,
97    struct timeval *timeout);
98    /* Open a UNIX domain socket. */
99```
100
101## Stream utilities
102
103If you need to copy some data from one stream to another, you will be please to
104know that the streams API provides a standard way to do this:
105
106```c
107PHPAPI size_t php_stream_copy_to_stream(php_stream *src,
108    php_stream *dest, size_t maxlen);
109```
110
111If you want to copy all remaining data from the src stream, pass
112`PHP_STREAM_COPY_ALL` as the maxlen parameter, otherwise maxlen indicates the
113number of bytes to copy. This function will try to use mmap where available to
114make the copying more efficient.
115
116If you want to read the contents of a stream into an allocated memory buffer,
117you should use:
118
119```c
120PHPAPI size_t php_stream_copy_to_mem(php_stream *src, char **buf,
121    size_t maxlen, int persistent);
122```
123
124This function will set buf to the address of the buffer that it allocated, which
125will be maxlen bytes in length, or will be the entire length of the data
126remaining on the stream if you set maxlen to `PHP_STREAM_COPY_ALL`. The buffer
127is allocated using `pemalloc()`. You need to call `pefree()` to release the
128memory when you are done. As with `copy_to_stream`, this function will try use
129mmap where it can.
130
131If you have an existing stream and need to be able to `seek()` it, you can use
132this function to copy the contents into a new stream that can be `seek()ed`:
133
134```c
135PHPAPI int php_stream_make_seekable(php_stream *origstream, php_stream **newstream);
136```
137
138It returns one of the following values:
139
140```c
141#define PHP_STREAM_UNCHANGED 0 /* orig stream was seekable anyway */
142#define PHP_STREAM_RELEASED  1 /* newstream should be used; origstream is no longer valid */
143#define PHP_STREAM_FAILED    2 /* an error occurred while attempting conversion */
144#define PHP_STREAM_CRITICAL  3 /* an error occurred; origstream is in an unknown state; you should close origstream */
145```
146
147`make_seekable` will always set newstream to be the stream that is valid if the
148function succeeds. When you have finished, remember to close the stream.
149
150NOTE: If you only need to seek forward, there is no need to call this function,
151as the `php_stream_seek` can emulate forward seeking when the whence parameter
152is `SEEK_CUR`.
153
154NOTE: Writing to the stream may not affect the original source, so it only makes
155sense to use this for read-only use.
156
157NOTE: If the origstream is network based, this function will block until the
158whole contents have been downloaded.
159
160NOTE: Never call this function with an origstream that is referenced as a
161resource! It will close the origstream on success, and this can lead to a crash
162when the resource is later used/released.
163
164NOTE: If you are opening a stream and need it to be seekable, use the
165`STREAM_MUST_SEEK` option to php_stream_open_wrapper();
166
167```c
168PHPAPI int php_stream_supports_lock(php_stream * stream);
169```
170
171This function will return either 1 (success) or 0 (failure) indicating whether
172or not a lock can be set on this stream. Typically you can only set locks on
173stdio streams.
174
175## Casting streams
176
177What if your extension needs to access the `FILE*` of a user level file pointer?
178You need to "cast" the stream into a `FILE*`, and this is how you do it:
179
180```c
181FILE * fp;
182php_stream * stream; /* already opened */
183
184if (php_stream_cast(stream, PHP_STREAM_AS_STDIO, (void*)&fp, REPORT_ERRORS) == FAILURE)    {
185    RETURN_FALSE;
186}
187```
188
189The prototype is:
190
191```c
192PHPAPI int php_stream_cast(php_stream * stream, int castas, void ** ret, int show_err);
193```
194
195The `show_err` parameter, if non-zero, will cause the function to display an
196appropriate error message of type `E_WARNING` if the cast fails.
197
198`castas` can be one of the following values:
199
200```txt
201PHP_STREAM_AS_STDIO - a stdio FILE*
202PHP_STREAM_AS_FD - a generic file descriptor
203PHP_STREAM_AS_SOCKETD - a socket descriptor
204```
205
206If you ask a socket stream for a `FILE*`, the abstraction will use fdopen to
207create it for you. Be warned that doing so may cause buffered data to be lost
208if you mix ANSI stdio calls on the FILE* with php stream calls on the stream.
209
210If your system has the fopencookie function, php streams can synthesize a
211`FILE*` on top of any stream, which is useful for SSL sockets, memory based
212streams, data base streams etc. etc.
213
214In situations where this is not desirable, you should query the stream to see if
215it naturally supports `FILE *`. You can use this code snippet for this purpose:
216
217```c
218if (php_stream_is(stream, PHP_STREAM_IS_STDIO)) {
219    /* can safely cast to FILE* with no adverse side effects */
220}
221```
222
223You can use:
224
225```c
226PHPAPI int php_stream_can_cast(php_stream * stream, int castas)
227```
228
229to find out if a stream can be cast, without actually performing the cast, so to
230check if a stream is a socket you might use:
231
232```c
233if (php_stream_can_cast(stream, PHP_STREAM_AS_SOCKETD) == SUCCESS)  {
234    /* it can be a socket */
235}
236```
237
238Please note the difference between `php_stream_is` and `php_stream_can_cast`;
239`stream_is` tells you if the stream is a particular type of stream, whereas
240`can_cast` tells you if the stream can be forced into the form you request. The
241former doesn't change anything, while the later *might* change some state in the
242stream.
243
244## Stream internals
245
246There are two main structures associated with a stream - the `php_stream`
247itself, which holds some state information (and possibly a buffer) and a
248`php_stream_ops` structure, which holds the "virtual method table" for the
249underlying implementation.
250
251The `php_streams` ops struct consists of pointers to methods that implement
252read, write, close, flush, seek, gets and cast operations. Of these, an
253implementation need only implement write, read, close and flush. The gets method
254is intended to be used for streams if there is an underlying method that can
255efficiently behave as fgets. The ops struct also contains a label for the
256implementation that will be used when printing error messages - the stdio
257implementation has a label of `STDIO` for example.
258
259The idea is that a stream implementation defines a `php_stream_ops` struct, and
260associates it with a `php_stream` using `php_stream_alloc`.
261
262As an example, the `php_stream_fopen()` function looks like this:
263
264```c
265PHPAPI php_stream * php_stream_fopen(const char * filename, const char * mode)
266{
267    FILE * fp = fopen(filename, mode);
268    php_stream * ret;
269
270    if (fp) {
271        ret = php_stream_alloc(&php_stream_stdio_ops, fp, 0, 0, mode);
272        if (ret)
273            return ret;
274
275        fclose(fp);
276    }
277    return NULL;
278}
279```
280
281`php_stream_stdio_ops` is a `php_stream_ops` structure that can be used to
282handle `FILE*` based streams.
283
284A socket based stream would use code similar to that above to create a stream to
285be passed back to fopen_wrapper (or it's yet to be implemented successor).
286
287The prototype for php_stream_alloc is this:
288
289```c
290PHPAPI php_stream * php_stream_alloc(php_stream_ops * ops, void * abstract,
291        size_t bufsize, int persistent, const char * mode)
292```
293
294* `ops` is a pointer to the implementation,
295* `abstract` holds implementation specific data that is relevant to this
296  instance of the stream,
297* `bufsize` is the size of the buffer to use - if 0, then buffering at the
298  stream
299* `level` will be disabled (recommended for underlying sources that implement
300  their own buffering - such a `FILE*`)
301* `persistent` controls how the memory is to be allocated - persistently so that
302  it lasts across requests, or non-persistently so that it is freed at the end
303  of a request (it uses pemalloc),
304* `mode` is the stdio-like mode of operation - php streams places no real
305  meaning in the mode parameter, except that it checks for a `w` in the string
306  when attempting to write (this may change).
307
308The mode parameter is passed on to `fdopen/fopencookie` when the stream is cast
309into a `FILE*`, so it should be compatible with the mode parameter of `fopen()`.
310
311## Writing your own stream implementation
312
313* **RULE #1**: when writing your own streams: make sure you have configured PHP
314  with `--enable-debug`.
315  Some great great pains have been taken to hook into the Zend memory manager to
316  help track down allocation problems. It will also help you spot incorrect use
317  of the STREAMS_DC, STREAMS_CC and the semi-private STREAMS_REL_CC macros for
318  function definitions.
319
320* RULE #2: Please use the stdio stream as a reference; it will help you
321  understand the semantics of the stream operations, and it will always be more
322  up to date than these docs :-)
323
324First, you need to figure out what data you need to associate with the
325`php_stream`. For example, you might need a pointer to some memory for memory
326based streams, or if you were making a stream to read data from an RDBMS like
327MySQL, you might want to store the connection and rowset handles.
328
329The stream has a field called abstract that you can use to hold this data. If
330you need to store more than a single field of data, define a structure to hold
331it, allocate it (use pemalloc with the persistent flag set appropriately), and
332use the abstract pointer to refer to it.
333
334For structured state you might have this:
335
336```c
337struct my_state {
338    MYSQL conn;
339    MYSQL_RES * result;
340};
341
342struct my_state * state = pemalloc(sizeof(struct my_state), persistent);
343
344/* initialize the connection, and run a query, using the fields in state to
345 * hold the results */
346
347state->result = mysql_use_result(&state->conn);
348
349/* now allocate the stream itself */
350stream = php_stream_alloc(&my_ops, state, 0, persistent, "r");
351
352/* now stream->abstract == state */
353```
354
355Once you have that part figured out, you can write your implementation and
356define the your own php_stream_ops struct (we called it my_ops in the above
357example).
358
359For example, for reading from this weird MySQL stream:
360
361```c
362static size_t php_mysqlop_read(php_stream * stream, char * buf, size_t count)
363{
364    struct my_state * state = (struct my_state*)stream->abstract;
365
366    if (buf == NULL && count == 0)  {
367        /* in this special case, php_streams is asking if we have reached the
368         * end of file */
369        if (... at end of file ...)
370            return EOF;
371        else
372            return 0;
373    }
374
375    /* pull out some data from the stream and put it in buf */
376    ... mysql_fetch_row(state->result) ...
377    /* we could do something strange, like format the data as XML here,
378        and place that in the buf, but that brings in some complexities,
379        such as coping with a buffer size too small to hold the data,
380        so I won't even go in to how to do that here */
381}
382```
383
384Implement the other operations - remember that write, read, close and flush are
385all mandatory. The rest are optional. Declare your stream ops struct:
386
387```c
388php_stream_ops my_ops = {
389    php_mysqlop_write, php_mysqlop_read, php_mysqlop_close,
390    php_mysqlop_flush, NULL, NULL, NULL,
391    "Strange MySQL example"
392}
393```
394
395That's it!
396
397Take a look at the STDIO implementation in streams.c for more information about
398how these operations work.
399
400The main thing to remember is that in your close operation you need to release
401and free the resources you allocated for the abstract field. In the case of the
402example above, you need to use mysql_free_result on the rowset, close the
403connection and then use pefree to dispose of the struct you allocated. You may
404read the stream->persistent field to determine if your struct was allocated in
405persistent mode or not.
406