History log of /PHP-8.4/ext/mbstring/mbstring.c (Results 1 – 25 of 871)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
# c34d4fbb 11-Oct-2024 David Carlier

Fix GH-16360 mb_substr overflow on start and length arguments.

occurs when they are negated to start working from the end instead
when set with ZEND_LONG_MIN.


# bf70d9ba 06-Oct-2024 Niels Dossche <7771979+nielsdos@users.noreply.github.com>

Fix GH-16261: Reference invariant broken in mb_convert_variables()

The behaviour is weird in the sense that the reference must get
unwrapped. What ended up happening is that when destroy

Fix GH-16261: Reference invariant broken in mb_convert_variables()

The behaviour is weird in the sense that the reference must get
unwrapped. What ended up happening is that when destroying the old
reference the sources list was not cleaned properly. We add handling for
that. Normally we would use use ZEND_TRY_ASSIGN_STRINGL but that doesn't
work here as it would keep the reference and change values through
references (see bug #26639).

Closes GH-16272.

show more ...


# d840200c 05-Oct-2024 Yuya Hamada

Fix GH-16229: Address overflowed in mb_send_mail when empty string


# 5853cdb7 20-Aug-2024 Gina Peter Bnayard

Use "must not" instead of "cannot" wording


# 9a2fdbec 18-Aug-2024 Gina Peter Bnayard

ext/mbstring: Use standard wording for ValueError


# 25a51461 01-Jun-2024 Gina Peter Banyard

Clean-up unused headers (#14365)

* ext/mbstring.c: clean-up headers and include intrinsics


# 48d5ae98 29-May-2024 Gina Peter Banyard

ext/standard: Refactor exec.c public APIs to use zend_string pointers (#14353)

* Pull zend_string* from INI directive

* Ensure that mail.force_extra_parameters INI directive does no

ext/standard: Refactor exec.c public APIs to use zend_string pointers (#14353)

* Pull zend_string* from INI directive

* Ensure that mail.force_extra_parameters INI directive does not have any nul bytes

* ext/standard: Make php_escape_shell_cmd() take a zend_string* instead of char*

This saves on an expensive strlen() computation

* Convert E_ERROR to ValueError in php_escape_shell_cmd()

* ext/standard: Make php_escape_shell_arg() take a zend_string* instead of char*

This saves on an expensive strlen() computation

* Convert E_ERROR to ValueError in php_escape_shell_arg()

show more ...


# c96e8946 22-Nov-2023 Gina Peter Banyard

ext/mbstring: Check encoding passed to mb_http_output() has no null bytes


# 86dfbadc 19-Nov-2023 Gina Peter Banyard

ext/mbstring: Always pass length to php_mb_get_encoding_or_pass()

We have access to this information, so propagate it instead of calling strlen().
This also removes the newly introduced

ext/mbstring: Always pass length to php_mb_get_encoding_or_pass()

We have access to this information, so propagate it instead of calling strlen().
This also removes the newly introduced _ex() variant.

show more ...


# b61479bb 18-Nov-2023 Gina Peter Banyard

ext/mbstring: Pass string length to _php_mb_ini_mbstring_http_output_set()


# f8137084 24-Apr-2024 Niels Dossche <7771979+nielsdos@users.noreply.github.com>

Fix GH-13815: mb_trim() inaccurate $characters default value (#13820)

Because the default characters are defined in the stub file, and the
stub file is UTF-8 (typically), the characters

Fix GH-13815: mb_trim() inaccurate $characters default value (#13820)

Because the default characters are defined in the stub file, and the
stub file is UTF-8 (typically), the characters are encoded in the string
as UTF-8. When using a different character encoding, there is a mismatch
between what mb_trim expects and the UTF-8 encoded string it gets.

One way of solving this is by making the characters argument nullable,
which would mean that it always uses the internal code path that has the
unicode codepoints that are defaulted actually stored as codepoint
numbers instead of in a string.

Co-authored-by: @ranvis

show more ...


# 2cfd9df1 10-Apr-2024 David Carlier

Fix GH-13932: Attempt to fix mbstring on windows build (msvc).

Build failure due to lack of VLA support in older compiler versions.


# 3394efc6 18-Mar-2024 Alex Dowad

Fix infinite loop in mb_encode_mimeheader


# 4d51bfa2 20-Mar-2024 tekimen

[RFC] Add mb_ucfirst and mb_lcfirst functions (#13161)


# 8128d17c 24-Jan-2024 Niels Dossche <7771979+nielsdos@users.noreply.github.com>

Fix build warning in ext/mbstring


# 67051eb8 11-Jan-2024 Alex Dowad

Fix segfault caused by use of 'pass' encoding when mbstring converts multipart form POST data

When mbstring.encoding_translation=1, and PHP receives an (RFC1867)
form-based file upload,

Fix segfault caused by use of 'pass' encoding when mbstring converts multipart form POST data

When mbstring.encoding_translation=1, and PHP receives an (RFC1867)
form-based file upload, and the Content-Disposition HTTP header contains
a filename for the uploaded file, PHP will internally invoke mbstring
code to 1) try to auto-detect the text encoding of the filename, and if
that succeeds, 2) convert the filename to internal text encoding.

In such cases, the candidate text encodings which are considered during
"auto-detection" are those listed in the INI parameter
mbstring.http_input. Further, mbstring.http_input is one of the few
contexts where mbstring allows the magic string "pass" to appear in
place of an actual text encoding name.

Before mbstring's encoding auto-detection function was reimplemented,
the old implementation would never return "pass", even if "pass" was the
only candidate it was given to choose from. It is not clear if this was
intended by the original developers or not. This behavior was the result
of some rather subtle details of the implementation.

After mbstring's auto-detection function was reimplemented, if the new
implementation was given only one candidate to choose, and it was not
running in 'strict' mode, it would always return that candidate, even
if the candidate was the non-encoding "pass".

The upshot of all of this: Previously, if
mbstring.encoding_translation=1 and mbstring.http_input=pass, encoding
conversion of RFC1867 filenames would never be attempted. But after
the reimplementation, encoding 'conversion' would occur (uselessly).

Further, in December 2022, I reimplemented the relevant bit of
encoding conversion code. When doing this, I never bothered to
implement encoding/decoding routines for the non-encoding "pass",
because I thought that they would never be used. Well, in the one case
described above, those routines *would* have been used, had they
actually existed. Because they didn't exist, we get a nice NULL pointer
dereference and ensuing segfault instead.

Instead of 'fixing' this by adding encoding/decoding routines for the
non-encoding "pass", I have modified the function which the RFC1867
form-handling code invokes to auto-detect input encoding. This function
will never return "pass" now, just like the previous implementation.

Thanks to the GitHub user 'tstangner' for reporting this bug.

show more ...


# 1e92d47f 14-Jan-2024 Alex Dowad

Do not allow zend.script_encoding to be set to 'pass'

When investigating another bug reported by GitHub user 'tstangner',
I discovered that PHP segfaults when the INI parameter
zend.

Do not allow zend.script_encoding to be set to 'pass'

When investigating another bug reported by GitHub user 'tstangner',
I discovered that PHP segfaults when the INI parameter
zend.script_encoding is set to "pass". This bug dates back to
December 2022 (caused by yours truly in 953864661a).

If any PHP users in the wild were actually setting zend.script_encoding
to "pass" (which would be an utterly useless thing to do), I expect that
someone would have filed a bug report by now. The absence of such bug
reports is evidence that nobody is doing this.

Hence, it seems that the best fix is simply to disallow "pass" as a
choice for zend.script_encoding. The internal function
'php_mb_zend_encoding_list_parser' which I am modifying to accomplish
this has no other in-tree callers, aside from the 'exif' extension.
Further, exif only calls the function with a few hard-coded values, and
none of them are the string "pass", so this change will not have any
impact on exif.

show more ...


# 927adfb1 20-Dec-2023 Cristian Rodríguez

Use a single version of mempcpy(3) (#12257)

While __php_mempcpy is only used by ext/standard/crypt_sha*, the
mempcpy "pattern" is used everywhere.

This commit removes __php_memp

Use a single version of mempcpy(3) (#12257)

While __php_mempcpy is only used by ext/standard/crypt_sha*, the
mempcpy "pattern" is used everywhere.

This commit removes __php_mempcpy, adds zend_mempcpy and transforms
open-coded parts into function calls.

show more ...


# e8141973 20-Dec-2023 Alex Dowad

Fix bug in mb_get_substr_slow (sometimes outputs wrong number of characters)

Thanks to Maurício Fauth for finding and reporting this bug.

The bug was introduced in October 2022. It

Fix bug in mb_get_substr_slow (sometimes outputs wrong number of characters)

Thanks to Maurício Fauth for finding and reporting this bug.

The bug was introduced in October 2022. It originally only affected
text encodings which do not have a fixed byte width per characters
and for which mbstring does not have an mblen_table. However, I recently
made another change to mbstring, such that mb_substr no longer relies
on the mblen_table even if one is available. Because of this change,
the bug earlier introduced in October 2022 now affected a greater
number of text encodings, including UTF-8.

show more ...


# cffdeb81 14-Dec-2023 Alex Dowad

Add specialized implementation of mb_strcut for GB18030

For GB18030, it is not generally possible to identify character
boundaries without scanning through the entire string. Therefore,

Add specialized implementation of mb_strcut for GB18030

For GB18030, it is not generally possible to identify character
boundaries without scanning through the entire string. Therefore,
implement mb_strcut using a similar strategy as the mblen_table based
implementation in mbstring.c. The difference is that for GB18030, we
need to look at two leading bytes to determine the byte length of a
multi-byte character.

The new implementation is 4-5x faster for short strings, and more than
10x faster for long strings. (Part of the reason why this new code has
such a great performance advantage is because it is replacing code
based on the older text conversion filters provided by libmbfl, which
were quite slow.)

The behavior is the same as before for valid GB18030 strings; for
some invalid strings, mb_strcut will choose different 'cut' points
as compared to before. (Clang's libFuzzer was used to compare the
old and new implementations, searching for test cases where they had
different behavior; no such cases were found.)

show more ...


# 6da8b93e 22-Nov-2023 Gina Peter Banyard

ext/mbstring: Refactor mb_get_info()


# a6775c30 22-Nov-2023 Gina Peter Banyard

ext/mbstring: Refactor mb_trim_width() to take size_t arguments


# ec348a12 06-Dec-2023 Alex Dowad

Character indices used by mb_strpos and mb_substr have same meaning, even on invalid strings

Starting many years ago, libmbfl included a 'mblen_table' for selected
text encodings. This t

Character indices used by mb_strpos and mb_substr have same meaning, even on invalid strings

Starting many years ago, libmbfl included a 'mblen_table' for selected
text encodings. This table allows looking up the byte length of a
(possibly multi-byte) character from the value of the first byte.
libmbfl uses these tables to optimize certain operations; if a
text-processing operation can be performed using an mblen_table,
it may not be necessary to decode the text to codepoints. Since
libmbfl's decoding filters are generally slow, this improves
performance.

Since mbstring is (or was) based on libmbfl, it has always used
these mblen_tables to implement some functions. This design has a
significant downside. Let me explain:

While some mbstring functions are implemented by converting input text
to codepoints and operating on the codepoints, others operate directly
on the original input bytes (using an mblen_table to identify character
boundaries). Both of these implementation styles, if correctly coded,
yield equivalent results on valid strings. However, on strings which
contain encoding errors, the results are often different.

When decoding byte strings to codepoints using some text encoding,
mbstring uses the non-existent codepoint 0xFFFFFFFF to represent a
byte sequence which cannot be decoded. Then, when mbstring indexes into
the resulting sequence of codepoints, the index of any particular
character depends on the number of such 'error markers' which were
produced during the decoding process. In contrast, when an mblen_table
is used to split a byte sequence into characters, there is no question
of counting encoding errors; rather, table lookups into the mblen_table
are used to repeatedly 'bite off' some number of bytes (which are
treated as one 'character'). In the presence of encoding errors, these
two methods of mapping between byte indices and character indices are
inherently different and will rarely agree.

(For completeness, it must be said that some internal mbstring code
which operates only on UTF-8 text uses a third method for mapping
between byte indices and character indices, that is: counting
non-continuation UTF-8 bytes, which are all bytes whose binary
representation is NOT like 0b10xxxxxx. This method happens to agree with
the method which involves decoding the input text to codepoints and then
counting the codepoints.)

I have been aware of this issue for years, but only recently became
aware that in the case of mb_strstr, mb_strpos, and mb_substr,
this issue can cause seriously unintuitive behavior (and even security
vulnerabilities). This was reported by Stefan Schiller.

Stefan Schiller shared the following example for mb_strstr:

var_dump(mb_strstr("\xf0start", "start", false, "UTF-8"));
// string(2) "rt"

Similarly, when mb_strpos and mb_substr are used to identify and
extract a substring from a string with encoding errors, Stefan Schiller
pointed out that the extracted portion may be completely different than
desired. This is because (for UTF-8 strings) mb_strpos works by counting
non-continuation bytes, but mb_substr uses an mblen_table.

Since some mbstring functions *cannot* be implemented using an
mblen_table, as long as mblen_tables are used, similar inconsistencies
cannot be totally avoided. But the mblen_tables are critical to
mbstring's performance. Or are they? Benchmarking mb_substr on various
UTF-8, SJIS, and EUC-JP strings revealed something interesting.
On all SJIS and EUC-JP test cases, mb_substr was slightly faster when
the mblen_table based code was deleted. For some UTF-8 test cases, the
mblen_table-based code was a tiny bit faster, while for others the
fallback code was a touch faster; in no case was the difference
significant.

Therefore, the simple fix is to delete the mblen_table-based
implementation of mb_substr.

Aside from making the function behave consistently with other mbstring
functions on invalid strings, there is ONE case where behavior is now
different on valid strings: that is, on SJIS-Mac (MacJapanese) strings
which contain any of the following code units:

0x85AB-0x85AD, 0x85BF, 0x85C0, 0x85C1, 0x8645, 0x864B, 0x865D, 0x869E,
0x86CE, 0x86D3-0x86D5, 0x86D6, 0x8971, 0x8792, 0x879D, 0x87FB, 0x87FC,
0xEB41, 0xEB42, 0xEB50, 0xEB5B, 0xEB5D, 0xEB60-0xEB6E, and all from
0xEB81 and above.

All of these SJIS-Mac code units share the (very unusual) property that
they do not correspond to any one Unicode codepoint. When converting
from SJIS-Mac to Unicode, these must be converted to 2, 3, 4, or 5
codepoints each.

The previous, mblen_table-based implementation of mb_substr would treat
all of these SJIS-Mac byte sequences as 'one character'. Now, they are
treated as multiple characters (one for each of the Unicode codepoints
which they decode to). The new behavior is more consistent with other
mbstring functions.

I don't know if SJIS-Mac users will like this change or not (probably
most will never notice), but the BC break is justified by the very
real security impact of the previous, inconsistent behavior.

Finally, I should comment on whether similar changes are needed
elsewhere. The remaining functions which use an mblen_table are:
mb_str_split, mb_strcut, and various search functions (such as
mb_strpos). The search functions are only affected now when they
receive a positive 'offset' parameter specifying where to start
searching from.

The search functions should definitely be fixed so they do not use
an mblen_table to implement the 'offset' parameter. I am not convinced
that there is any good reason to change mb_str_split and mb_strcut.

show more ...


# 88ba9dc6 07-Dec-2023 Gina Peter Banyard

ext/mbstring: Always throw ValueErrors for invalid mb_http_input() type


# e74bf42c 06-Dec-2023 Gina Peter Banyard

ext/mbstring: Check conversion map only has integers


12345678910>>...35