Fix #79371: mb_strtolower (UTF-32LE): stack-buffer-overflow

We make sure that negative values are properly compared.

Fixed bug #76319

While at it, also make sure that mbstring case conversion takes
into account the specified substitution character and substitution
mode.

year++

year++

year++

fix c89 compat

Fixed bug #65544 and #71298

Implement full case mapping

Implement full case mapping according to SpecialCasing.txt and
also full case folding according to CaseFolding.txt (F). There
are a number of caveats:

Implement full case mapping

Implement full case mapping according to SpecialCasing.txt and
also full case folding according to CaseFolding.txt (F). There
are a number of caveats:

* Only language-agnostic and unconditional full case mapping
is implemented. The only language-agnostic conditional case
mapping rule relates to Greek sigma in final position
(Final_Sigma). Correctly handling this requires both arbitrary
lookahead and lookbehind, which would require some larger
changes to how the case mapping is implemented. This is a
possible future extension.
* The only language-specific handling that is implemented is
for Turkish dotted/undotted Is, if the ISO-8859-9 encoding
is used. This matches the previous behavior and makes sure
that no codepoints not supported by the encoding are
produced. A future extension would be to also handle the
Turkish mappings specified by SpecialCasing.txt based on
the mbfl internal language.
* Full case folding is implemented, but case-insensitive mb_*
operations continue to use simple case folding. The reason is
that full case folding of the haystack string may change the
position at which a match occurred. This would have to be
mapped back into the position in the original string.
* mb_convert_case() exposes both the full and the simple case
mapping / folding, where full is the default. The constants
are:

* MB_CASE_LOWER (used by mb_strtolower)
* MB_CASE_UPPER (used by mb_strtolower)
* MB_CASE_TITLE
* MB_CASE_FOLD
* MB_CASE_LOWER_SIMPLE
* MB_CASE_UPPER_SIMPLE
* MB_CASE_TITLE_SIMPLE
* MB_CASE_FOLD_SIMPLE (used by case-insensitive operations)

show more ...

Use case-folding for case insensitive comparisons

Instead of using lowercasing.

Use MPH for case maps

Instead of performing a binary search, use a hashtable to store
the case maps. In particular a minimal perfect hash construction
is used, which does not require

Use MPH for case maps

Instead of performing a binary search, use a hashtable to store
the case maps. In particular a minimal perfect hash construction
is used, which does not require collision resolution (but does
use an auxiliary table for the hash perturbation).

show more ...

Change layout of case mapping table

Previously the case mapping table was segregated by the type of
the character (upper, lower, title) and always stored the other
two variants (key,

Change layout of case mapping table

Previously the case mapping table was segregated by the type of
the character (upper, lower, title) and always stored the other
two variants (key, other1, other2). Now the table is segregated
by the target type (key, other). As only very few characters have
more than one target this only slightly increases the size of the
table.

The advantage of this layout is that we only need to perform a
single table lookup in the case table. Previously, depending on
the case that was hit, either one lookup in the property table,
or two lookups in the property table and one lookup in the case
table were required.

This changes the layout from libunicode in the OpenLDAP project
-- however, the last commit there was over 10 years ago, so I
don't see value in keeping this in sync.

show more ...

Another fix for bug #69267

mb_strtoupper() was converting lowercase characters into
titlecase characters, instead of uppercase characters. Luckily
there are only very few characters

Another fix for bug #69267

mb_strtoupper() was converting lowercase characters into
titlecase characters, instead of uppercase characters. Luckily
there are only very few characters with a distinct titlecase
representation, so this mostly worked out okay...

show more ...

Partial fix for bug #69267

This pulls in 60a25c72ba389f53b0621ca250bc99f3b295d43f from the
OpenLDAP project.

Directly use encodings instead of no_encoding in libmbfl

In particular strings now store encoding rather than the
no_encoding.

I've also pruned out libmbfl APIs that existed in

Directly use encodings instead of no_encoding in libmbfl

In particular strings now store encoding rather than the
no_encoding.

I've also pruned out libmbfl APIs that existed in two forms, one
using no_encoding and the other using encoding. We were not actually
using any of the former.

show more ...

Reduce number of encoding conversions in case conversion

Don't indirect through UCS4BE, instead directly work on wchars
using a custom filter.

This replaces the pipeline
u

Reduce number of encoding conversions in case conversion

Don't indirect through UCS4BE, instead directly work on wchars
using a custom filter.

This replaces the pipeline
utf8 -> wchar -> ucs4be -> wchar -case-> wchar -> ucs4be -> wchar -> utf8
with
utf8 -> wchar -case-> -> wchar -> utf8

show more ...

Optimize php_unicode_tolower/upper for ASCII

Directly accept encoding in php_unicode_convert_case()

As a side-effect mb_strtolower() and mb_strtoupper() now correctly
handle a NULL encoding parameter by using the internal encoding.

Directly accept encoding in php_unicode_convert_case()

As a side-effect mb_strtolower() and mb_strtoupper() now correctly
handle a NULL encoding parameter by using the internal encoding.
This is what caused the two test changes.

show more ...

Optimize php_unicode_is_prop()

Do not try to extract the properties from a bitmask. Instead make
the function variadic and pass all properties individually.

Also add a php_unico

Optimize php_unicode_is_prop()

Do not try to extract the properties from a bitmask. Instead make
the function variadic and pass all properties individually.

Also add a php_unicode_is_prop1() function to check only a single
property.

show more ...

Avoid unnecessary encoding lookups in mbstring

Extract part of php_mb_convert_encoding that does the actual work
and use it whenever we already know the encoding.

Update copyright headers to 2017

Update copyright headers to 2017

Update copyright headers to 2017

Happy new year (Update copyright to 2016)

bump year

bump year