Implement conditional casing for Greek letter sigma when title-casing text

Adjust code which checks if encoding is ISO-8859-9 when converting case

Instead of checking the 'encoding number' to see if we are converting
case for ISO-8859-9 text, compare pointers i

Adjust code which checks if encoding is ISO-8859-9 when converting case

Instead of checking the 'encoding number' to see if we are converting
case for ISO-8859-9 text, compare pointers instead.

This should free up 1 register in php_unicode_convert_case.

show more ...

Implement Unicode conditional casing rules for Greek letter sigma

The capital Greek letter sigma (Σ) should be lowercased as σ except
when it appears at the end of a word; in that case,

Implement Unicode conditional casing rules for Greek letter sigma

The capital Greek letter sigma (Σ) should be lowercased as σ except
when it appears at the end of a word; in that case, it should be
lowercased as the special form ς.

This rule is included in the Unicode data file SpecialCasing.txt.
The condition for applying the rule is called "Final_Sigma" and is
defined in Unicode technical report 21. The rule is:

• For the special casing form to apply, the capital letter sigma must
be preceded by 0 or more "case-ignorable" characters, preceded by
at least 1 "cased" character.
• Further, capital sigma must NOT be followed by 0 or more
case-ignorable characters and then at least 1 cased character.

"Case-ignorable" characters include certain punctuation marks, like
the apostrophe, as well as various accent marks. There are actually
close to 500 different case-ignorable characters, including accent marks
from Cyrillic, Hebrew, Armenian, Arabic, Syriac, Bengali, Gujarati,
Telugu, Tibetan, and many other alphabets. This category also includes
zero-width spaces, codepoints which indicate RTL/LTR text direction,
certain musical symbols, etc.

Since the rule involves scanning over "0 or more" of such
case-ignorable characters, it may be necessary to scan arbitrarily far
to the left and right of capital sigma to determine whether the special
lowercase form should be used or not. However, since we are trying to
be both memory-efficient and CPU-efficient, this implementation limits
how far to the left we will scan. Generally, we scan up to 63 characters
to the left looking for a "cased" character, but not more.

When scanning to the right, we go up to the end of the string if
necessary, even if it means scanning over thousands of characters.

Anyways, it is almost impossible to imagine that natural text will
include "words" with more than 63 successive apostrophes (for example)
followed by a capital sigma.

Closes GH-8096.

show more ...

Mark UTF-8 strings emitted by mbstring functions as valid UTF-8

We now have a couple of mbstring functions which have fast paths for
strings marked as 'valid UTF-8'. Later, we may likely

Mark UTF-8 strings emitted by mbstring functions as valid UTF-8

We now have a couple of mbstring functions which have fast paths for
strings marked as 'valid UTF-8'. Later, we may likely have more. So
that these fast paths can be used more frequently, mark UTF-8 strings
emitted by mbstring as 'valid UTF-8'. This is always a correct thing
to do, because mbstring never returns invalid UTF-8 as the result of
a conversion (or similar) operation.

Internally, we do have a conversion mode which deliberately emits
invalid UTF-8 in some cases. (This is done to prevent unwanted matches
when we are converting strings to UTF-8 before performing matching
operations on them.) For such strings, don't set the 'valid UTF-8' flag.
It probably wouldn't hurt anything to set it, because strings generated
using that special conversion mode should *never* be returned to
userland, and I don't think we do anything with them which cares about
the IS_STR_VALID_UTF8 flag... but still, it would likely cause
confusion for developers.

show more ...

Speed boost for mb_stripos (when not using UTF-8)

Instead of case-folding a string and then converting it to UTF-8 as a
separate operation, why not convert it to UTF-8 at the same time a

Speed boost for mb_stripos (when not using UTF-8)

Instead of case-folding a string and then converting it to UTF-8 as a
separate operation, why not convert it to UTF-8 at the same time as
we fold case?

For non-UTF-8 encodings, this typically makes mb_stripos about 2x
faster.

show more ...

Use uint32_t for 'illegal_substchar' codepoint in mbstring

This value is a wchar, so the best type for it is uint32_t.

Make enum for valid case_mode values (for php_unicode_convert_case)

Use fast text conversion filters for mb_convert_case, mb_strtoupper, mb_strtolower

Speed increase is only about 50% for title casing, but 2-3x for other
forms of case conversion.

Optimize mbstring upper/lowercasing: use fast path in more cases

The 'fast path' in the uppercase/lowercase functions for Unicode text can be used
for a slightly greater range of charact

Optimize mbstring upper/lowercasing: use fast path in more cases

The 'fast path' in the uppercase/lowercase functions for Unicode text can be used
for a slightly greater range of characters. This is not expected to have a big
impact on performance, since the number of characters which will use the 'fast path'
is only increased by about 50-60, and these are not very commonly used characters...
but still, it doesn't cost anything.

show more ...

Remove redundant NULL checks in mbstring

Whoever originally wrote mbstring seems to have a deathly fear of NULL
pointers lurking behind every corner. A common pattern is that one
fun

Remove redundant NULL checks in mbstring

Whoever originally wrote mbstring seems to have a deathly fear of NULL
pointers lurking behind every corner. A common pattern is that one
function will check if a pointer is NULL, then pass it to another
function, which will again check if it is NULL, then pass to yet another
function, which will yet again check if it is NULL... it's NULL checks
all the way down.

Remove all the NULL checks in places where pointers could not possibly
be NULL.

show more ...

Return bool from php_unicode_is_prop()

Don't use sentinel value for unicode property lookup

0xffff was used to mark character properties without any members.
This made the code unnecessarily complicated, because we need to

Don't use sentinel value for unicode property lookup

0xffff was used to mark character properties without any members.
This made the code unnecessarily complicated, because we need to
check for 0xffff values when looking up the property ranges. We
can simply encode this as an empty set of ranges.

show more ...

Fixed some spaces used instead of tabs

Update http->https in license (#6945)

1. Update: http://www.php.net/license/3_01.txt to https, as there is anyway server header "Location:" to https.
2. Update few license 3.0 to 3.01 as

Update http->https in license (#6945)

1. Update: http://www.php.net/license/3_01.txt to https, as there is anyway server header "Location:" to https.
2. Update few license 3.0 to 3.01 as 3.0 states "php 5.1.1, 4.1.1, and earlier".
3. In some license comments is "at through the world-wide-web" while most is without "at", so deleted.
4. fixed indentation in some files before |

show more ...

Don't guard mbstring code with #ifdef HAVE_MBSTRING

This is just a very silly feature of mbstring -- you can compile the source files with
HAVE_MBSTRING undefined, and it will all just c

Don't guard mbstring code with #ifdef HAVE_MBSTRING

This is just a very silly feature of mbstring -- you can compile the source files with
HAVE_MBSTRING undefined, and it will all just compile to (almost) nothing. What is the
use of this? Why compile the source files and link against them if you don't want the
mbstring extension? It doesn't make any kind of sense.

show more ...

Remove redundant includes from mbstring (and make sure correct config.h is used)

Very interesting... it turns out that when Valgrind support was enabled,
`#include "config.h"` from withi

Remove redundant includes from mbstring (and make sure correct config.h is used)

Very interesting... it turns out that when Valgrind support was enabled,
`#include "config.h"` from within mbstring was actually including the file "config.h"
from Valgrind, and not the one from mbstring!!

This is because -I/usr/include/valgrind was added to the compiler invocation _before_
-Iext/mbstring/libmbfl.

Make sure we actually include the file which was intended.

show more ...

Optimize php_unicode_convert_case (cuts mbstring case conversion time ~15%)

This function uses various subfunctions to convert case of Unicode wchars.
Previously, these subfunctions woul

Optimize php_unicode_convert_case (cuts mbstring case conversion time ~15%)

This function uses various subfunctions to convert case of Unicode wchars.
Previously, these subfunctions would store the case-converted characters in
a buffer, and the parent function would then pass them (byte by byte) to
the next filter in the filter chain.

Rather than passing around that buffer, it's better for the subfunctions to
directly pass the case-converted bytes to the next filter in the filter chain.
This speeds things up nicely.

show more ...

Fix [-Wundef] warning in MBString extension

Fix #79371: mb_strtolower (UTF-32LE): stack-buffer-overflow

We make sure that negative values are properly compared.

Remove mention of PHP major version in Copyright headers

Closes GH-4732.

Use EMPTY_SWITCH_DEFAULT_CASE in php_unicode.c

Avoids a potentially uninitialized variable warning.

Remove local variables

This patch removes the so called local variables defined per
file basis for certain editors to properly show tab width, and
similar settings. These are mainly

Remove local variables

This patch removes the so called local variables defined per
file basis for certain editors to properly show tab width, and
similar settings. These are mainly used by Vim and Emacs editors
yet with recent changes the once working definitions don't work
anymore in Vim without custom plugins or additional configuration.
Neither are these settings synced across the PHP code base.

A simpler and better approach is EditorConfig and fixing code
using some code style fixing tools in the future instead.

This patch also removes the so called modelines for Vim. Modelines
allow Vim editor specifically to set some editor configuration such as
syntax highlighting, indentation style and tab width to be set in the
first line or the last 5 lines per file basis. Since the php test
files have syntax highlighting already set in most editors properly and
EditorConfig takes care of the indentation settings, this patch removes
these as well for the Vim 6.0 and newer versions.

With the removal of local variables for certain editors such as
Emacs and Vim, the footer is also probably not needed anymore when
creating extensions using ext_skel.php script.

Additionally, Vim modelines for setting php syntax and some editor
settings has been removed from some *.phpt files. All these are
mostly not relevant for phpt files neither work properly in the
middle of the file.

show more ...

Remove yearly range from copyright notice

Fixed bug #76319

While at it, also make sure that mbstring case conversion takes
into account the specified substitution character and substitution
mode.

year++