Fix GH-11300: license issue: restricted unicode license headers

Closes GH-11572.

Fix conversion of EUC-TW text (and add test suite)

- Treat text which ends abruptly in the middle of a multi-byte
character as erroneous.
- Don't allow ASCII control characters to

Fix conversion of EUC-TW text (and add test suite)

- Treat text which ends abruptly in the middle of a multi-byte
character as erroneous.
- Don't allow ASCII control characters to appear in the middle of a
multi-byte character.
- If an illegal byte appears in the middle of a multi-byte character,
go back to the initial state rather than trying to finish the
multi-byte character.
- There was a bug in the file with the conversion tables, which set the
'maximum codepoint which can be converted using table A2' using the
size of table A1, not table A2. This meant that several hundred
Unicode codepoints which should have been able to be converted to
EUC-TW were flagged as erroneous instead.
- When a sequence which cannot possibly be a prefix of a valid
multi-byte character is found, immediately flag it as an error, rather
than waiting to read more bytes first.
- Allow characters in CNS-11643 plane 1 to be encoded as 4-byte
sequences (although they can also be encoded as 2-byte sequences).
This is allowed by the standard for EUC-TW text.

show more ...

Bugfixes for findInvalidChars (helper for mbstring test suite)

Test EUC-JP and Shift-JIS more thoroughly

Previously, the unit tests for these text encodings covered all mappings
from legacy -> Unicode, and all _reversible_ mappings from Unicode -> l

Test EUC-JP and Shift-JIS more thoroughly

Previously, the unit tests for these text encodings covered all mappings
from legacy -> Unicode, and all _reversible_ mappings from Unicode -> legacy.
However, we should also test the few Unicode -> legacy mappings which
are not reversible.

show more ...

Fix issues with mbstring encoding tests

I made some mistakes on this code, which meant that not everything which
should be tested was actually being tested.

Test cases for mbstring encodings are less repetitive

Add test suite for CP1252 encoding

Also remove a bogus test (bug62545.phpt) which wrongly assumed that all invalid
characters in CP1251 and CP1252 should map to Unicode 0xFFFD (REPLACEME

Add test suite for CP1252 encoding

Also remove a bogus test (bug62545.phpt) which wrongly assumed that all invalid
characters in CP1251 and CP1252 should map to Unicode 0xFFFD (REPLACEMENT
CHARACTER).

mbstring has an interface to specify what invalid characters should be
replaced with; it's called `mb_substitute_character`. If a user wants to see
the Unicode 'replacement character', they can specify that using
`mb_substitute_character`. But if they specify something else, we should
follow that.

show more ...

Add test suite for ISO-8859-x encoding verification and conversion