Rebuild scanner, add NEWS/UPGRADING

ext tokenizer port + cleanup unused lexer states

we basically added a mechanism to store the token stream during parsing
and exposed the entire parser stack on the tokenizer extension th

ext tokenizer port + cleanup unused lexer states

we basically added a mechanism to store the token stream during parsing
and exposed the entire parser stack on the tokenizer extension through
an opt in flag: token_get_all($src, TOKEN_PARSE).

this change allows easy future language enhancements regarding context
aware parsing & scanning without further maintance on the tokenizer
extension while solves known inconsistencies "parseless" tokenizer
extension has when it handles `__halt_compiler()` presence.

show more ...

implement context sensitive language with lexical feedback

The implementation has no regression risks, has an even smaller footprint
compared to the previous attempt involving a pure lex

implement context sensitive language with lexical feedback

The implementation has no regression risks, has an even smaller footprint
compared to the previous attempt involving a pure lexical approach, is higly
predictable and higly configurable.

To turn a word semi-reserved you only need to edit the "SEMI_RESERVED" parser rule,
it's an inclusive list of all the words that should be matched as T_STRING on specific contexts.
Example:

```
method_modifiers function returns_ref indentifier '(' parameter_list ')' ...
```

instead of:

```
method_modifiers function returns_ref T_STRING '(' parameter_list ')' ...
```

TODO: port ext tokenizer

show more ...

More cleanup (mainly retab)

Rebase Joe's anon classes implementation

Merge branch 'coroutineDelegation' of https://github.com/bwoebi/php-src

Throw ParseException from lexer

Primarily to avoid getting fatal errors from token_get_all().

Implemented using a magic E_ERROR token, which the lexer emits to
force a parser fa

Throw ParseException from lexer

Primarily to avoid getting fatal errors from token_get_all().

Implemented using a magic E_ERROR token, which the lexer emits to
force a parser failure.

show more ...

Improved parser (make use_type reuseable)

fix compilation warnings, rename batch_use to group_use

require trailing T_NS_SEPARATOR #1005

allow non mixed use declarations, CC @nikic #1005

```
use function N\S { foo, bar, etc };
use const N\S { FOO, BAR, ETC };
```

Related tests:

```
make test

allow non mixed use declarations, CC @nikic #1005

```
use function N\S { foo, bar, etc };
use const N\S { FOO, BAR, ETC };
```

Related tests:

```
make test -j4 TESTS="-m Zend/tests/ns_0{88..92}*.phpt"
```

All namespace tests:

```
make test -j4 TESTS="Zend/tests/ns_*.phpt"
```

show more ...

first implementation of batch use compilation

Added yield from operator

WIP

Merge branch 'pull-request/1007'

* pull-request/1007:
Fix associativity to match Perl
Blast off to space.

Márcio Almada remove dead tokens: T_CHARACTER, T_BAD_CHARACTER.

Fix associativity to match Perl

Blast off to space.

Implement return types

RFC is documented here: https://wiki.php.net/rfc/return_types

Remove assignment of new by reference

bump year

trailing whitespace removal

Removed parsing support traits to have extends and implements.

Decoupled class declaration statement into more granular pieces.

first shot remove TSRMLS_* things