Add missing regex anchor

Move token constants into separate class

As these are shared between Php5 and Php7 parsers they should be
in some common place, otherwise we'd have to always reference either
one or

Move token constants into separate class

As these are shared between Php5 and Php7 parsers they should be
in some common place, otherwise we'd have to always reference either
one or the other.

show more ...

Fix lexer tests for HHVM

HHVM does not throw warnings from token_get_all()
for invalid chars and unterminated comments. This
is not really integral functionality, so I'm okay
to

Fix lexer tests for HHVM

HHVM does not throw warnings from token_get_all()
for invalid chars and unterminated comments. This
is not really integral functionality, so I'm okay
to just skip it to get a passing HHVM build.

show more ...

Support token position attributes in lexer

Also change endFilePos semantics to refer to the last character that
is *included* in the token, rather than one past the last character.
T

Support token position attributes in lexer

Also change endFilePos semantics to refer to the last character that
is *included* in the token, rather than one past the last character.
This ensures that all end* attributes have the same semantics.

show more ...

Support file offsets in emulative lexer as well

Also run normal lexer tests against emulative lexer and fix a bug
in __halt_compiler() handling found as a result.

Add optional startFilePos and endFilePos attributes

The lexer can now optionally add startFilePos and endFilePos
attributes, which are offsets in to the lexed code string.

The e

Add optional startFilePos and endFilePos attributes

The lexer can now optionally add startFilePos and endFilePos
attributes, which are offsets in to the lexed code string.

The end offset currently points one past the last character of
the token - this is pending further discussion.

The attributes are not added by default and have to be enabled
using the new 'usedAttributes' lexer option:

$lexer = new Lexer([
'usedAttributes' => [
'comments', 'startLine', 'endLine',
'startFilePos', 'endFilePos'
]
]);

show more ...

Use stricter assertions where possible

Port library to use namespaces, with BC for old names