Add missing void return types (#997)

Replace startLexing() with tokenize()

For now Lexer::getTokens() still exists, but should probably be
removed.

Move attribute handling into parser

The Lexer now only provides the tokens to the parser, while the
parser is responsible for determining which attributes are placed
on notes. This o

Move attribute handling into parser

The Lexer now only provides the tokens to the parser, while the
parser is responsible for determining which attributes are placed
on notes. This only needs to be done when the attributes are
actually needed, rather than for all tokens.

This removes the usedAttributes lexer option (and lexer options
entirely). The attributes are now enabled unconditionally. They
have less overhead now, and the need to explicitly enable them for
some use cases (e.g. formatting-preserving printing) doesn't seem
like a good tradeoff anymore.

There are some additional changes to the Lexer interface that
should be done after this, and the docs / upgrading guide haven't
been adjusted yet.

show more ...

Fix emulative lexer with default error handler

If no error handler is provided, explicitly create one, so we don't
end up calling handleError() on null.

Format tests as well

The unnecessary parentheses for "new" are a bit annoying, but I
can live with it...

Move token mapping from lexer to parser

This allows a different token mapping per parser.

Use PHP 8.0 token representation

Migrate everything to use PhpToken-compatible token representation,
rather than the legacy array/string representation.

Support keywords in namespaced names

Implement emulation of PHP 8 T_NAME_* tokens

Like comment emulation, this is unconditional, as it is required
for core functionality.

Fix handling of unterminated comment with trailing newline

Fixes #688.

Canonicalize to PHP 8 comment token format

The trailing newline is no longer part of the comment token.

Add end line / file position / token position to comments

updates via "rectorphp/rector" (#573)

- "global" -> remove unused "use" statements
- "phpunit" -> fix "@covers" comments
- "phpunit" -> replace "->will($this->returnValue()" with "->

updates via "rectorphp/rector" (#573)

- "global" -> remove unused "use" statements
- "phpunit" -> fix "@covers" comments
- "phpunit" -> replace "->will($this->returnValue()" with "->willReturn()"
- "UseTest.php" -> add missing namespace
- "composer.json" -> use "autoload-dev"
- remove -> "require_once" usage in the tests (use autoload-dev via composer.json)

-> most of the changes are done automatically by "https://github.com/rectorphp/rector"

show more ...

Use methods instead of annotations for expected exceptions (#533)

Refactoring tests

Store start token position in comments

add strict_types to tests + fix explode type

[cs] apply same to tests

Fix Lexer errorHandling when there is no tokens

I encounter an issue when no tokens exist. An error `Undefined offset -1` is triggered.

Bump phpunit version (#381)

Introduce ErrorHandler

Add ErrorHandler interface, as well as ErrorHandler\Throwing
and ErrorHandler\Collecting. The error handler is passed to
Parser::parse(). This supersedes the t

Introduce ErrorHandler

Add ErrorHandler interface, as well as ErrorHandler\Throwing
and ErrorHandler\Collecting. The error handler is passed to
Parser::parse(). This supersedes the throwOnError option.

NameResolver now accepts an ErrorHandler in the ctor.

show more ...

Support recovery from lexer errors

Lexer::startLexing() no longer throws, instead errors can be fetched
using Lexer::getErrors().

Lexer errors now also contain full line and pos

Support recovery from lexer errors

Lexer::startLexing() no longer throws, instead errors can be fetched
using Lexer::getErrors().

Lexer errors now also contain full line and position information.

show more ...

Add hasLeadingNewline attribute to InlineHTML

Use this attribute to not print an extra newline if the original
code did not have it.

Add start file offset to Comments

Add missing regex anchor