Also format the grammar directory

Add PHP 8 parser with correct concatenation precedence

The PHP 7 and PHP 8 parsers use the same grammar file and only
differ in token precedence.

Move Tokens::T_* to Php7::T_*

Drop the separate tokens class, move them into the parser.

Remove PHP 5 parser

Moved phpy pseudo lang functions to separate file

[PHP 8.0] Add attributes support (#661)

Adds support for PHP 8 attributes, represented using `AttrGroup` nodes
containing `Attribute` nodes. The `attrGroup` subnode is added to all
n

[PHP 8.0] Add attributes support (#661)

Adds support for PHP 8 attributes, represented using `AttrGroup` nodes
containing `Attribute` nodes. The `attrGroup` subnode is added to all
nodes that can have attributes.

This is still missing FPPP support.

Co-authored-by: Nikita Popov <nikita.ppv@gmail.com>

show more ...

Fixed rebuildParsers.php (#686)

phpyacc is a sh file and should not be run via the php interpreter, but directly.

Place Nop nodes more precisely

If the Nop is created to collect comments, place it directly after
the comment, instead of at the next non-whitespace character.

Default to using php-yacc to generate the parsers

Accept KMYACC env var in rebuildParsers.php

Remove -l flag from kmyacc invocation

For PHP this doesn't do anything anyway.

Fix attributes for zero-length nop nodes

Previously zero-length nop nodes used the lookahead start attributes
and current end attributes. This choice ends up being somewhat weird,
be

Fix attributes for zero-length nop nodes

Previously zero-length nop nodes used the lookahead start attributes
and current end attributes. This choice ends up being somewhat weird,
because the end attributes will be the at the last non-whitespace,
non-comment token, which might be quite far back. More problematically,
we may not have encountered any non-discarded token if we're at the
start of the file, in which case we will have no end attributes to
assign.

Change things to use a canonical "zero-length" node representation,
where the end position (token & file) will be exactly one before the
start position.

Fixes #589.

show more ...

Flexible doc: Validate end label indentation

Move doc string parsing logic from rebuildParsers.php and
String_::parseDocString() into ParserAbstract. This stuff is
going to get compl

Flexible doc: Validate end label indentation

Move doc string parsing logic from rebuildParsers.php and
String_::parseDocString() into ParserAbstract. This stuff is
going to get complicated now.

For now only implement the validation of the indentation on the
end label.

show more ...

Preserve comments on empty blocks (#382)

Always use Identifier nodes

The parser will now always generate Identifier nodes (for
non-namespaced identifiers). This obsoletes the useIdentifierNodes
parser option.

Node

Always use Identifier nodes

The parser will now always generate Identifier nodes (for
non-namespaced identifiers). This obsoletes the useIdentifierNodes
parser option.

Node constructors still accepts strings and will implicitly create
an Identifier wrapper. Identifier implement __toString(), so that
outside of strict-mode many things continue to work without changes.

show more ...

Drop useNopStatements option

Don't require useNopStatements=false for format preservation

Instead assign attributes on Nop nodes and in the pretty printer
specially handle end<start offsets. It's a somewhat weird ca

Don't require useNopStatements=false for format preservation

Instead assign attributes on Nop nodes and in the pretty printer
specially handle end<start offsets. It's a somewhat weird case,
but not wrong per se given the meaning the offsets have.

show more ...

Drop last vestiges of consistent var mode flag

Make useExpressionStatement only supported mode

Add first shot at format preserving pretty printer

Add expression statement mode

Add VarLikeIdentifier

For representing Identifiers that have an implicit leading $.

With this done, maybe go one step further?
* Rename VarLikeIdentifier -> VarIdentifier / Var

Add VarLikeIdentifier

For representing Identifiers that have an implicit leading $.

With this done, maybe go one step further?
* Rename VarLikeIdentifier -> VarIdentifier / VarName
* Use VarIdentifier / VarName also as an inner node in Variable.
Not sure if this adds any real value.

show more ...

Add useConsistentVariableNodes mode

The parameter case is a bit weird, because the subnode is called
"name" here, rather than "var". Nothing we can do about that in
this version thou

Add useConsistentVariableNodes mode

The parameter case is a bit weird, because the subnode is called
"name" here, rather than "var". Nothing we can do about that in
this version though.

The two parser options might be merged. I've kept it separate,
because I think this variable representation should become the
default (or even only representation) in the future, while I'm
less sure about the Identifier thing.

show more ...

Add useIdentifierNodes mode to parser

In this mode non-namespaced names that are currently represented
using strings will be represented using Identifier nodes instead.
Identifier no

Add useIdentifierNodes mode to parser

In this mode non-namespaced names that are currently represented
using strings will be represented using Identifier nodes instead.
Identifier nodes have a string $name subnode and coerce to string.

This allows preserving attributes and in particular location
information on identifiers.

show more ...

Fix GroupUse prefix attribute assignment