Usage of basic components

This document explains how to use the parser, the pretty printer and the node traverser.

Bootstrapping

To bootstrap the library, include the autoloader generated by composer:

require 'path/to/vendor/autoload.php';

Additionally, you may want to set the xdebug.max_nesting_level ini option to a higher value:

ini_set('xdebug.max_nesting_level', 3000);

This ensures that there will be no errors when traversing highly nested node trees. However, it is preferable to disable Xdebug completely, as it can easily make this library more than five times slower.

Parsing

In order to parse code, you first have to create a parser instance:

use PhpParser\ParserFactory;
use PhpParser\PhpVersion;

// Parser for the version you are running on.
$parser = (new ParserFactory())->createForHostVersion();

// Parser for the newest PHP version supported by the PHP-Parser library.
$parser = (new ParserFactory())->createForNewestSupportedVersion();

// Parser for a specific PHP version.
$parser = (new ParserFactory())->createForVersion(PhpVersion::fromString('8.1'));

Which version you should target depends on your use case. In many cases you will want to use the host version, as people typically analyze code for the version they are running on. However, when analyzing arbitrary code you are usually best off using the newest supported version, which tends to accept the widest range of code (unless there are breaking changes in PHP).

The createXYZ() methods optionally accept an array of lexer options. Some use cases that require customized lexer options are discussed in the lexer documentation.

Subsequently, you can pass PHP code (including the opening <?php tag) to the parse() method in order to create a syntax tree. If a syntax error is encountered, a PhpParser\Error exception will be thrown by default:

<?php
use PhpParser\Error;
use PhpParser\ParserFactory;

$code = <<<'CODE'
<?php
function printLine($msg) {
    echo $msg, "\n";
}
printLine('Hello World!!!');
CODE;

$parser = (new ParserFactory())->createForHostVersion();

try {
    $stmts = $parser->parse($code);
    // $stmts is an array of statement nodes
} catch (Error $e) {
    echo 'Parse Error: ', $e->getMessage(), "\n";
}

A parser instance can be reused to parse multiple files.

Node dumping

To dump the abstract syntax tree in human-readable form, a NodeDumper can be used:

<?php
use PhpParser\NodeDumper;

$nodeDumper = new NodeDumper;
echo $nodeDumper->dump($stmts), "\n";

For the sample code from the previous section, this will produce the following output:

array(
    0: Stmt_Function(
        attrGroups: array(
        )
        byRef: false
        name: Identifier(
            name: printLine
        )
        params: array(
            0: Param(
                attrGroups: array(
                )
                flags: 0
                type: null
                byRef: false
                variadic: false
                var: Expr_Variable(
                    name: msg
                )
                default: null
            )
        )
        returnType: null
        stmts: array(
            0: Stmt_Echo(
                exprs: array(
                    0: Expr_Variable(
                        name: msg
                    )
                    1: Scalar_String(
                        value:

                    )
                )
            )
        )
    )
    1: Stmt_Expression(
        expr: Expr_FuncCall(
            name: Name(
                name: printLine
            )
            args: array(
                0: Arg(
                    name: null
                    value: Scalar_String(
                        value: Hello World!!!
                    )
                    byRef: false
                    unpack: false
                )
            )
        )
    )
)

You can also use the php-parse script to obtain such a node dump by calling it either with a file name or code string:

vendor/bin/php-parse file.php
vendor/bin/php-parse "<?php foo();"

This can be very helpful if you want to quickly check how certain syntax is represented in the AST.

Node tree structure

Looking at the node dump above, you can see that $stmts for this example code is an array of two nodes, a Stmt_Function and a Stmt_Expression. The corresponding class names are:

Stmt_Function -> PhpParser\Node\Stmt\Function_
Stmt_Expression -> PhpParser\Node\Stmt\Expression

The additional _ at the end of the first class name is necessary, because Function is a reserved keyword. Many node class names in this library have a trailing _ to avoid clashing with a keyword.

As PHP is a large language there are approximately 140 different nodes. In order to make working with them easier they are grouped into three categories:

PhpParser\Node\Stmts are statement nodes, i.e. language constructs that do not return a value and can not occur in an expression. For example a class definition is a statement. It doesn't return a value, and you can't write something like func(class A {});.
PhpParser\Node\Exprs are expression nodes, i.e. language constructs that return a value and thus can occur in other expressions. Examples of expressions are $var (PhpParser\Node\Expr\Variable) and func() (PhpParser\Node\Expr\FuncCall).
PhpParser\Node\Scalars are nodes representing scalar values, like 'string' (PhpParser\Node\Scalar\String_), 0 (PhpParser\Node\Scalar\LNumber) or magic constants like __FILE__ (PhpParser\Node\Scalar\MagicConst\File). All PhpParser\Node\Scalars extend PhpParser\Node\Expr, as scalars are expressions, too.
There are some nodes not in either of these groups, for example names (PhpParser\Node\Name) and call arguments (PhpParser\Node\Arg).

The Node\Stmt\Expression node is somewhat confusing in that it contains both the terms "statement" and "expression". This node distinguishes expr, which is a Node\Expr, from expr;, which is an "expression statement" represented by Node\Stmt\Expression and containing expr as a sub-node.

Every node has a (possibly zero) number of subnodes. You can access subnodes by writing $node->subNodeName. The Stmt\Echo_ node has only one subnode exprs. So in order to access it in the above example you would write $stmts[0]->exprs. If you wanted to access the name of the function call, you would write $stmts[0]->exprs[1]->name.

All nodes also define a getType() method that returns the node type. The type is the class name without the PhpParser\Node\ prefix and \ replaced with _. It also does not contain a trailing _ for reserved-keyword class names.

It is possible to associate custom metadata with a node using the setAttribute() method. This data can then be retrieved using hasAttribute(), getAttribute() and getAttributes().

By default, the parser adds the startLine, endLine, startTokenPos, endTokenPos, startFilePos, endFilePos and comments attributes. comments is an array of PhpParser\Comment[\Doc] instances.

The pre-defined attributes can also be accessed using getStartLine() instead of getAttribute('startLine'), and so on. The last doc comment from the comments attribute can be obtained using getDocComment().

Pretty printer

The pretty printer component compiles the AST back to PHP code according to a specified scheme. Currently, there is only one scheme available, namely PhpParser\PrettyPrinter\Standard.

use PhpParser\Error;
use PhpParser\ParserFactory;
use PhpParser\PrettyPrinter;

$code = "<?php echo 'Hi ', hi\\getTarget();";

$parser = (new ParserFactory())->createForHostVersion();
$prettyPrinter = new PrettyPrinter\Standard();

try {
    // parse
    $stmts = $parser->parse($code);

    // change
    $stmts[0]         // the echo statement
          ->exprs     // sub expressions
          [0]         // the first of them (the string node)
          ->value     // it's value, i.e. 'Hi '
          = 'Hello '; // change to 'Hello '

    // pretty print
    $code = $prettyPrinter->prettyPrint($stmts);

    echo $code;
} catch (Error $e) {
    echo 'Parse Error: ', $e->getMessage(), "\n";
}

The above code will output:

echo 'Hello ', hi\getTarget();

As you can see, the source code was first parsed using PhpParser\Parser->parse(), then changed and then again converted to code using PhpParser\PrettyPrinter\Standard->prettyPrint().

The prettyPrint() method pretty prints a statements array. It is also possible to pretty print only a single expression using prettyPrintExpr().

The prettyPrintFile() method can be used to print an entire file. This will include the opening <?php tag and handle inline HTML as the first/last statement more gracefully.

There is also a pretty-printing mode which retains formatting for parts of the AST that have not been changed, which requires additional setup.

Read more: Pretty printing documentation

Node traversal

The above pretty printing example used the fact that the source code was known and thus it was easy to write code that accesses a certain part of a node tree and changes it. Normally this is not the case. Usually you want to change / analyze code in a generic way, where you don't know how the node tree is going to look like.

For this purpose the parser provides a component for traversing and visiting the node tree. The basic structure of a program using this PhpParser\NodeTraverser looks like this:

use PhpParser\NodeTraverser;
use PhpParser\ParserFactory;
use PhpParser\PrettyPrinter;

$parser        = (new ParserFactory())->createForHostVersion();
$traverser     = new NodeTraverser;
$prettyPrinter = new PrettyPrinter\Standard;

// add your visitor
$traverser->addVisitor(new MyNodeVisitor);

try {
    $code = file_get_contents($fileName);

    // parse
    $stmts = $parser->parse($code);

    // traverse
    $stmts = $traverser->traverse($stmts);

    // pretty print
    $code = $prettyPrinter->prettyPrintFile($stmts);

    echo $code;
} catch (PhpParser\Error $e) {
    echo 'Parse Error: ', $e->getMessage();
}

The corresponding node visitor might look like this:

use PhpParser\Node;
use PhpParser\NodeVisitorAbstract;

class MyNodeVisitor extends NodeVisitorAbstract {
    public function leaveNode(Node $node) {
        if ($node instanceof Node\Scalar\String_) {
            $node->value = 'foo';
        }
    }
}

The above node visitor would change all string literals in the program to 'foo'.

All visitors must implement the PhpParser\NodeVisitor interface, which defines the following four methods:

public function beforeTraverse(array $nodes);
public function enterNode(\PhpParser\Node $node);
public function leaveNode(\PhpParser\Node $node);
public function afterTraverse(array $nodes);

The beforeTraverse() method is called once before the traversal begins and is passed the nodes the traverser was called with. This method can be used for resetting values before traversal or preparing the tree for traversal.

The afterTraverse() method is similar to the beforeTraverse() method, with the only difference that it is called once after the traversal.

The enterNode() and leaveNode() methods are called on every node, the former when it is entered, i.e. before its subnodes are traversed, the latter when it is left.

All four methods can either return the changed node or not return at all (i.e. null) in which case the current node is not changed.

The enterNode() method can additionally return the value NodeVisitor::DONT_TRAVERSE_CHILDREN, which instructs the traverser to skip all children of the current node. To furthermore prevent subsequent visitors from visiting the current node, NodeVisitor::DONT_TRAVERSE_CURRENT_AND_CHILDREN can be used instead.

Both methods can additionally return the following values:

NodeVisitor::STOP_TRAVERSAL, in which case no further nodes will be visited.
NodeVisitor::REMOVE_NODE, in which case the current node will be removed from the parent array.
NodeVisitor::REPLACE_WITH_NULL, in which case the current node will be replaced with null.
An array of nodes, which will be merged into the parent array at the offset of the current node. I.e. if in array(A, B, C) the node B should be replaced with array(X, Y, Z) the result will be array(A, X, Y, Z, C).

Instead of manually implementing the NodeVisitor interface you can also extend the NodeVisitorAbstract class, which will define empty default implementations for all the above methods.

Read more: Walking the AST

The NameResolver node visitor

One visitor that is already bundled with the package is PhpParser\NodeVisitor\NameResolver. This visitor helps you work with namespaced code by trying to resolve most names to fully qualified ones.

For example, consider the following code:

use A as B;
new B\C();

In order to know that B\C really is A\C you would need to track aliases and namespaces yourself. The NameResolver takes care of that and resolves names as far as possible.

After running it, most names will be fully qualified. The only names that will stay unqualified are unqualified function and constant names. These are resolved at runtime and thus the visitor can't know which function they are referring to. In most cases this is a non-issue as the global functions are meant.

Additionally, the NameResolver adds a namespacedName subnode to class, function and constant declarations that contains the namespaced name instead of only the shortname that is available via name.

Read more: Name resolution documentation

Example: Converting namespaced code to pseudo namespaces

A small example to understand the concept: We want to convert namespaced code to pseudo namespaces, so it works on 5.2, i.e. names like A\\B should be converted to A_B. Note that such conversions are fairly complicated if you take PHP's dynamic features into account, so our conversion will assume that no dynamic features are used.

We start off with the following base code:

use PhpParser\ParserFactory;
use PhpParser\PrettyPrinter;
use PhpParser\NodeTraverser;
use PhpParser\NodeVisitor\NameResolver;

$inDir  = '/some/path';
$outDir = '/some/other/path';

$parser        = (new ParserFactory())->createForNewestSupportedVersion();
$traverser     = new NodeTraverser;
$prettyPrinter = new PrettyPrinter\Standard;

$traverser->addVisitor(new NameResolver); // we will need resolved names
$traverser->addVisitor(new NamespaceConverter); // our own node visitor

// iterate over all .php files in the directory
$files = new \RecursiveIteratorIterator(new \RecursiveDirectoryIterator($inDir));
$files = new \RegexIterator($files, '/\.php$/');

foreach ($files as $file) {
    try {
        // read the file that should be converted
        $code = file_get_contents($file->getPathName());

        // parse
        $stmts = $parser->parse($code);

        // traverse
        $stmts = $traverser->traverse($stmts);

        // pretty print
        $code = $prettyPrinter->prettyPrintFile($stmts);

        // write the converted file to the target directory
        file_put_contents(
            substr_replace($file->getPathname(), $outDir, 0, strlen($inDir)),
            $code
        );
    } catch (PhpParser\Error $e) {
        echo 'Parse Error: ', $e->getMessage();
    }
}

Now lets start with the main code, the NamespaceConverter. One thing it needs to do is convert A\\B style names to A_B style ones.

use PhpParser\Node;

class NamespaceConverter extends \PhpParser\NodeVisitorAbstract
{
    public function leaveNode(Node $node) {
        if ($node instanceof Node\Name) {
            return new Node\Name(str_replace('\\', '_', $node->toString()));
        }
    }
}

The above code profits from the fact that the NameResolver already resolved all names as far as possible, so we don't need to do that. We only need to create a string with the name parts separated by underscores instead of backslashes. This is what str_replace('\\', '_', $node->toString()) does. (If you want to create a name with backslashes either write $node->toString() or (string) $node.) Then we create a new name from the string and return it. Returning a new node replaces the old node.

Another thing we need to do is change the class/function/const declarations. Currently they contain only the shortname (i.e. the last part of the name), but they need to contain the complete name including the namespace prefix:

use PhpParser\Node;
use PhpParser\Node\Stmt;

class NodeVisitor_NamespaceConverter extends \PhpParser\NodeVisitorAbstract
{
    public function leaveNode(Node $node) {
        if ($node instanceof Node\Name) {
            return new Node\Name(str_replace('\\', '_', $node->toString()));
        } elseif ($node instanceof Stmt\Class_
                  || $node instanceof Stmt\Interface_
                  || $node instanceof Stmt\Function_) {
            $node->name = str_replace('\\', '_', $node->namespacedName->toString());
        } elseif ($node instanceof Stmt\Const_) {
            foreach ($node->consts as $const) {
                $const->name = str_replace('\\', '_', $const->namespacedName->toString());
            }
        }
    }
}

There is not much more to it than converting the namespaced name to string with _ as separator.

The last thing we need to do is remove the namespace and use statements:

use PhpParser\Node;
use PhpParser\Node\Stmt;
use PhpParser\NodeVisitor;

class NodeVisitor_NamespaceConverter extends \PhpParser\NodeVisitorAbstract
{
    public function leaveNode(Node $node) {
        if ($node instanceof Node\Name) {
            return new Node\Name(str_replace('\\', '_', $node->toString()));
        } elseif ($node instanceof Stmt\Class_
                  || $node instanceof Stmt\Interface_
                  || $node instanceof Stmt\Function_) {
            $node->name = str_replace('\\', '_', $node->namespacedName->toString();
        } elseif ($node instanceof Stmt\Const_) {
            foreach ($node->consts as $const) {
                $const->name = str_replace('\\', '_', $const->namespacedName->toString());
            }
        } elseif ($node instanceof Stmt\Namespace_) {
            // returning an array merges is into the parent array
            return $node->stmts;
        } elseif ($node instanceof Stmt\Use_) {
            // remove use nodes altogether
            return NodeVisitor::REMOVE_NODE;
        }
    }
}

That's all.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

2_Usage_of_basic_components.markdown

2_Usage_of_basic_components.markdown

Usage of basic components

Bootstrapping

Parsing

Node dumping

Node tree structure

Pretty printer

Node traversal

The NameResolver node visitor

Example: Converting namespaced code to pseudo namespaces

Files

2_Usage_of_basic_components.markdown

Latest commit

History

2_Usage_of_basic_components.markdown

File metadata and controls

Usage of basic components

Bootstrapping

Parsing

Node dumping

Node tree structure

Pretty printer

Node traversal

The NameResolver node visitor

Example: Converting namespaced code to pseudo namespaces