Provide a Builder/Functions for creating ASTs without a CST

[nametable]

I find that a builder to create an AST programmatically would be helpful for generating code targeting a language built using Langium. My current approach is to use the CompositeGeneratorNode syntax to manually craft text which should be compliant with my language's syntax, however this can be cumbersome and prone to error.

My thought is that if a person implements a formatter for an AST, it should be able to produce the CST and thus spit out valid syntax. The construction of the AST node types however currently requires setting the parent $container of each AST node.

Is there a recommended way to do something like this already, or would adding AST node generator functions/utilities to Langium potentially make sense?

[msujew]

Hey @nametable,

My thought is that if a person implements a formatter for an AST, it should be able to produce the CST and thus spit out valid syntax.

Note that the current formatter implementation operates on the existing CST, and does not produce a CST (i.e. it simply shifts the existing text nodes, and does not create any nodes).

The construction of the AST node types however currently requires setting the parent $container of each AST node.

You can just set the $container property to undefined! and then run the linkContentToContainer function on everything.

Is there a recommended way to do something like this already, or would adding AST node generator functions/utilities to Langium potentially make sense?

The recommendation is to create a manual serializer right now (as you're already doing). What you're looking for is an automatic serializer. However, building a completely generic, grammar agnostic serializer is really complicated, and while it can be done (see Xtext's serializer), we currently don't have any plans to work on that.

[spoenemann]

See this article about developing a generic serializer: https://www.typefox.io/blog/lionweb-langium/

[nametable]

Interesting read, is there a current issue for tracking the design and implementation of such a serializer. I'm just trying to think of possible solutions, as this is an interesting problem.

For the comments, I wonder if "full fidelity parsing" as described here could be a solution to that. In his case, they used leading and trailing "trivia".

Also maybe its philosophical, but if the grammar rules define overlapping alternatives, does it matter which one is chosen during serialization, if parsing the produced text would produce the same AST again? 😅

[spoenemann]

There's no issue or plan yet. Maybe @JohannesMeierSE can say more.

In any case, it's a huge endeavor. Xtext has this feature, and it's very complex:
https://github.com/eclipse-xtext/xtext/tree/main/org.eclipse.xtext/src/org/eclipse/xtext/serializer