Complex Variable Names

[eylvisaker]

Is your feature request related to a problem? Please describe.
I'd like to be able to have the variable system within YarnSpinner be able to handle some more complex variable names, so that I can use it to interact with objects within my game. For example, an if statement like either of these would be really useful to me:

<<if $prince_handsome.is_alive >>
<<if ${prince_handsome.is_alive} >>

Both of the above fail to parse with token recognition errors. The first considers the . to be the start of an unrecognized token, and the second rejects the ${ as an unrecognized token.

Describe the solution you'd like
What I'd like is for YarnSpinner to see the token ${ and consider everything between that token and } as a variable name, and then pass it to the variable storage so that I can process it. I will handle the processing of object properties, etc. in my own back end code and the proper type conversions when returning values to YarnSpinner. I really just need it to allow me to somehow specific more complex variable names which I can then evaluate as expressions in another system.

(FYI, I intend to do evaluate the variable names by passing them as an expression to my Lua engine, because it has all the correct context for the objects I intend to use within my yarn files.)

Describe alternatives you've considered
I've tried a variety of other characters as delimiters between the object and its property but nothing really seems to work (except underscores, but that's no good for my purposes because they are part of valid variable identifiers).

I'd be happy to fiddle with the YarnSpinner source code myself to implement this, if someone could offer any pointers on what is the right place in the source to start looking at, and any gotchas I should be aware of.

[eylvisaker]

I was looking at the lexer grammar file, it looks like it might be a pretty easy change around line 350, but I am not sure how to run the antlr tool to regenerate the C# file. Is there any documentation on that?

[desplesda]

I use the ANTLR Visual Studio Code extension, which 1. is a pleasant way to interact with ANTLR grammars, and also automatically regenerates the lexer and parser when they change: https://github.com/mike-lischke/vscode-antlr4

For the specific grammar you're proposing: I'm in favour of the feature, but I think I'd like to reserve that specific grammar (${var_with arbitrary.text}) (in particular, the use of braces) for a different feature. My reasoning is that we already use braces inline in the text to represent "this is an expression", and if you wanted to cite a such variable inside a line, you'd end up with nested braces in which each pair has a totally different syntactic and semantic meaning.

As an alternative, can I suggest Swift's approach, which the use of backticks:

$`arbitrary.text here`

(This is how Swift handles the situation in which you want to name a variable after a keyword, like internal; it strikes me that the feature is similar enough to this use case, and the use of quote marks more clearly suggests a feature to do with text parsing rather than, for example, a more formal structured data model that isn't currently in the language.

[eylvisaker]

OK makes sense, yeah backticks are perfect. I'll test this out and send a PR when I'm ready.

[McJones]

If you don't want to do any modification to the grammar and just have a solution right this second you could build a function that takes in a variable and the property name you want to access inside and returns a bool once you handle that function, eg:

<<if bool_property_accessor($prince_handsome, "is_alive")>>

I think the real concern here is the the ID token supports a-z, A-Z, 0-9, and _.
Is it worth looking at allowing more symbols in here?
I know we have intentionally not put in support for the . operator so that we can reserve it if we ever want to be able to create structured data types in Yarn.
So what symbols should be allowed?
I would say the ones that shouldn't be allowed are: + - / * % = # . { } [ ] ( ) making the change to the grammar for ID:

ID : [a-zA-Z] ~( '$' | '+' | '-' | '/' | '*' | '%' | '=' | '#' | '.' | '{' | '}' | '[' | ']' | '(' | ')' | '/r' | '/n' | ' ' | '/t')* ;

This doesn't solve the initial issue of the . still won't work (and I think it shouldn't ever allow a . in a variable name) but it means you could define your own operator, such as the ~.

With all of that said I think the $`arbitrary text` in expression is a good idea and worth having anyways just so that if people do need to use reserved symbols they can.

[eylvisaker]

Yeah I don't need something that immediate that I can't invest in an improvement for the parser. I definitely don't want to go with a syntax that complicated. I want the dialog code to be as readable as possible. I thought about adding an eval function but then my yarn scripts are going to be littered with these eval statements and readability will suffer.

My scripting code is going to be a mixture of yarn spinner and Lua - so what's I'm trying to accomplish is to have a way to use my existing Lua engine to handle all the object/property parsing so that I only have to maintain one system, and more importantly - the syntax and documentation for how to access object properties can be the same for both the Lua code and the Yarn files.

Ideally what I want is to have a syntax to escape an arbitrary expression from Yarn Spinner so that I can evaluate it on the Lua side and return the result to the yarn program. For that, symbols like { } ( ) . are really useful. Using backticks to as escape characters gives me that flexibility pretty well I think.

[desplesda]

This is an addition to the language that I’d like to see. Given that its inclusion wouldn’t be a breaking change to the grammar (because backticks in a variable expression are currently not permitted, so permitting them wouldn’t break existing code), I don’t think we need to add this to 2.0’s queue; instead, I’d like to add this to a minor release, possibly 2.1.

[eylvisaker]

Here's a PR: #269
Also added documentation: YarnSpinnerTool/Docs-1.0#5

[McJones]

I discussed this a bit on the linked PR but the tldr is the idea is not as needed AND has more unintentional side-effects than we first realised when we started the discussion around this idea back on the discord.
The desired functionality is 99% captured as part of the work done in 1f41dfc.

As such I am closing this issue.
Thanks to everyone involved.