Add ability to use the generated parser instead of the source grammar

[akovari]

These changes add a new option to the grammar configuration and make it possible to use the generated parser instead of the source grammar (known limitation currently).

It is not bulletproof and won't work in many cases (when Java is not the target language for example), but this can still be helpful in many situations. If there is no generated parser, the old way of using the source grammar is used as a fallback. This shouldn't really break anything working now, it is just a new option.

Please consider this addition and let me know if it needs additional work, or if it is simply against the philosophy of this plugin and cannot be merged.

[bjansen]

While I believe the concept is interesting, I also think it has several drawbacks that will prevent a "correct" implementation: loading project dependencies in the IDE classpath is something that IntelliJ never does AFAIK. There's just too much risk of breaking everything:

• on projects with many dependencies, you might end up filling the non-heap memory of the JVM with class definitions
• the IDE and its plugins already provide lots of dependencies, what happens if your project also depends on another version of one of these dependencies?
• you load the classpath once and for all, which means if the user adds/removes dependencies from their project, the ANTLR plugin will be "out of sync"
• which leads to probable memory leaks, loaded classes are never unloaded

I believe these are reasons similar to those that pushed JetBrains to externalize their Maven support in a separate process. It's cleaner, and when something changes you just kill and restart the process and voila.

Unfortunately for us, this is a bit complicated to setup: we'd need to wrap the ANTLR runtime in a separate process and find a way to communicate with it. This means sending input, and receiving some kind of parsing result back. I'm not saying it's impossible, but I think it's a lot of work.

As for your proposed changes, I don't think we can accept them, sorry. We've already had stability problems with the current interpreter on bad grammars (infinite loops, OutOfMemoryExceptions), and I fear that messing with the IDE classpath is going to cause more problems.

[akovari]

Thank you for the feedback. Just to clarify, are you referring to a situation when your generated parser/lexer classes have too many dependencies? Which piece of the code do you think could cause the memory leak?

Are you talking about this piece?

OrderEnumerator.orderEntries(project).runtimeOnly().classes().usingCache().getPathsList().getPathList()

I didn't think this loads all classes, I thought it just lists whatever is on IDEA's project build path, but I might be wrong.

I certainly agree with you that this implementation is not bullet-proof, and sometimes it requires restarting IDEA to load the correct version of the class. If the classpath changes, then yes, it is expected that you have to rebuild the project. Maybe I could find a way to hook the loading of the parser class to the project compilation somehow. Perhaps that could avoid the need to restart the IDE?

In general, I am willing to maintain my own fork of this, since it is extremely useful for grammars I work with. It would be easier to have this merged, long-term, and I am willing to do additional work here if there is a chance of merging it eventually, but if your fear of these changes is too great, then I understand and please feel free to reject this.