Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type mappings transformation #1154

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Commits on Nov 14, 2024

  1. Initial checkin for type mapping removal transformer and experimentin…

    …g w/ Jinjava.
    
    Major things to figure out: 1) how to deal with compositing parts of jinja templates (maybe it's just snippets vended as resources and a user writes their own top-level template) and 2) how to suppress reading any part of the json body when only the headers need to be transformed to prevent parsing the json/ndjson for every request through the system (probably minor though since most setups will probably be doing full-transforms on the vast majority of data flowing).
    
    Signed-off-by: Greg Schohn <[email protected]>
    gregschohn committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    21fb34b View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2024

  1. Further building out the jinjava and index type mappings transformers…

    … so that they can fit w/in the transformation ecosystems.
    
    Signed-off-by: Greg Schohn <[email protected]>
    gregschohn committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    926e6e8 View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2024

  1. Checkpoint on jinjava and loading template transformations.

    This supports preparsing templates and loading default or specific versions.  Those features are likely now in flux to be replaced with a better UX.
    
    Signed-off-by: Greg Schohn <[email protected]>
    gregschohn committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    9bd5c60 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2024

  1. Checkpoint add support for macros and feature flags for different par…

    …ts of jinjava transforms (see 'is_enabled').
    
    These changes will reduce the efficiency of this type of transform, but w/out it, it isn't clear how we'll utilize dynamic typing.  My hope is that template logic is small relative to documents - and compute is cheap, so these are good tradeoffs to make.
    
    Signed-off-by: Greg Schohn <[email protected]>
    gregschohn committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    f1e1d50 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2024

  1. Cleanup + added a "preserve" flag to let templates inline source node…

    …s after the templating...
    
    That preserve splicing not only improves efficiency, it also lets us pass messages through a template that might have non-textual data, like the ByteBufs, etc.
    
    Signed-off-by: Greg Schohn <[email protected]>
    gregschohn committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    dd98377 View commit details
    Browse the repository at this point in the history
  2. Add jinja-style regex matching to jinjava transformations.

    Signed-off-by: Greg Schohn <[email protected]>
    gregschohn committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    357a6a6 View commit details
    Browse the repository at this point in the history

Commits on Nov 23, 2024

  1. Checkpoint - I've added a new invoke_macro java function to lookup a …

    …macro by name and invoke it since jinjava can't deal with invoking macros with dynamic names.
    
    There are some rough tests in place that don't fully work.  The original ambition was to allow a user to specify both a route and an action for a given map in a macro but since macros only output rendered content, they're not well-suited to chain complex data (like regex matches) between macros.  As such, there will be significant design changes to the jinja routing mechanisms.
    
    Signed-off-by: Greg Schohn <[email protected]>
    gregschohn committed Nov 23, 2024
    Configuration menu
    Copy the full SHA
    12c0cfa View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2024

  1. Configuration menu
    Copy the full SHA
    e8dfb3f View commit details
    Browse the repository at this point in the history
  2. Assorted jinjava template changes for replayer...

    Switched out Google's re2j for Java's implementation.  I didn't realize that the java implementation is closer to python regexes and that re2j (which I had thought was closer to python).
    Minor tweaks to the route jinja template to make it easier to read when it's invoked.
    Switched the replayer template to use the route functionality, which is considerably easier to get one's head around.
    TODO -
    * I'd like to make the feature flag names the same as the macro name for the route when the feature flag name is omitted.
    * The test cases for the type mappings sanitization need a lot of work.
    * When indices are created that DO NOT use type mappings, we should handle those more gracefully.
    
    Signed-off-by: Greg Schohn <[email protected]>
    gregschohn committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    93dc5c8 View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2024

  1. Setup regex type mappings where one can specify index and types as re…

    …gexes and the target index can be specified with backreferences.
    
    Notice that there's no way to discard data with regexes like there is with the type mappings (and the static type mappings do yield to regexes when no match was found).
    Other changes:
    The regex capture filter now has a guava cache for compiled patterns.
    When static and regex mappings are null, we'll use a default (.*), (.*) -> \1_\2 mapping.
    Enhanced some tests to pick up more corner cases.
    
    Signed-off-by: Greg Schohn <[email protected]>
    gregschohn committed Nov 25, 2024
    Configuration menu
    Copy the full SHA
    4b32d42 View commit details
    Browse the repository at this point in the history