Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement 'schema translate' without copying data #33

Open
thomasdfischer opened this issue May 12, 2018 · 0 comments
Open

implement 'schema translate' without copying data #33

thomasdfischer opened this issue May 12, 2018 · 0 comments
Labels
backlog help wanted Extra attention is needed

Comments

@thomasdfischer
Copy link
Member

thomasdfischer commented May 12, 2018

Problem

There is an edge case in schema translate where, if any part of the input data is parsed as a map[interface{}]interface{}, then data must be copied. This is because all of the JSON/YAML/TOML/etc. packages do not support encoding map[interface{}]interface{} objects. To allow translate to still work, the map[interface{}]interface{} object is converted to a map[string]interface{} object by

  1. Making a new map[string]interface{}
  2. Looping through the original map, inserting all the values into the new map (which has string keys instead of interface{} keys)

The conversion is so costly that translating an 8MB file takes 8 seconds on my ThinkPad T450s. This is not acceptable.

Possible Solutions

  1. Write custom decoders and encoders which automatically attempt to convert interface{} keys to strings.

  2. Find someone who knows more about Go's type system to refactor the current code so that data is cast to the correct type without copying it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlog help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant