Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support new snowplow data model #102

Open
sphinks opened this issue Feb 26, 2021 · 2 comments
Open

Support new snowplow data model #102

sphinks opened this issue Feb 26, 2021 · 2 comments

Comments

@sphinks
Copy link
Contributor

sphinks commented Feb 26, 2021

Describe the feature

It looks like Snowplow has switched to the new of processing raw data https://github.com/snowplow/data-models and call new data model as V1. Current implementation of DBT module does not suits it anymore. New data model should be supported/adopted by DBT snowplow module.

Describe alternatives you've considered

Possible workaround - create by DBT module all snowplow tables by parsing JSON fields in EVENTS table. E.g. create table com_snowplowanalytics_snowplow_web_page_1 by parsing field EVENTS.contexts_com_snowplowanalytics_snowplow_web_page_1

Who will this benefit?

All users who has using DBT and Snwoplow at the same time on their projects.

@sphinks
Copy link
Contributor Author

sphinks commented Mar 1, 2021

@jtcohen6 @drewbanin I'd like to get your insights regarding what is the best way to implement workaround? I have implemented workaround models to new snowflake data model and going to do PR. However, the main question how to configure such switch in logic. I see two options:

  1. Introduce some special variable (quite easy to implement, but not so clear for user what is it).
  2. Just reuse variables like 'snowplow:context:web_page': e.g. value of 'snowplow:events' (atomic.events) = 'snowplow:context:web_page', looks like that we are using snowflake layout with single table and start using workaround solution. (more clear for user, but could be fragile).

@sphinks
Copy link
Contributor Author

sphinks commented Jun 29, 2021

There is an update: Snowplow team is working on new DBT package: snowplow/snowplow#4466

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant