Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenMetadata 0.12.1.2 available; fixes data profiling with Trino #1114

Closed
MichaelTiemannOSC opened this issue Oct 7, 2022 · 6 comments
Closed
Assignees

Comments

@MichaelTiemannOSC
Copy link

Describe the Problem

OpenMetadata has a data profiling feature (one of the two major surface features of the ingestion functionality). It was broken for Trino before 0.12.1.2. We would like to be able to use this major feature of OpenMetadata, that presently works not at all.

The updated package is ready to go on PyPi: pip install openmetadata-ingestion (now version 0.12.1.2)

Steps to Reproduce

  1. Go to https://openmetadata-openmetadata.apps.odh-cl2.apps.os-climate.org/service/databaseServices/OSC-Trino/ingestions
  2. Click the 'Run' tab of the profiler_essd ingestion (which will run the profiler)
  3. Notice that it fails, due to an unexpected '%' character

Expected behaviour

OpenMetadata should be able to profile data that sits inside of Trino.

Screenshots

N/A

Additional context

This is a blocker for data profiling.

@durandom
Copy link
Member

/assign @redmikhail

@MichaelTiemannOSC
Copy link
Author

I just learned about this from OM channel: ./bootstrap/bootstrap-storage.sh migrate-all

@MichaelTiemannOSC
Copy link
Author

OpenMetadata 0.12.2 was released a few weeks ago. 0.13-preview is available as of yesterday, and may be the best way to align our development efforts with OM.

xref #1121
xref os-climate/os_c_data_commons#183

@redmikhail
Copy link

/assign @rynofinn

@eb-oss eb-oss assigned eb-oss and unassigned rynofinn Jan 26, 2023
@eb-oss
Copy link
Contributor

eb-oss commented Jan 26, 2023

@MichaelTiemannOSC How does this square with issue os-climate/os_c_data_commons#183 ? Do we need both open? We're actively working on that issue, but I came across this one and was wondering if it needs any separate handling.

@MichaelTiemannOSC
Copy link
Author

I see this as the urgent need to get a good current version of OM (that can support data profiling), so platform related. The other issue is more developer community oriented. It depends on this, and will be enabled for work once we have the OM we need. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants