Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add variant and metrics to events_plot mapping #223

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

fhaase2
Copy link

@fhaase2 fhaase2 commented Dec 8, 2023

We had some issues with duplicated plots in our ClearML deployment.

The reason was that Elastiscearch infered text for these fields and we do an aggregation query.

This PR sets them explicity to type keyword, which should make it more stable given that field inference works slightly different in different Elasticsearch versions.

@Ponsaille
Copy link

Happy new year!
Were are we on this PR, would be really nice to have this additional stability! @jkhenning

@jkhenning
Copy link
Member

Hi @Ponsaille, sorry for the delay 🙏
How would this work is you only add these to the schema? The server won't store them there, I think

@evg-allegro
Copy link
Contributor

Hi @Ponsaille , @fhaase2 which version of ES do you use? We already have this mapping in events.json file and it is applied to all the events indices. While more specific mappings just add missing fields for specific event types. Thus the resulting indices have the mapping that is combination of events.json and then a more specific one like events_plot.json or events_log.json etc. This works for us in ES versions from 5.x to 7.x including.
When you restart the apiserver container the mappings are applied as part of the apply_mappings script. Do you see any errors from it in the apiserver log?

@fhaase2
Copy link
Author

fhaase2 commented Jan 25, 2024

We use ES 8.5.1. And per default, the indices created automatically by ClearML have the wrong mappings in this version. So this PR makes sure that the created index has correct mapping, even in ES > 7.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants