-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
/locations too strict in checking station coordinates #105
Comments
I assume the check you are referring to is the one where we check that In some sense I think this check failing is a good indication that the ingested data is "wrong". |
I know @jo-asplin-met-no is currently working on a new gRPC method for use in |
But I think having two divergent time series bringing down the entire |
Ok, temporary solution. We check that the diverging points are not too far apart, if within tolerance we sort the coordinates and choose the first one? |
At MET Norway it is not unusual that the lat,lon location of a station can change from time to time (though I'm not sure about exactly how often it happens - I could probably dig up some stats). There are essentially two reasons for this: 1) a station is physically moved (for various reasons - in 1940 the station at Blindern (our Oslo office) was moved 300m), and 2) the lat or lon component is slightly updated, typically to comply with some (internal) standard wrt. number of decimal places. It was 2) that triggered this morning's test failure as a result of a GUI tool insisting to update the lon component of the station from four to five decimal places (from 19.4737 to 19.47367) even if the position was not even the metadata that my colleague wanted to update (he wanted to update the "short name" of the station, but had to "fix" the lon component too!). This just proves how vulnerable these things can be in practice whenever human editing is involved. Anyway, we need to support these cases, and I agree with Amund in preferring the most recent known position for a station, since that is most likely to "stabilize the system" quickly (at least wrt. both 1) and 2) at MET Norway I guess). The challenge then IMO is how to present/document deviations to the end user. How much of a problem will this be for an end user anyway? (Note: in MET Norway's case it is possible to distinguish between 1) and 2) in case that is relevant for the end user.) |
There is no concept of station in the store, only timeseries. The current But taking the last position in itself doesn't solve this problem, as there could be different positions with the same timestamp, from different timeseries/parameters. Anyway, I agree with proceeding if the positions are close enough. Any approach that gives a consistent output is fine with me, although it feels strange to use the "smallest" coordinate. |
I opened a PR with a draft for getting coordinates in a consistent manner. |
I updated Issue #107 with a detailed sketch of a possible solution where the data store takes responsibility. @Teddy-1000 and @lukas-phaf: please comment! (the temporary solution proposed by Amund is probably good to have for now anyway, though!) |
Stations can move or have their position updated to be more precise. As in the example below. I think we should disable this check for now, and find a better way to make sure the position does not diverge too much between time series. In this case the position have moved 1.2 meters, but breaks the entire /locations endpoint of the API. I think such a check should be made somewhere else. As we do not want data with errors accepted in to the system at all, and we should treat the datastore as a source of truth.
The text was updated successfully, but these errors were encountered: