Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nested data #72

Open
SymbolixAU opened this issue Mar 29, 2020 · 7 comments
Open

nested data #72

SymbolixAU opened this issue Mar 29, 2020 · 7 comments

Comments

@SymbolixAU
Copy link
Contributor

SymbolixAU commented Mar 29, 2020

continuing this discussion

as well as list_columns, have a list_data, or combine_lists, or something, which will put all the list columns into a single data.frame column.

Will also need the reverse, to split_lsits or sommat.

And will only allow one-level of nesting.

@SymbolixAU
Copy link
Contributor Author

@mdsumner

before I dive into this, given your progress with anglr and your own developments, and the development in geovctrs / geom, would it be better to get a fully formed design for what this 'nested data' will look like, so it can help the whole spatial ecosystem?

@mdsumner
Copy link
Contributor

mdsumner commented Apr 1, 2020

Maybe, I definitely feel responsibility to do this - but the fact is I still don't have a "complete landscape" vision - I've really settled into building componentry, and leveraging data frame nesting and splitting and grouping - I'm much more into the tibble/tidyr approach than you are I see.

I like your colour_values() idiom here, but have worried me how specific that is and is kinda reinventing nested data frames. But, it's simpler - and totally trivial to convert back and forth, so I think that's fine. silicate isn't currently super suited to tracking data, it can store them and convert between forms - but there's a better way I think.

But, why are you working with sf anyway? Why not store the coordinates and the track attributes together in a df as is perfectly natural? Don't you end up creating work having to find the y values, and then the temperature values by a totally different mechanism? Why does sf help here? What format does mapdeck send into deck.gl?

@mdsumner
Copy link
Contributor

mdsumner commented Apr 1, 2020

Oh it's googlepolylines yeah? So that is totally streamlined and necessary, and you have to use a separate mechanism for the aesthetics. And how you have it is a neat generalization of the SF vertex model, so I think it's perfectly fine - and is something others can easily leverage.

@mdsumner
Copy link
Contributor

mdsumner commented Apr 1, 2020

Oh and sorry I totally forgot about the list_data option you already promised ...

@dcooley
Copy link
Owner

dcooley commented Apr 1, 2020

Oh it's googlepolylines yeah

actually, no, not any more.

That was my original way of reducing data size as it's sent to the browser, specifically for googleway, but, deck.gl's binary (columnar) format is more powerful & quicker, so now I send a long data.frame. Hence my sf_to_df() functions ;)

@dcooley
Copy link
Owner

dcooley commented Apr 1, 2020

So I'm not fixed to sf structures at all, I only need columns of data. But I've focused on sf purely because it's widely used at the moment.

@mdsumner
Copy link
Contributor

mdsumner commented Apr 1, 2020

Oh right, understood - thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants