Tidier.jl roadmap

[kdpsingh]

I thought it would be useful to have a pinned issue that summarizes which function/macros we are working on building out, and which tidyverse (or related) package they comes from. This doesn’t represent the entire set of functions we need to capture but is intended to give a sense of direction for this project.

Note one key difference in function names below between Tidier.jl and tidyverse: Tidier.jl functions relating to data types are named after the Julia types and not after R types. This is because the data types aren't consistent across languages, and Julia allows for more granularity than R. For example, we plan to use as_string() in Tidier.jl rather than as_character() because strings are collections of characters in Julia.

dplyr

• @glimpse - @zhezhaozz
• @relocate - @zhezhaozz
• @drop_na - @kdpsingh
• lead and lag - @kdpsingh
• as_integer, as_float, and as_string
• is_integer, is_float, and is_string

tidylog

• support for all functions - @kdpsingh

forcats

We are using the categorical type from CategoricalArrays.jl, so functions will be prefixed with cat_* instead of fct_*.

• categorical and as_categorical
• cat_rev
• cat_reorder
• cat_infreq
• cat_relevel
• cat_lump

ggplot2

• Lots to handle, will do initial implementation using AlgebraOfGraphics.jl - @kdpsingh

lubridate

• support for mdy, dmy, and ymd functions, along with counterparts that support *_hms.

Developer resources

• Put together style guide for macros in Tidier.jl
• Develop contributor guide and code of conduct
• Adopt JuliaFormatter style for this package, keeping in mind tidyverse style guide conventions

Strategic decisions to revisit

• Should we spin this package off into its own org?
• Should we convert Tidier.jl into a meta-package that re-exports underlying packages TidierData.jl and TidierPlots.jl? This is something we may want to consider if other packages start to depend on Tidier.jl. Once AlgebraOfGraphics and CairoMakie are included as dependencies, this package will get quite hefty.

[pdimens]

The AOG/CairoMakie deps will be a very appreciable increase in TTFX. I think it makes sense to follow the metapackage format. A separate org would be useful for organization and democratization, which would have a lot of value to the growing number of contributors.

I like the idea of Tidier.jl reexporting TidyData.jl and TidyPlots.jl (and whatever else). Notably, it makes more sense to me, linguistically, for the sub-packages to be Tidy_____.jl and not Tidier____.jl. My rationale is that tidydata and tidyplots would make your workflow tidier. And yes, I'd offer to make the logos for those ☺️

[kdpsingh]

Thanks @pdimens. My rationale for TidierData instead of TidyData was that if we set up a Tidier org, then it makes more sense to me for the connected packages to have a Tidier prefix.

I was curious to see how Julia 1.9 will impact TTFX for plotting. I know including plotting packages will increase install time by quite a bit, but I'm hopeful that TTFX may be okay with Julia 1.9 since I think Makie uses SnoopPrecompile.

And yes, absolutely would love your magic touch on logos 🤩!

[kdpsingh]

As a side note, the name "Tidier" is already taken as a GitHub organization. Considering "TidierOrg" as an alternative or "TidierJulia". Or may reach out to the current "Tidier" org to see if they are willing to give up the name.

[ViralBShah]

Just a note - Usually Julia orgs on github have Julia as the prefix. Not that you have to do that too, but JuliaTidier sounds better to my ear than TidierJulia.

[kdpsingh]

Thanks @ViralBShah. Appreciate the note! I was only thinking TidierJulia bc "tidier" is an adjective (as compared to JuliaData or JuliaPlots).

Am leaning towards just Tidier (if I can get permission from existing owner) or TidierOrg (similar to MakieOrg).

If this repo is transferred to an org, is there anything special I need to do when I register it?

No urgency and happy to cross that bridge when we get there.

[ViralBShah]

Github will forward the old URL - but we should change the URL in the General registry after the transition.

[pdimens]

@kdpsingh I should have time this week to draft up some icons for the new packages (Dates, Cats, Strings). The big question is, would you like the background color of the icons to follow the same blue, or are you interested in different colors?

[kdpsingh]

Thanks @pdimens! My thinking was that since TidierPlots already has the same shade of blue, we should just keep the same blue for all the logos so that they are instantly recognizable as being part of Tidier. I'm open to different colors as well. What do you think?

Also, 2 other things would be really helpful if you have the bandwidth.

1. Can we create a new TidierData.jl logo that is identical to the current Tidier.jl logo except with the text reading as TidierData.jl?

2. Since Tidier.jl will be transformed into a meta-package that will include plotting, we can add scatterplot points to the Tidier.jl logo so that it combines data analysis and plotting? (Kind of like the TidierOrg logo)

Appreciate your creativity and time, and totally open to your ideas!

[pdimens]

1. Not a problem to update tidier/data.
2. I can keep the blue, or I can retroactively recolor all the constituent packages and keep the TidierOrg as blue. I'll first stick to the blue and bc I agree it's something of a trademark.

[kdpsingh]

Feel free to make it colorful if you'd like! As long as the main Tidier.jl one stays as the same blue.

It may be cool if we end up making stickers to have different-colored ones. Totally up to you!

[pdimens]

Some drafts I slapped together today

[kdpsingh]

Love it! Only suggestion would be to make the background colors more different from one another so that it produces a more rainbow-like effect. Right now the pink and purple are kind of close to one another, as are the two greenish ones.

[kdpsingh]

Also tagging @drizk1 to take a look.

[drizk1]

@pdimens these logos look great thank you! I do agree with Karandeep that a little more contrast (perhaps shifting to a navy for one ?) between them might be nice to help them really pop when they're in print one day. Thanks for the great work tho!

[kdpsingh]

Also, last thought: feel free to change the color on the TidierPlots logo too. The only one that should remain the same color is Tidier.jl. Thank you so, so much! So excited.

[pdimens]

How about these:

[kdpsingh]

Looks great to me! Only thing I'd consider is making TidierPlots background darker so that the white lettering is more readable. The grey background is actually kind of funny since it's the default background theme in ggplot.

@drizk1 and @rdboyes what do you think? If everyone agrees, feel free to tweak and share files with me or add to PRs.

[kdpsingh]

Looking at it again, I can definitely read TidierPlots so free to leave as-is or just darken the outer portion slightly. I trust your judgment!

[pdimens]

The grey background was a whimsical nod to ggplots default theme, yeah. Haha. I'll make it a bit darker

[drizk1]

These look great! The softness of the colors is really nice.

[pdimens]

Here are 3 variants. My favorite is number 2

option 1

option 2

option 3

[rdboyes]

I like number 1, but they all look good!

[kdpsingh]

I like number 3 but I'm also good with number 2. @rdboyes, thoughts?

[pdimens]

Lol fight amongst yourselves.

[kdpsingh]

This package is @rdboyes's baby so I would go with number 1.

[pdimens]

aiight. PRs incoming

[kdpsingh]

Yay, awesome. Question: Can we also update the Tidier.jl logo to include plotted dots (to indicate that it’s a meta-package that will enable data analysis and plotting?) It’s essentially going to re-export all of the other Tidier* packages.

[pdimens]

@kdpsingh like dis?

[kdpsingh]

Purrrrerfect (as the TidierCats would say). Let's do it!

[jdiaz97]

Not part of the tidyverse, but the '"rio" package from R is quite nice.
Does Julia have something like that? if not, do you guys think an implementation would be good?

[kdpsingh]

It does!

FileIO already handles multiple different formats with a single function for reading and another for writing.

https://juliaio.github.io/FileIO.jl/stable/

[kdpsingh]

Here are all the supported formats: https://juliaio.github.io/FileIO.jl/stable/registry/#Registry-table

[frankiethull]

Hi!

I am building with some tables in Julia and was thinking about you all... Is a TidierTables.jl part of the roadmap? Was thinking along the lines of of the gt package in R.

[kdpsingh]

There's a package PrettyTables.jl that is currently undergoing a remake inspired by the {gt} R package: https://discourse.julialang.org/t/current-state-and-the-future-of-prettytables-jl/118455

I would check in on that package.