Skip to content
This repository has been archived by the owner on Jan 26, 2021. It is now read-only.

Output transformed data to public JSON files #44

Open
patricksheehan opened this issue Jul 25, 2020 · 3 comments
Open

Output transformed data to public JSON files #44

patricksheehan opened this issue Jul 25, 2020 · 3 comments

Comments

@patricksheehan
Copy link
Contributor

We want to provide easily-digestible data so folks can benefit from the transformations we apply, and the sources we integrate. To avoid hosting structure, the current proposal is to provide public JSON file(s) in our repo which are updated when we run our ETL.

@patricksheehan
Copy link
Contributor Author

Via @rypan CSV + json probably worthwhile depending on the user

@lucasmbrown
Copy link
Contributor

A couple notes on this:

  1. One option to create a JSON API is to just update a JSON file in a github repo every day. We can do the same with a CSV in a github repo, and post the data both as a CSV and as a JSON file.

  2. We haven't been making sure to add every field to a single dataframe/sheet, so we might need to recombine this and make sure we're including all the fields we create in a single file.

  3. Do we have a sense of whether anyone might use this data if we put it out there? I think it'd be super valuable to put it out there, just curious if we've generated any interest.

  4. By publishing an API we have a lot less flexibility on changing field names etc, because we are now supporting external users. So let's make sure we advertise that this is an alpha or beta API and things might be changing frequently, and also be thoughtful about trying not to change things too frequently if possible.

@devinhunt
Copy link
Collaborator

I could use these data sources for the site as well. I believe gh-pages can act as a CORS-open endpoint as well.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants