Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate data processing from source data to csvs of regional scaling factors for transportation electrification #310

Open
1 task done
BainanXia opened this issue Sep 20, 2022 · 0 comments
Assignees
Labels
data collection Input data (demand, hydro/solar/wind) feature request Request for a new feature. (Only lives in Backlog)

Comments

@BainanXia
Copy link
Collaborator

BainanXia commented Sep 20, 2022

🚀

  • Is your feature request essential for your project?

Describe the workflow you want to enable

Currently, the intermediate csvs of regional scaling factors for urban areas and rural areas, which are the inputs of transportation electrification profile generation, are hosted in the repo. We would like to automate the data processing from source data to these csvs. Data download and processing are done in a semi manual way described in Breakthrough Energy Dropbox/BESExternalCollaborations/Transportation Electrification Modeling/VMT data processing from source data to csvs 09_20_22/

Describe your proposed implementation, if applicable

Likewise in other time series profile generation procedure, if the source data is stable, we can create a function to download the data and process on the fly, otherwise, we will host a copy of the source data in our blob storage container and go through the data intake process.

Additional context

To validate the implemented logic is correct, the csvs generated by the automatic process should be consistent with the csvs currently hosted in the repo.

@BainanXia BainanXia added data collection Input data (demand, hydro/solar/wind) feature request Request for a new feature. (Only lives in Backlog) labels Sep 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data collection Input data (demand, hydro/solar/wind) feature request Request for a new feature. (Only lives in Backlog)
Projects
None yet
Development

No branches or pull requests

5 participants