utils.py create; functions to load/save dict of df + datetime index s…

…et; data in data/processed folder
timothyyu · Feb 7, 2019 · c0d56dc · c0d56dc
1 parent 0b199b4
commit c0d56dc
Show file tree

Hide file tree

Showing 12 changed files with 507 additions and 25 deletions.
diff --git a/data/interim/clean_data.xlsx b/data/interim/clean_data.xlsx
diff --git a/data/interim/clean_data_futures.xlsx → data/interim/clean_data_future.xlsx b/data/interim/clean_data_futures.xlsx → data/interim/clean_data_future.xlsx
diff --git a/data/interim/clean_data_index.xlsx b/data/interim/clean_data_index.xlsx
diff --git a/data/processed/clean_data.xlsx b/data/processed/clean_data.xlsx
diff --git a/data/processed/clean_data_future.xlsx b/data/processed/clean_data_future.xlsx
diff --git a/data/processed/clean_data_index.xlsx b/data/processed/clean_data_index.xlsx
diff --git a/docs/changelog.md b/docs/changelog.md
@@ -1,7 +1,15 @@
 # Changelog
 
-2019-02-06
+## 2019-02-07
+- Refactors from `1_data_clean_load_datetime.ipynb` notebook:
+  - `utils.py` for `frames_to_excel(),dictmap_load(), dictmap_datetime()` functions in `wsae_lstm` source/root folder
+    - Functions to load and save a dictionary of dataframes to and from disk
+    - `clean_data.py` updated with function imports from `utils.py`
+    - `frames_to_excel()` can now accept optional `key_order` kwarg
+- Minor syntax/variable name reference consistency changes
+- Data in `data/processed` has date column in datetime object format
 
-- clean_dataset.py in `wsae_lstm` folder to clean raw dataset, output stored in `data/interim` folder (refactored from `notebooks\0_data_clean_load.ipynb`)
+## 2019-02-06
+- clean_dataset.py in `wsae_lstm` folder to clean raw dataset, output stored in `data/interim` folder (refactored from `notebooks\0_data_clean_load.ipynb` notebook)
 - `readme.md` update with repository structure section & other minor clarification changes 
 
diff --git a/notebooks/0_data_clean_load.ipynb b/notebooks/0_data_clean_load.ipynb
@@ -922,9 +922,9 @@
     "'djia index future data',\n",
     "]\n",
     "list_of_tuples = [(key, dict_dataframes[key]) for key in key_order]\n",
-    "dict_dataframes_futures = OrderedDict(list_of_tuples)\n",
+    "dict_dataframes_future = OrderedDict(list_of_tuples)\n",
     "\n",
-    "frames_to_excel(dict_dataframes_futures,\"../data/interim/clean_data_futures.xlsx\")"
+    "frames_to_excel(dict_dataframes_future,\"../data/interim/clean_data_future.xlsx\")"
    ]
   },
   {