- Instructor: Meredith Franklin
- Teaching Assistant: Caitlin Harrigan and Zhibo Zhang
- Email: [email protected], please put "JSC370" in the subject line.
- Location: ES4000 and zoom
- Time: Tuesday and Thursdays, 3-5pm
- Office hours: By Appointment
- Course Forum: Piazza
- Course syllabus
- Lab materials
Topics/Weekly Activities | Due Dates by 11:59 pm Thursdays |
|
---|---|---|
Week 1 January 11 lecture January 13 lab |
Introduction to Data Science tools: R, markdown | Lab 1 |
Week 2 January 17/18 (guest speaker/lecture January 20 lab |
Aaron Sonabend (Google) 1/17 @1pm zoom Version Control & Reproducible Research, Git |
Lab 2, Reflection |
Week 3 January 25 guest speaker/lecture January 27 lab |
Stefanie Nickels (Verily) Exploratory Data Analysis |
Lab3, Reflection |
Week 4 January 31/February 1 (guest speaker/lecture) February 3 (lab) |
Kathy Evans (NBA) Data visualization |
HW1, Lab4, Reflection |
Week 5 February 9 (guest speaker/lecture) February 10 (lab) |
Graduate student panel (U of T) Data cleaning and wrangling |
Lab5, Reflection |
Week 6 February 14/15 (guest speaker/lecture) February 17 (lab) |
Paul Varghese (Verily) Regular Expressions, Big Data, Data scraping, using APIs |
HW2, Lab6, Reflection |
Week 7 February 22/24 |
Reading Week | |
Week 8 March 1 (guest speaker/lecture) March 3 (lab) |
Lisa Strug(U of T) Text mining |
Midterm, Lab8, Reflection |
Week 9 March 7/8 (guest speaker/lecture) March 10 (lab) |
Alistair Johnson (Sick Kids) High performance computing, cloud computing |
HW3, Lab9, Reflection |
Week 10 March 15 (guest speaker/lecture) March 17 (lab) |
Ellen Stephenson (U of T) ML (elastic net, xgboost) |
Lab10, Reflection |
Week 11 March 22 (guest speaker/lecture) March 24 (lab) |
Amy Braverman(NASA) Interactive visualization and effective data communication I |
HW4. Lab10, Reflection |
Week 12 March 29 (guest speaker/lecture) March 31 (lab) |
Sofia Ruiz (National University of Rosario) and Yunyi Shen (U Wisconsin-Madison) Interactive visualization and effective data communication II |
Lab12, Reflection |
Week 13 April 4/5 (guest speaker/lecture) April 6 (lab) |
Radu Craiu (U of T) Final Presentations |
HW5, Lab 13, Reflection, Final Project |
Task | % of Grade |
---|---|
Labs | 10 |
Guest speaker reflections | 5 |
Homework (5) | 50 |
Midterm report | 10 |
Final project | 25 |
[1] https://github.com/JSC370/jsc370.github.io
- The Plain Person’s Guide to Plain Text Social Science: Why you should write data-based reports using plain-text tools.
- Markdown tutorial: An interactive tutorial to practice using Markdown.
- Markdown cheatsheet: Useful one-page reminder of Markdown syntax.
- RMarkdown Cheatsheet An overview of Markdown and RMarkdown conventions.
- RStudio Cheatsheets Other quick guides, including a more comprehensive RMarkdown reference and a information about using RStudio's IDE, and some of the main tools in R.
- R Style Guide. Write readable code.
- Jenny Bryan's Stat 545. Notes and tutorials for a Data Analysis course taught by Jennifer Bryan at the University of British Columbia. Lots of useful material.
- knitr demos Documentation and examples for
knitr
by its author, Yihui Xie. There is also a knitr book covering the same ground in more detail. - Rmarkdown documentation from the makers of RStudio. Lots of good examples.
- Plain Person's Guide The git repository for this project.
- Karl Broman's Tutorials and Guides Accurate and concise guides to many of the tools and topics described here, including getting started with reproducible research, using git and GitHub, and working with knitr.
- Makefiles for OCR and converting Shapefiles. Some further examples of
Makefiles
in the data-analysis pipeline, by Lincoln Mullen
- Apple's Developer Tools Unix toolchain. Install directly with
xcode-select --install
, or just try to use e.g.git
from the terminal and have OS X prompt you to install the tools. - Homebrew package manager. A convenient way to install several of the tools here, including Emacs and Pandoc.
- R. A platform for statistical computing.
- knitr. Reproducible plain-text documents from within R.
- Python and SciPy. Python is a general-purpose programming language increasingly used in data manipulation and analysis.
- RStudio. An IDE for R. The most straightforward way to get into using R and RMarkdown.
- TeX and LaTeX. A typesetting and document preparation system. You can write files in
.tex
format directly, but it is more useful to just have it available in the background for other tools to use. The MacTeX Distribution is the one to install for macOS. - Pandoc. Converts plain-text documents to and from a wide variety of formats. Can be installed with Homebrew. Be sure to also install
pandoc-citeproc
for processing citations and bibliographies, andpandoc-crossref
for producing cross-references and labels. - Git. Version control system. Installs with Apple's Developer Tools, or get the latest version via Homebrew.
- GNU Make. You tell
make
what the steps are to create the pieces of a document or program. As you edit and change the various pieces, it automatically figures out which pieces need to be updated and recompiled, and issues the commands to do that. See Karl Broman's Minimal Make for a short introduction. Make will be installed automatically with Apple's developer tools. - lintr and flycheck. Tools that nudge you to write neater code.
- Backblaze. Secure off-site backup.
- GitHub. Host public Git repositories for free. Pay to host private ones. Also a source for publicly available code (e.g. R packages and utilities) written by other people.
- Marked 2. Live HTML previewing of Markdown documents. Mac OS X only.
- Sublime Text. Python-based text editor.
- Zotero, Mendeley, and Papers are citation managers that incorporate PDF storage, annotation and other features. Zotero is free to use. Mendeley has a premium tier. Papers is a paid application after a trial period. I don't use these tools much, but that's not for any strong principled reason---mostly just intertia. If you use one and want to integrate with the material here, just make sure it can export to BibTeX/BibLaTeX files. Papers, which I've used most recently, can handily output citation keys in pandoc's format amongst several others.
Many of these websites have API to download the data. We recommend you using APIs to get data.
- NIH Cancer Surveillance
- World Health Organization WHO data
- UniProt data
- The Gene Ontology Project
- US Center for Disease Control and Prevention Data
- California Health and Human Services Open Data Portal
- Covid Data CovidTracker
- Figshare data repository
- Zenodo data repository
- Harvard Dataverse
- Elsevier Developers API
- Toronto open data
- British Columbia open data
- Ontario Data Catalogue
- Los Angeles city data
- Los Angeles crime data
- Google Earth Engine
- Google Dataset Search
- FiveThirtyEight open data
- World Bank open data
- US Open Data Initiative DATA.GOV
- US Census Data National Historical Geographic Information System (NHGIS)
- Canada Census Data
- Twitter Developers API
- GitHub Developers API
- Instagram Developers API
- LinkedIn Developers API
- Zillow Developers API