README.md

Deep Learning course

In this github repo, I share materials corresponding to two courses that I teach at the University of Calgary:

• ENEL645 - Data Mining & Machine Learning
• ENSF 619.02 - Advanced Image Analysis and Machine Learning

This repo is constantly being updated. Initially, we used TensorFlow as the deep learning, but we will be adding PyTorch examples since it is the dominant deep learning framework at the moment.

ENEL 645 - Winter 2024

Important Dates

• Assignment 01 due January 30th at midnight - Submit in a D2L dropbox
• Assignment 02 due March 6th at midnight - Submit in a D2L dropbox
• Midterm is on February 28th in the classroom
• Midterm accommodation for those who missed the previous date is on March 6th in the classroom
• Final Projects is due March 27th - Submit in a D2L dropbox
• Reading week 18-24 February | March 29th Good Friday | April 1st Easter Monday

Assignments and Final Project Instructions and Rubric

• Assignment 01
• Assignment 02
• Final Project

Templates for final project:

• Overleaf
• MS Word

Lectures and Tutorials

Week 0

• Python recap
• NumPy recap
• L: Python Bootcamp
• T: Avoid Loops

Week 1

• M: L: Course Overview
• W: L: Fundamentals ML
• F:
  • L: Overfitting and Regularization
  • T: Overfitting and Regularization

Week 2

• M:
  • L: Deep Learning Intuition
  • T: Softmax, cross-entropy
• W:
  • L: Fully Connected Neural Networks
  • T: Fully Connected NN: 2D Synthetic Example
• F: T: Fully Connected NN: Image Classification

Week 3

• M:
  • T: Fully Connected NN - Revisited
  • T: Fully Connected NN - PyTorch
• W:
  • T: Weights and Biases
• F: - L: Data Normalization
  • L: Convolutional Neural Networks

Week 4

• M: T: Convolutional Neural Networks - Image Classification
• W:
  • L:Transfer Learning
  • T: Transfer Learning - PyTorch
  • T: Transfer Learning - TensorFlow
• F: Garbage classifier - images (T - cluster)

Week 5

• M: L: Adversarial Models for Data Generation and Domain Adaptation
• W: L: U-net Model
• F: T: U-net model for signal denoising

Week 6

• M: L:Transformers and Attention
• W: T: GPT - The contents are in the GPT folder.
• F: L:Self-supervised learning

Week 7

• M: - Reading week
• W: - Reading week
• F: - Reading week

Week 8

• M: Self-supervised learning (T - Peyman)
• W: - Midterm
• F: Domain Adaptation (L)

Week 9

• M: Responsible AI (L - Mahsa)
• W: Midterm (2nd chance)
• F: Responsible AI (T - Mahsa)

Week 10

• M: XAI (L - Mahsa)
• W: XAI (T - Mahsa)
• • F: Physics Informed NNs (L - Natalia)

Week 11

• M: Physics Informed NNs (T - Natalia)
• W: Graph NNs (L - Natalia)
• F: Graph NNs (T - Natalia)

Week 12

• M: Traditional ML
• W: (spare class if things need to be shifted)
• F:Good Friday

Week 13

• M: Easter Monday
• W: Final projects
• F: Final projects

Week 14

• M: Final projects

ENSF 619.02

Important Dates

• Assignment 01 is due February 26th at midnight - Submit in a D2L dropbox
• Assignment 02 is a reading assignment - Dates will be decided in the first week of class.
• Quiz 01 is on February 2nd in the classroom. Bring your laptop.
• Quiz 02 is on March 1st in the classroom. Bring your laptop.
• Final Projects is due March 29th - Submit in a D2L dropbox
• Reading week 18-24 February | March 29th Good Friday | April 1st Easter Monday

Assignments and Final Project Instructions and Rubric

• Assignment 01
• Assignment 02
• Final Project

Templates for final project:

• Overleaf
• MS Word

Lectures and Tutorials

Week 0

• Python recap
• NumPy recap
• L: Python Bootcamp
• T: Avoid Loops

Week 1

• M:

  • Meet and Greet
  • L: Course Overview

• F:

  • L: Fundamentals ML
  • T: Overfitting and Regularization
  • L: Overfitting and Regularization
  • Define presentation dates

Week 2

• M:
  • L: Deep Learning Intuition
  • T: Softmax, cross-metropy
  • L: Fully Connected Neural Networks
• F:
  • L: Data Normalization
  • T: Fully Connected NN: Image Classification

Week 3

• M:

  • L: Convolutional Neural Networks
  • T: Convolutional Neural Networks - Image Classification

• F:

  • L:Parameters and memory
  • L:Transfer Learning
  • T: Transfer Learning
  • T: Garbage classifier - images (cluster)

Week 4

• M:

  • L: The U-net model
  • T: Medical image segmentation cluster

• F:

  • Quiz #01
  • Paper presentations - TBD

Week 5

• M:
  • He et al, “Deep Residual Learning for Image Recognition”, CVPR 2016. Presenter: Jet Penner.
  • Tan, M. and Le, Q., 2019, May. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning (pp. 6105-6114). PMLR. Presenter: Yobbahim Javier Israel Perez Vite.
• F:
  • Zhou, K., Yang, Y., Hospedales, T. and Xiang, T., 2020, April. Deep domain-adversarial image generation for domain generalisation. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 07, pp. 13025-13032). - Saad Bin Ashraf, Md. Afif Al Mamun
  • Godard, C., Mac Aodha, O., Firman, M. and Brostow, G.J., 2019. Digging into self-supervised monocular depth estimation. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 3828-3838). - Nour Khalefa

Week 5

• M:
  • He, K., Gkioxari, G., Dollár, P. and Girshick, R., 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 2961-2969). - Zahra Arabi, Mariam Mazoor
  • Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S. and Uszkoreit, J., 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. - Wamika Jha and Syed Ali John Naqvi