Adapt and write up the markdown files

README.md

@@ -1,14 +1,7 @@
-# Pumas-AI Workshop Templates
+# Pumas-AI Introduction to Machine Learning and Neural Networks
 
 [![CC BY-SA 4.0](https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg)](http://creativecommons.org/licenses/by-sa/4.0/)
 
-## How to use this template
-
-1. Click on the green button `Use this template`
-1. Edit all the `PLACEHOLDER` in `mkdocs.yml` with respect to `site_name`, `repo_name` and `repo_url`.
-1. Edit all the `PLACEHOLDER` in `docs/index.md`, `docs/reference.md` and `docs/instructor.md`.
-1. Add appropriate content to `docs/index.md`, `docs/reference.md` and `docs/instructor.md`.
-
 ## How to contribute
 
 We use [Material for MkDocs](https://github.com/squidfunk/mkdocs-material)
@@ -22,9 +15,8 @@ We use [Material for MkDocs](https://github.com/squidfunk/mkdocs-material)
 
 ## Authors
 
-- Author 1 - <[email protected]>
-- Author 2 - <[email protected]>
-- Author 3 - <[email protected]>
+- Andreu Vall - <[email protected]>
+- Niklas Korsbo - <[email protected]>
 
 ## License
 

docs/code_of_conduct.md

@@ -1,5 +1,5 @@
 ---
-title: Code of Conduct for Pumas-AI Workshop PLACEHOLDER
+title: Code of Conduct for Pumas-AI Introduction to Machine Learning and Neural Networks
 description: Participants and Instructors must follow this at all times.
 ---
 

docs/contribute.md

@@ -5,7 +5,7 @@ title: How to Contribute
 [![CC BY-SA 4.0](https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg)](http://creativecommons.org/licenses/by-sa/4.0/)
 
 If you want to contribute to this workshop,
-please open a pull request at [`PumasAI-Labs/PLACEHOLDER`](https://github.com/PumasAI-Labs/PLACEHOLDER).
+please open a pull request at [`PumasAI-Labs/MachineLearning-and-NeuralNetworks`](https://github.com/PumasAI-Labs/MachineLearning-and-NeuralNetworks).
 
 By submitting a pull request,
 you are in accordance that your contribution will be licensed under [Creative Commons Attribution-ShareAlike 4.0 International](http://creativecommons.org/licenses/by-sa/4.0/).

docs/index.md

@@ -1,24 +1,26 @@
 ---
-title: Pumas-AI Workshop PLACEHOLDER
-description: CHANGE ME.
+title: Pumas-AI Introduction to Machine Learning and Neural Networks
+description: Pumas-AI Introduction to machine learning and neural networks using DeepPumas.
 ---
 
 [![CC BY-SA 4.0](https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg)](http://creativecommons.org/licenses/by-sa/4.0/)
 
-Short summary about the workshop.
+This workshop provides an introduction to **machine learning and neural networks in DeepPumas**. Through a series of examples and exercises, it showcases the DeepPumas functionality and syntax for machine learning and neural networks, and introduces fundamental concepts and Generalization. 
 
 !!! success "Prerequisites"
 
-    This workshop does PLACEHOLDER and PLACEHOLDER.
-    We recommend users being familiar with PLACEHOLDER, especially PLACEHOLDER.
-
-    The formal requirements are the PLACEHOLDER WORKSHOP WITH LINK.
+    The workshop is targeted to users familiar with statistical modeling and pharmacometrics who want to take advantage of DeepPumas to augment their models with machine learning components. Previous experience in machine learning is helpful but not required. We recommend users being familiar with the Pumas ecosystem and workflows, as covered in the [NLME Modeling Workshop](https://pumasai-labs.github.io/NLME-Model/) and its pre-requisites.
 
 ## Schedule
 
 | Time (HH:MM) | Activity | Description                              |
 | ------------ | -------- | ---------------------------------------- |
 | 00:00        | Setup    | Download files required for the workshop |
+| 00:05        | A simple machine learning model | Work on `1-linear_regression.jl` |
+| 00:10        | Capturing complex relationships | Work on `2-complex-relationships.jl` |
+| 00:20        | Bias-variance tradeoff | Work on `3-bias-variance_tradeoff.jl` |
+| 00:35        | Generalization | Work on `4-generalization.jl` |
+| 00:50        | Closing remarks | Q&A and feedback |
 
 ## Get in touch
 
@@ -27,9 +29,8 @@ please send an email to <[email protected]>.
 
 ## Authors
 
-- Author 1 - <[email protected]>
-- Author 2 - <[email protected]>
-- Author 3 - <[email protected]>
+- Andreu Vall - <[email protected]>
+- Niklas Korsbo - <[email protected]>
 
 ## License
 

docs/instructors.md

@@ -1,10 +1,46 @@
 ---
-title: Instructor's Notes for Pumas-AI Workshop PLACEHOLDER
+title: Instructor's Note for Pumas-AI Introduction to Machine Learning and Neural Networks
 ---
 
 [![CC BY-SA 4.0](https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg)](http://creativecommons.org/licenses/by-sa/4.0/)
 
-PLACEHOLDER.
+!!! success "Note"
+
+    The code included in the workshop is split into separated scripts just for the sake of clarity. Each script can only be executed after the previous ones have been executed.
+
+!!! success "Note"
+
+    The examples and exercises included in the workshop are technically very simple. This is because they are not meant as ready-made recipes, but rather as a basic material with which fundamental concepts can be presented and discussed.
+
+## Suggested itinerary
+
+### A simple machine learning model
+
+`01-linear_regression.jl` can be thought as a warm-up. A trivial synthetic dataset of pairs of scalars, $\{ (x_i, y_i) \}_{i=1}^N$, is generated by drawing $x_i$ from a uniform distribution and computing $y_i = x_i + \varepsilon_i$, where $\varepsilon_i$ is normally distributed noise. The dataset is not quite interesting, but it provides a good chance to introduce the concepts of [supervised learning](https://en.wikipedia.org/wiki/Supervised_learning), [empirical risk minimization](https://en.wikipedia.org/wiki/Empirical_risk_minimization) and to reflect on the fact that, in the wild, the modeler is mostly unaware of which patterns may underlie the data.
+
+This dataset is fitted with a linear regression model implemented in DeepPumas. The linear regression model can be seen as the most basic form of a [multilayer perceptron](https://en.wikipedia.org/wiki/Multilayer_perceptron) and thus, the connection between vocabulary and approaches in statistical modeling and machine learning can be highlighted. More interestingly, the implementation and fit of a linear regression model introduce the DeepPumas-specific `preprocess`, `MLPDomain` and `fit`.
+
+### Capturing complex relationships
+
+`02-complex_relationships.jl` starts by generating a "more complex" dataset. The new dataset, given by the relationship $y_i = x_i ^ 2 + \varepsilon_i$, is also quite uninteresting, but taken together, both datasets provide a pretext to discuss the need to adequate model complexity with the actual complexity required for the task at hand (exercise 2.2).
+
+The DeepPumas `MLPDomain`, briefly presented earlier, is used to implement a multilayer perceptron with one hidden layer. The occasion should be used to inspect the syntax required to specify the number of layers, the number of units, and the activation functions in an `MLPDomain`, and to remind the users that these details can be recalled through `?MLPDomain`.
+
+### Bias-variance tradeoff
+
+`03-bias-variance_tradeoff.jl` discusses the training (or fitting) of machine learning models and the [bias-variance tradeoff](https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff). To showcase examples of so-called underfitted and overfitted models, two main aspects are investigated, namely for how many epochs (or iterations) a machine learning model is fitted, the complexity of the machine learning model, and the relationship between the two.
+
+Exercise 3.1 deals with the number of training epochs. It introduces the ability to pass options to the optimizer through `optim_options`, in particular `optim_options = (; iterations = NUM_ITERATIONS)`. It shows that, for a machine learning model of complexity reasonably suited to the task at hand, both too few and too many training epochs can be detrimental. Exercises 3.2 and 3.3 further bring the effect of model complexity into the mix.
+
+This section may be striking for some users, since, if a machine learning model had the right level of complexity, and it reached "the right solution", how could training it longer possibly harm? Can different neural networks lead to similarly good solutions? This is a good moment to discuss the basic paradigm in which neural networks operate, including overparameterization, [universal approximation theorem](https://en.wikipedia.org/wiki/Universal_approximation_theorem), existence of many local minima (and most likely absence of a global one), importance of careful design and fitting, and ability to cast a modeling problem as an optimization one.
+
+### Generalization
+
+`04-generalization.jl` elaborates on the ability of machine learning models to make accurate predictions on unseen data. The section starts by presenting the concept of withheld (or validation, or test) data, as well as its crucial role in training machine learning models. Then, a bias-variance figure is constructed. Time should be taken to inspect and explain the figure in detail, and to draw connections to the previous section.
+
+The concept of [regularization](https://en.wikipedia.org/wiki/Regularization_(mathematics)) is introduced, along with the DeepPumas syntax to add regularization to an `MLPDomain`. This is a good time to describe regularization approaches based on the addition of a penalization term to the loss function, but the existence of other approaches such as early stopping, data augmentation and dropout should be mentioned.
+
+If time allows, introduce the larger concept of [model selection](https://en.wikipedia.org/wiki/Model_selection), maybe starting with the concepts of hyperparameter and hyperparemeter optimization. At this point of the workshop, good examples of hyperparameters are the learning rate of a gradient descent algorithm, the weight given to the regularization term in a loss function, or the number of layers and of units in a multilayer perceptron. The (arguably blurry) difference between model parameters and hyperparameters deserves attention. Finally, the DeepPumas `hyperopt` tool for programmatic hyperparameter tuning is demonstrated.
 
 ## Get in touch
 

docs/reference.md

@@ -1,18 +1,41 @@
 ---
-title: Reference Sheets for Pumas-AI Workshop PLACEHOLDER
+title: Reference Sheets for Pumas-AI Introduction to Machine Learning and Neural Networks
 ---
 
 [![CC BY-SA 4.0](https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg)](http://creativecommons.org/licenses/by-sa/4.0/)
 
 ## Key Points
 
-This can be either a markdown table or a list.
+- DeepPumas augments Pumas models with neural networks
+- This workshop focuses both on DeepPumas and on machine learning concepts
+- Fundamental concepts of machine learning and neural networks
+    - supervised learning
+    - empirical risk minimization 
+    - multilayer perceptron 
+    - bias-variance tradeoff
+    - training, underfitting, overfitting
+    - generalization
+    - regularization
+    - model selection
+    - hyperparameter optimization
+- DeepPumas basic functionalities to work with neural networks
+    - `preprocess`
+    - `MLPDomain`
+    - `fit`
+    - `optim_options`
+    - `hyperopt`
 
 ## Summary of Basic Commands
 
 | Action      | Command       | Observations          |
 | ----------- | ------------- | --------------------- |
-| placeholder | `placeholder` | this is a placeholder |
+| Get a supervised machine learning dataset ready for further use with DeepPumas | `preprocess` | Expects data as matrices `X`, `Y`, with samples stored as columns, and returns a `FitTarget` for further use with `fit` and `hyperopt` |
+| Construct a multilayer perceptron | `MLPDomain` | The constructor accepts parameters to specify the number of layers, the number of units in each layer, the activation functions, and the type of regularization |
+| Fit a multilayer perceptron to a preprocessed supervised machine learning dataset | `fit` | It can also fit other machine learning models |
+| Pass options to the optimizer | `optim_options` | A `NamedTuple` of options to be passed on to the optimizer. Here, we experiment with the number of iterations |
+| Automate fitting and hyperparameter tuning | `hyperopt` | Optimize the parameters and hyperparameters of a machine learning model, in particular, of a multilayer perceptron |
+
+<!-- TODO
 
 ## Glossary
 
@@ -22,7 +45,7 @@ This can be either a markdown table or a list.
 
 `term2`
 
-: Definition of the term two above.
+: Definition of the term two above. -->
 
 ## Get in touch
 

docs/waiver.md

@@ -1,5 +1,5 @@
 ---
-title: Waiver of Liability for Pumas-AI Workshop PLACEHOLDER
+title: Waiver of Liability for Pumas-AI Introduction to Machine Learning and Neural Networks
 ---
 
 [![CC BY-SA 4.0](https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg)](http://creativecommons.org/licenses/by-sa/4.0/)

mkdocs.yml

@@ -1,8 +1,8 @@
 # yaml-language-server: $schema=https://squidfunk.github.io/mkdocs-material/schema.json
-site_name: Pumas-AI Workshop PLACEHOLDER
-repo_name: PumasAI-Labs/Workshop-PLACEHOLDER
-repo_url: https://github.com/PumasAI-Labs/Workshop-PLACEHOLDER
-copyright: Copyright © 2023 Pumas-AI, Inc.
+site_name: Pumas-AI Introduction to Machine Learning and Neural Networks
+repo_name: PumasAI-Labs/MachineLearning-and-NeuralNetworks
+repo_url: https://github.com/PumasAI-Labs/MachineLearning-and-NeuralNetworks
+copyright: Copyright © 2024 Pumas-AI, Inc.
 plugins:
   - search
 theme:
@@ -55,6 +55,8 @@ extra:
 markdown_extensions:
   - tables
   - attr_list
+  - pymdownx.arithmatex:
+      generic: true
   - pymdownx.emoji:
       emoji_index: !!python/name:materialx.emoji.twemoji
       emoji_generator: !!python/name:materialx.emoji.to_svg
@@ -67,6 +69,12 @@ markdown_extensions:
   - pymdownx.details
   - pymdownx.superfences
   - def_list
+
+extra_javascript:
+  - javascripts/mathjax.js
+  - https://polyfill.io/v3/polyfill.min.js?features=es6
+  - https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js
+
 nav:
   - Home: index.md
   - Reference Sheet: reference.md