diff --git a/episodes/02-submit-job.Rmd b/episodes/02-submit-job.Rmd
index 1e21fea..a9c3a63 100644
--- a/episodes/02-submit-job.Rmd
+++ b/episodes/02-submit-job.Rmd
@@ -1,113 +1,174 @@
 ---
-title: "Using RMarkdown"
+title: "Submit a parallel job"
 teaching: 10
 exercises: 2
 ---
 
 :::::::::::::::::::::::::::::::::::::: questions 
 
-- How do you write a lesson using R Markdown and `{sandpaper}`?
+- How do you get a high performance computing cluster to run a program?
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: objectives
 
-- Explain how to use markdown with the new lesson template
-- Demonstrate how to include pieces of code, figures, and nested challenge blocks
+- Introduce a parallel R program
+- Submit a parallel R program to a job scheduler on a cluster
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ## Introduction
 
-This is a lesson created via The Carpentries Workbench. It is written in
-[Pandoc-flavored Markdown](https://pandoc.org/MANUAL.txt) for static files and
-[R Markdown][r-markdown] for dynamic files that can render code into output. 
-Please refer to the [Introduction to The Carpentries 
-Workbench](https://carpentries.github.io/sandpaper-docs/) for full documentation.
-
-What you need to know is that there are three sections required for a valid
-Carpentries lesson template:
-
- 1. `questions` are displayed at the beginning of the episode to prime the
-    learner for the content.
- 2. `objectives` are the learning objectives for an episode displayed with
-    the questions.
- 3. `keypoints` are displayed at the end of the episode to reinforce the
-    objectives.
-
-:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: instructor
-
-Inline instructor notes can help inform instructors of timing challenges
-associated with the lessons. They appear in the "Instructor View"
-
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-
-::::::::::::::::::::::::::::::::::::: challenge 
-
-## Challenge 1: Can you do it?
-
-What is the output of this command?
 
 ```r
-paste("This", "new", "lesson", "looks", "good")
+## This script describes two levels of parallelism:
+## Top level: Distributed MPI runs several copies of this entire script.
+##            Instances differ by their comm.rank() designation.
+## Inner level: The unix fork (copy-on-write) shared memory parallel execution
+##            of the mc.function() managed by parallel::mclapply()
+## Further levels are possible: multithreading in compiled code and communicator
+## splitting at the distributed MPI level.
+
+suppressMessages(library(pbdMPI))
+comm.print(sessionInfo())
+
+## get node name
+host = system("hostname", intern = TRUE)
+
+mc.function = function(x) {
+    Sys.sleep(1) # replace with your function for mclapply cores here
+    Sys.getpid() # returns process id
+}
+
+## Compute how many cores per R session are on this node
+local_ranks_query = "echo $OMPI_COMM_WORLD_LOCAL_SIZE"
+ranks_on_my_node = as.numeric(system(local_ranks_query, intern = TRUE))
+cores_on_my_node = parallel::detectCores()
+cores_per_R = floor(cores_on_my_node/ranks_on_my_node)
+cores_total = allreduce(cores_per_R)  # adds up over ranks
+
+## Run mclapply on allocated cores to demonstrate fork pids
+my_pids = parallel::mclapply(1:cores_per_R, mc.function, mc.cores = cores_per_R)
+my_pids = do.call(paste, my_pids) # combines results from mclapply
+##
+## Same cores are shared with OpenBLAS (see flexiblas package)
+##            or for other OpenMP enabled codes outside mclapply.
+## If BLAS functions are called inside mclapply, they compete for the
+##            same cores: avoid or manage appropriately!!!
+
+## Now report what happened and where
+msg = paste0("Hello World from rank ", comm.rank(), " on host ", host,
+             " with ", cores_per_R, " cores allocated\n",
+             "            (", ranks_on_my_node, " R sessions sharing ",
+             cores_on_my_node, " cores on this host node).\n",
+             "      pid: ", my_pids, "\n")
+comm.cat(msg, quiet = TRUE, all.rank = TRUE)
+
+
+comm.cat("Total R sessions:", comm.size(), "Total cores:", cores_total, "\n",
+         quiet = TRUE)
+comm.cat("\nNotes: cores on node obtained by: detectCores {parallel}\n",
+         "       ranks (R sessions) per node: OMPI_COMM_WORLD_LOCAL_SIZE\n",
+         "       pid to core map changes frequently during mclapply\n",
+         quiet = TRUE)
+
+finalize()
 ```
 
-:::::::::::::::::::::::: solution 
-
-## Output
- 
-```output
-[1] "This new lesson looks good"
+## Submit a job on a cluster
+
+
+:::::::::::::::::::::::: solution
+
+## Slurm
+
+
+```bash
+#!/bin/bash
+#SBATCH -J hello
+#SBATCH -A CSC489
+#SBATCH -p batch
+#SBATCH --nodes=4
+#SBATCH --mem=0
+#SBATCH -t 00:00:10
+#SBATCH -e ./hello.e
+#SBATCH -o ./hello.o
+#SBATCH --open-mode=truncate
+
+## above we request 4 nodes and all memory on the nodes
+
+## assumes this repository was cloned in your home area
+cd ~/R4HPC/code_1
+pwd
+
+## modules are specific to andes.olcf.ornl.gov
+module load openblas/0.3.17-omp
+module load flexiblas
+flexiblas add OpenBLAS $OLCF_OPENBLAS_ROOT/lib/libopenblas.so
+export LD_PRELOAD=$OLCF_FLEXIBLAS_ROOT/lib64/libflexiblas.so
+module load r
+echo -e "loaded R with FlexiBLAS"
+module list
+
+## above supplies your R code with FlexiBLAS-OpenBLAS on Andes
+## but matrix computation is not used in the R illustration below
+
+# An illustration of fine control of R scripts and cores on several nodes
+# This runs 4 R sessions on each of 4 nodes (for a total of 16).
+#
+# Each of the 16 hello_world.R scripts will calculate how many cores are
+# available per R session from environment variables and use that many
+# in mclapply.
+#
+# NOTE: center policies may require dfferent parameters
+#
+# runs 4 R sessions per node
+mpirun --map-by ppr:4:node Rscript hello_balance.R
 ```
-
 :::::::::::::::::::::::::::::::::
 
 
-## Challenge 2: how do you nest solutions within challenge blocks?
+:::::::::::::::::::::::: solution
 
-:::::::::::::::::::::::: solution 
+## PBS
 
-You can add a line with at least three colons and a `solution` tag.
+```bash
+#!/bin/bash
+#PBS -N hello
+#PBS -A DD-21-42
+#PBS -l select=4:mpiprocs=16
+#PBS -l walltime=00:00:10
+#PBS -q qprod
+#PBS -e hello.e
+#PBS -o hello.o
 
-:::::::::::::::::::::::::::::::::
-::::::::::::::::::::::::::::::::::::::::::::::::
-
-## Figures
+cat $BASH_SOURCE 
+cd ~/R4HPC/code_1
+pwd
 
-You can also include figures generated from R Markdown:
-
-```{r pyramid, fig.alt = "pie chart illusion of a pyramid", fig.cap = "Sun arise each and every morning"}
-pie(
-  c(Sky = 78, "Sunny side of pyramid" = 17, "Shady side of pyramid" = 5), 
-  init.angle = 315, 
-  col = c("deepskyblue", "yellow", "yellow3"), 
-  border = FALSE
-)
-```
+## module names can vary on different platforms
+module load R
+echo "loaded R"
 
-Or you can use standard markdown for static figures with the following syntax:
+## prevent warning when fork is used with MPI
+export OMPI_MCA_mpi_warn_on_fork=0
+export RDMAV_FORK_SAFE=1
 
-`![optional caption that appears below the figure](figure url){alt='alt text for
-accessibility purposes'}`
+# Fix for warnings from libfabric/1.12 on Karolina
+module swap libfabric/1.12.1-GCCcore-10.3.0 libfabric/1.13.2-GCCcore-11.2.0 
 
-![You belong in The Carpentries!](https://raw.githubusercontent.com/carpentries/logo/master/Badge_Carpentries.svg){alt='Blue Carpentries hex person logo with no text.'}
-
-## Math
-
-One of our episodes contains $\LaTeX$ equations when describing how to create
-dynamic reports with {knitr}, so we now use mathjax to describe this:
+time mpirun --map-by ppr:4:node Rscript hello_balance.R
+```
 
-`$\alpha = \dfrac{1}{(1 - \beta)^2}$` becomes: $\alpha = \dfrac{1}{(1 - \beta)^2}$
 
-Cool, right?
+:::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: keypoints 
 
-- Use `.md` files for episodes when you want static content
-- Use `.Rmd` files for episodes when you need to generate output
-- Run `sandpaper::check_lesson()` to identify any issues with your lesson
-- Run `sandpaper::build_lesson()` to preview your lesson locally
+- Parallel R code distributes work
+- There is shared memory and distributed memory parallelizm
+- You can test parallel code on your own local machine
+- There are several different job schedulers, but they share many similarities so you can learn a new one when needed
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
-[r-markdown]: https://rmarkdown.rstudio.com/
diff --git a/episodes/03-multicore.Rmd b/episodes/03-multicore.Rmd
index 1e21fea..4dd2792 100644
--- a/episodes/03-multicore.Rmd
+++ b/episodes/03-multicore.Rmd
@@ -1,113 +1,152 @@
 ---
-title: "Using RMarkdown"
+title: "Multicore"
 teaching: 10
 exercises: 2
 ---
 
 :::::::::::::::::::::::::::::::::::::: questions 
 
-- How do you write a lesson using R Markdown and `{sandpaper}`?
+- Can parallelization decrease time to solution for my program?
+- What is machine learning?
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: objectives
 
-- Explain how to use markdown with the new lesson template
-- Demonstrate how to include pieces of code, figures, and nested challenge blocks
+- Introduce machine learning, in particular the random forest algorithm
+- Demonstrate serial and parallel implementations of the random forest algorithm
+- Show that statistical machine learning models can be used to classify data after training on an existing dataset
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ## Introduction
 
-This is a lesson created via The Carpentries Workbench. It is written in
-[Pandoc-flavored Markdown](https://pandoc.org/MANUAL.txt) for static files and
-[R Markdown][r-markdown] for dynamic files that can render code into output. 
-Please refer to the [Introduction to The Carpentries 
-Workbench](https://carpentries.github.io/sandpaper-docs/) for full documentation.
 
-What you need to know is that there are three sections required for a valid
-Carpentries lesson template:
+### Serial Implementation
 
- 1. `questions` are displayed at the beginning of the episode to prime the
-    learner for the content.
- 2. `objectives` are the learning objectives for an episode displayed with
-    the questions.
- 3. `keypoints` are displayed at the end of the episode to reinforce the
-    objectives.
-
-:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: instructor
+```r
+suppressMessages(library(randomForest))
+data(LetterRecognition, package = "mlbench")
+set.seed(seed = 123)
 
-Inline instructor notes can help inform instructors of timing challenges
-associated with the lessons. They appear in the "Instructor View"
+n = nrow(LetterRecognition)
+n_test = floor(0.2 * n)
+i_test = sample.int(n, n_test)
+train = LetterRecognition[-i_test, ]
+test = LetterRecognition[i_test, ]
 
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
+rf.all = randomForest(lettr ~ ., train, ntree = 500, norm.votes = FALSE)
+pred = predict(rf.all, test)
 
-::::::::::::::::::::::::::::::::::::: challenge 
+correct = sum(pred == test$lettr)
+cat("Proportion Correct:", correct/(n_test), "\n")
+```
 
-## Challenge 1: Can you do it?
 
-What is the output of this command?
+### Parallel Multicore Implementation
 
 ```r
-paste("This", "new", "lesson", "looks", "good")
+library(parallel)                                       #<<
+library(randomForest)
+data(LetterRecognition, package = "mlbench")
+set.seed(seed = 123, "L'Ecuyer-CMRG")                   #<<
+
+n = nrow(LetterRecognition)
+n_test = floor(0.2 * n)
+i_test = sample.int(n, n_test)
+train = LetterRecognition[-i_test, ]
+test = LetterRecognition[i_test, ]
+
+nc = as.numeric(commandArgs(TRUE)[2])                    #<<
+ntree = lapply(splitIndices(500, nc), length)            #<<
+rf = function(x, train) randomForest(lettr ~ ., train, ntree=x, #<<
+                                     norm.votes = FALSE)        #<<
+rf.out = mclapply(ntree, rf, train = train, mc.cores = nc)      #<<
+rf.all = do.call(combine, rf.out)                        #<<
+
+crows = splitIndices(nrow(test), nc)                     #<<
+rfp = function(x) as.vector(predict(rf.all, test[x, ]))  #<<
+cpred = mclapply(crows, rfp, mc.cores = nc)              #<<
+pred = do.call(c, cpred)                                 #<<
+
+correct <- sum(pred == test$lettr)
+cat("Proportion Correct:", correct/(n_test), "\n")
 ```
-
-:::::::::::::::::::::::: solution 
-
-## Output
- 
-```output
-[1] "This new lesson looks good"
+:::::::::::::::::::::::: solution
+
+### SLURM submission script
+
+```bash
+#!/bin/bash
+#SBATCH -J rf
+#SBATCH -A CSC143
+#SBATCH -p batch
+#SBATCH --nodes=1
+#SBATCH -t 00:40:00
+#SBATCH --mem=0
+#SBATCH -e ./rf.e
+#SBATCH -o ./rf.o
+#SBATCH --open-mode=truncate
+
+cd ~/R4HPC/code_2
+pwd
+
+## modules are specific to andes.olcf.ornl.gov
+module load openblas/0.3.17-omp
+module load flexiblas
+flexiblas add OpenBLAS $OLCF_OPENBLAS_ROOT/lib/libopenblas.so
+export LD_PRELOAD=$OLCF_FLEXIBLAS_ROOT/lib64/libflexiblas.so
+module load r
+echo -e "loaded R with FlexiBLAS"
+module list
+
+time Rscript rf_serial.r
+time Rscript rf_mc.r --args 1
+time Rscript rf_mc.r --args 2
+time Rscript rf_mc.r --args 4
+time Rscript rf_mc.r --args 8
+time Rscript rf_mc.r --args 16
+time Rscript rf_mc.r --args 32
+time Rscript rf_mc.r --args 64
 ```
-
-:::::::::::::::::::::::::::::::::
-
-
-## Challenge 2: how do you nest solutions within challenge blocks?
-
-:::::::::::::::::::::::: solution 
-
-You can add a line with at least three colons and a `solution` tag.
-
 :::::::::::::::::::::::::::::::::
-::::::::::::::::::::::::::::::::::::::::::::::::
-
-## Figures
 
-You can also include figures generated from R Markdown:
-
-```{r pyramid, fig.alt = "pie chart illusion of a pyramid", fig.cap = "Sun arise each and every morning"}
-pie(
-  c(Sky = 78, "Sunny side of pyramid" = 17, "Shady side of pyramid" = 5), 
-  init.angle = 315, 
-  col = c("deepskyblue", "yellow", "yellow3"), 
-  border = FALSE
-)
+:::::::::::::::::::::::: solution
+
+### PBS submission script
+
+```bash
+#!/bin/bash
+#PBS -N rf
+#PBS -l select=1:ncpus=128
+#PBS -l walltime=00:05:00
+#PBS -q qexp
+#PBS -e rf.e
+#PBS -o rf.o
+
+cd ~/R4HPC/code_2
+pwd
+
+module load R
+echo "loaded R"
+
+time Rscript rf_serial.r
+time Rscript rf_mc.r --args 1
+time Rscript rf_mc.r --args 2
+time Rscript rf_mc.r --args 4
+time Rscript rf_mc.r --args 8
+time Rscript rf_mc.r --args 16
+time Rscript rf_mc.r --args 32
+time Rscript rf_mc.r --args 64
+time Rscript rf_mc.r --args 128
 ```
+:::::::::::::::::::::::::::::::::
 
-Or you can use standard markdown for static figures with the following syntax:
-
-`![optional caption that appears below the figure](figure url){alt='alt text for
-accessibility purposes'}`
-
-![You belong in The Carpentries!](https://raw.githubusercontent.com/carpentries/logo/master/Badge_Carpentries.svg){alt='Blue Carpentries hex person logo with no text.'}
-
-## Math
-
-One of our episodes contains $\LaTeX$ equations when describing how to create
-dynamic reports with {knitr}, so we now use mathjax to describe this:
-
-`$\alpha = \dfrac{1}{(1 - \beta)^2}$` becomes: $\alpha = \dfrac{1}{(1 - \beta)^2}$
-
-Cool, right?
 
 ::::::::::::::::::::::::::::::::::::: keypoints 
 
-- Use `.md` files for episodes when you want static content
-- Use `.Rmd` files for episodes when you need to generate output
-- Run `sandpaper::check_lesson()` to identify any issues with your lesson
-- Run `sandpaper::build_lesson()` to preview your lesson locally
+- To evaluate the fitted model, the availabe data is split into training and testing sets
+- Parallelization decreases the training time
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
-[r-markdown]: https://rmarkdown.rstudio.com/
diff --git a/episodes/04-blas.Rmd b/episodes/04-blas.Rmd
index 1e21fea..e35400a 100644
--- a/episodes/04-blas.Rmd
+++ b/episodes/04-blas.Rmd
@@ -1,113 +1,176 @@
 ---
-title: "Using RMarkdown"
+title: "Blas"
 teaching: 10
 exercises: 2
 ---
 
 :::::::::::::::::::::::::::::::::::::: questions 
 
-- How do you write a lesson using R Markdown and `{sandpaper}`?
+- How much can parallel libraries improve time to solution for your program?
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: objectives
 
-- Explain how to use markdown with the new lesson template
-- Demonstrate how to include pieces of code, figures, and nested challenge blocks
+- Introduce the Basic Linear Algebra Subroutines (BLAS)
+- Show that BLAS routines are used from R for statistical calculations
+- Demonstrate that parallelization can improve time to solution
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ## Introduction
 
-This is a lesson created via The Carpentries Workbench. It is written in
-[Pandoc-flavored Markdown](https://pandoc.org/MANUAL.txt) for static files and
-[R Markdown][r-markdown] for dynamic files that can render code into output. 
-Please refer to the [Introduction to The Carpentries 
-Workbench](https://carpentries.github.io/sandpaper-docs/) for full documentation.
 
-What you need to know is that there are three sections required for a valid
-Carpentries lesson template:
-
- 1. `questions` are displayed at the beginning of the episode to prime the
-    learner for the content.
- 2. `objectives` are the learning objectives for an episode displayed with
-    the questions.
- 3. `keypoints` are displayed at the end of the episode to reinforce the
-    objectives.
-
-:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: instructor
-
-Inline instructor notes can help inform instructors of timing challenges
-associated with the lessons. They appear in the "Instructor View"
-
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-
-::::::::::::::::::::::::::::::::::::: challenge 
-
-## Challenge 1: Can you do it?
+```r
+library(flexiblas)
+flexiblas_avail()
+flexiblas_version()
+flexiblas_current_backend()
+flexiblas_list()
+flexiblas_list_loaded()
+
+getthreads = function() {
+  flexiblas_get_num_threads()
+}
+setthreads = function(thr, label = "") {
+  cat(label, "Setting", thr, "threads\n")
+  flexiblas_set_num_threads(thr)
+}
+setback = function(backend, label = "") {
+  cat(label, "Setting", backend, "backend\n")
+  flexiblas_switch(flexiblas_load_backend(backend))
+}
+
+#' PT
+#' A function to time one or more R expressions after setting the number of
+#' threads available to the BLAS library.
+#' 
+#' !!
+#' DO NOT USE PT RECURSIVELY
+#'
+#' Use: 
+#' variable-for-result = PT(your-num-threads, a-quoted-text-comment, {
+#'   expression
+#'   expression
+#'   ...
+#'   expression-to-assign
+#' })
+PT = function(threads, text = "", expr) {
+  setthreads(threads, label = text)
+  print(system.time({result = {expr}}))
+  result
+}
+```
 
-What is the output of this command?
 
 ```r
-paste("This", "new", "lesson", "looks", "good")
+source("flexiblas_setup.R")
+memuse::howbig(5e4, 2e3)
+parallel::detectCores()
+
+x = matrix(rnorm(1e8), nrow = 5e4, ncol = 2e3)
+beta = rep(1, ncol(x))
+err = rnorm(nrow(x))
+y = x %*% beta + err
+data = as.data.frame(cbind(y, x))
+names(data) = c("y", paste0("x", 1:ncol(x)))
+
+setback("OPENBLAS")
+# qr --------------------------------------
+for(i in 0:4) {
+  setthreads(2^i, "qr")
+  print(system.time((qr(x, LAPACK = TRUE))))
+}
+
+# prcomp --------------------------------------
+for(i in 0:4) {
+  setthreads(2^i, "prcomp")
+  print(system.time((prcomp(x))))
+}
+
+# princomp --------------------------------------
+for(i in 0:4) {
+  setthreads(2^i, "princomp")
+  print(system.time((princomp(x))))
+}
+
+# crossprod --------------------------------------
+for(i in 0:5) {
+  setthreads(2^i, "crossprod")
+  print(system.time((crossprod(x))))
+}
+
+# %*% --------------------------------------------
+for(i in 0:5) {
+  setthreads(2^i, "%*%")
+  print(system.time((t(x) %*% x)))
+}
 ```
 
+
+
 :::::::::::::::::::::::: solution 
 
-## Output
+## SLURM submision script
  
-```output
-[1] "This new lesson looks good"
+```bash
+#!/bin/bash
+#SBATCH -J flexiblas
+#SBATCH -A CSC489
+#SBATCH -p batch
+#SBATCH --nodes=1
+#SBATCH --mem=0
+#SBATCH -t 00:15:00
+#SBATCH -e ./flexiblas.e
+#SBATCH -o ./flexiblas.o
+#SBATCH --open-mode=truncate
+
+## assumes this repository was cloned in your home area
+cd ~/R4HPC/code_3
+pwd
+
+## modules are specific to andes.olcf.ornl.gov
+module load openblas/0.3.17-omp
+module load flexiblas
+flexiblas add OpenBLAS $OLCF_OPENBLAS_ROOT/lib/libopenblas.so
+export LD_PRELOAD=$OLCF_FLEXIBLAS_ROOT/lib64/libflexiblas.so
+module load r
+echo -e "loaded R with FlexiBLAS"
+module list
+
+Rscript flexiblas_bench.R
 ```
 
 :::::::::::::::::::::::::::::::::
 
 
-## Challenge 2: how do you nest solutions within challenge blocks?
 
 :::::::::::::::::::::::: solution 
 
-You can add a line with at least three colons and a `solution` tag.
+## PBS submission script
 
-:::::::::::::::::::::::::::::::::
-::::::::::::::::::::::::::::::::::::::::::::::::
+```bash
+#!/bin/bash
+#PBS -N fx
+#PBS -l select=1:ncpus=128,walltime=00:50:00
+#PBS -q qexp
+#PBS -e fx.e
+#PBS -o fx.o
 
-## Figures
+cd ~/R4HPC/code_3
+pwd
 
-You can also include figures generated from R Markdown:
+module load R
+echo "loaded R"
 
-```{r pyramid, fig.alt = "pie chart illusion of a pyramid", fig.cap = "Sun arise each and every morning"}
-pie(
-  c(Sky = 78, "Sunny side of pyramid" = 17, "Shady side of pyramid" = 5), 
-  init.angle = 315, 
-  col = c("deepskyblue", "yellow", "yellow3"), 
-  border = FALSE
-)
+time Rscript flexiblas_bench2.R
 ```
 
-Or you can use standard markdown for static figures with the following syntax:
-
-`![optional caption that appears below the figure](figure url){alt='alt text for
-accessibility purposes'}`
-
-![You belong in The Carpentries!](https://raw.githubusercontent.com/carpentries/logo/master/Badge_Carpentries.svg){alt='Blue Carpentries hex person logo with no text.'}
-
-## Math
-
-One of our episodes contains $\LaTeX$ equations when describing how to create
-dynamic reports with {knitr}, so we now use mathjax to describe this:
-
-`$\alpha = \dfrac{1}{(1 - \beta)^2}$` becomes: $\alpha = \dfrac{1}{(1 - \beta)^2}$
-
-Cool, right?
+:::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: keypoints 
 
-- Use `.md` files for episodes when you want static content
-- Use `.Rmd` files for episodes when you need to generate output
-- Run `sandpaper::check_lesson()` to identify any issues with your lesson
-- Run `sandpaper::build_lesson()` to preview your lesson locally
+- Many statistical calculations require matrix and vector operations
+- When libraries are used, setting their parameters appropriately can improve your time to solution
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
-
-[r-markdown]: https://rmarkdown.rstudio.com/
diff --git a/episodes/05-mpi.Rmd b/episodes/05-mpi.Rmd
index 1e21fea..74cade4 100644
--- a/episodes/05-mpi.Rmd
+++ b/episodes/05-mpi.Rmd
@@ -1,113 +1,98 @@
 ---
-title: "Using RMarkdown"
+title: "MPI - Distributed Memory Parallelizm"
 teaching: 10
 exercises: 2
 ---
 
 :::::::::::::::::::::::::::::::::::::: questions 
 
-- How do you write a lesson using R Markdown and `{sandpaper}`?
+- How do you utilize more than one shared memory node?
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: objectives
 
-- Explain how to use markdown with the new lesson template
-- Demonstrate how to include pieces of code, figures, and nested challenge blocks
+- Demonstrate how to submit a job on multiple nodes
+- Demonstrate that a program with distributed memory parallelizm can be run on a shared memory node
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ## Introduction
 
-This is a lesson created via The Carpentries Workbench. It is written in
-[Pandoc-flavored Markdown](https://pandoc.org/MANUAL.txt) for static files and
-[R Markdown][r-markdown] for dynamic files that can render code into output. 
-Please refer to the [Introduction to The Carpentries 
-Workbench](https://carpentries.github.io/sandpaper-docs/) for full documentation.
+### Hello World!
 
-What you need to know is that there are three sections required for a valid
-Carpentries lesson template:
-
- 1. `questions` are displayed at the beginning of the episode to prime the
-    learner for the content.
- 2. `objectives` are the learning objectives for an episode displayed with
-    the questions.
- 3. `keypoints` are displayed at the end of the episode to reinforce the
-    objectives.
-
-:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: instructor
-
-Inline instructor notes can help inform instructors of timing challenges
-associated with the lessons. They appear in the "Instructor View"
-
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-
-::::::::::::::::::::::::::::::::::::: challenge 
-
-## Challenge 1: Can you do it?
+```r
+suppressMessages(library(pbdMPI))
 
-What is the output of this command?
+my_rank = comm.rank()
+nranks = comm.size()
+msg = paste0("Hello World! My name is Rank", my_rank,
+             ". We are ", nranks, " identical siblings.")
+cat(msg, "\n")
 
-```r
-paste("This", "new", "lesson", "looks", "good")
+finalize()
 ```
 
+
 :::::::::::::::::::::::: solution 
 
-## Output
+#### SLURM submission script
  
-```output
-[1] "This new lesson looks good"
+```bash
+#!/bin/bash
+#SBATCH -J hello
+#SBATCH -A CSC143
+#SBATCH -p batch
+#SBATCH --nodes=1
+#SBATCH -t 00:40:00
+#SBATCH --mem=0
+#SBATCH -e ./hello.e
+#SBATCH -o ./hello.o
+#SBATCH --open-mode=truncate
+
+cd ~/R4HPC/code_5
+pwd
+
+## modules are specific to andes.olcf.ornl.gov
+module load openblas/0.3.17-omp
+module load flexiblas
+flexiblas add OpenBLAS $OLCF_OPENBLAS_ROOT/lib/libopenblas.so
+export LD_PRELOAD=$OLCF_FLEXIBLAS_ROOT/lib64/libflexiblas.so
+module load r
+echo -e "loaded R with FlexiBLAS"
+module list
+
+mpirun --map-by ppr:32:node Rscript hello_world.R
 ```
 
 :::::::::::::::::::::::::::::::::
 
 
-## Challenge 2: how do you nest solutions within challenge blocks?
-
 :::::::::::::::::::::::: solution 
 
-You can add a line with at least three colons and a `solution` tag.
+#### PBS Submission Script
 
-:::::::::::::::::::::::::::::::::
-::::::::::::::::::::::::::::::::::::::::::::::::
+```bash
+#!/bin/bash
+#PBS -N hello
+#PBS -l select=1:ncpus=32
+#PBS -l walltime=00:05:00
+#PBS -q qexp
+#PBS -e hello.e
+#PBS -o hello.o
 
-## Figures
+cd ~/R4HPC/code_5
+pwd
 
-You can also include figures generated from R Markdown:
+module load R
+echo "loaded R"
 
-```{r pyramid, fig.alt = "pie chart illusion of a pyramid", fig.cap = "Sun arise each and every morning"}
-pie(
-  c(Sky = 78, "Sunny side of pyramid" = 17, "Shady side of pyramid" = 5), 
-  init.angle = 315, 
-  col = c("deepskyblue", "yellow", "yellow3"), 
-  border = FALSE
-)
+mpirun --map-by ppr:32:node Rscript hello_world.R
 ```
 
-Or you can use standard markdown for static figures with the following syntax:
-
-`![optional caption that appears below the figure](figure url){alt='alt text for
-accessibility purposes'}`
-
-![You belong in The Carpentries!](https://raw.githubusercontent.com/carpentries/logo/master/Badge_Carpentries.svg){alt='Blue Carpentries hex person logo with no text.'}
-
-## Math
-
-One of our episodes contains $\LaTeX$ equations when describing how to create
-dynamic reports with {knitr}, so we now use mathjax to describe this:
-
-`$\alpha = \dfrac{1}{(1 - \beta)^2}$` becomes: $\alpha = \dfrac{1}{(1 - \beta)^2}$
-
-Cool, right?
-
 ::::::::::::::::::::::::::::::::::::: keypoints 
 
-- Use `.md` files for episodes when you want static content
-- Use `.Rmd` files for episodes when you need to generate output
-- Run `sandpaper::check_lesson()` to identify any issues with your lesson
-- Run `sandpaper::build_lesson()` to preview your lesson locally
+- One can run a distributed memory program on a shared memory node
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
-[r-markdown]: https://rmarkdown.rstudio.com/
diff --git a/episodes/06-pbdmpi.Rmd b/episodes/06-pbdmpi.Rmd
index 1e21fea..bcec314 100644
--- a/episodes/06-pbdmpi.Rmd
+++ b/episodes/06-pbdmpi.Rmd
@@ -1,113 +1,416 @@
 ---
-title: "Using RMarkdown"
+title: "pbdMPI - Parallel and Big Data interface to MPI"
 teaching: 10
 exercises: 2
 ---
 
 :::::::::::::::::::::::::::::::::::::: questions 
 
-- How do you write a lesson using R Markdown and `{sandpaper}`?
+- What types of functions does MPI provide?
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: objectives
 
-- Explain how to use markdown with the new lesson template
-- Demonstrate how to include pieces of code, figures, and nested challenge blocks
+- Demonstrate some of the functionality of the pbd bindings to the Message Passing Interface (MPI) 
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ## Introduction
 
-This is a lesson created via The Carpentries Workbench. It is written in
-[Pandoc-flavored Markdown](https://pandoc.org/MANUAL.txt) for static files and
-[R Markdown][r-markdown] for dynamic files that can render code into output. 
-Please refer to the [Introduction to The Carpentries 
-Workbench](https://carpentries.github.io/sandpaper-docs/) for full documentation.
+### Hello World in Serial
 
-What you need to know is that there are three sections required for a valid
-Carpentries lesson template:
+```r
+library( pbdMPI, quiet = TRUE )
 
- 1. `questions` are displayed at the beginning of the episode to prime the
-    learner for the content.
- 2. `objectives` are the learning objectives for an episode displayed with
-    the questions.
- 3. `keypoints` are displayed at the end of the episode to reinforce the
-    objectives.
+text = paste( "Hello, world from", comm.rank() )
+print( text )
 
-:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: instructor
+finalize()
+```
 
-Inline instructor notes can help inform instructors of timing challenges
-associated with the lessons. They appear in the "Instructor View"
+### Rank
 
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
+```r
+library( pbdMPI, quiet = TRUE )
 
-::::::::::::::::::::::::::::::::::::: challenge 
+my.rank <- comm.rank()
+comm.print( my.rank, all.rank = TRUE )
 
-## Challenge 1: Can you do it?
+finalize()
+```
 
-What is the output of this command?
+### Hello World in Parallel
 
 ```r
-paste("This", "new", "lesson", "looks", "good")
+library( pbdMPI, quiet = TRUE )
+
+print( "Hello, world print" )
+
+comm.print( "Hello, world comm.print" )
+
+comm.print( "Hello from all", all.rank = TRUE, quiet = TRUE )
+
+finalize()
 ```
 
-:::::::::::::::::::::::: solution 
+### Map Reduce
+
+```r
+library( pbdMPI , quiet = TRUE)
+
+## Your "Map" code
+n = comm.rank() + 1
 
-## Output
- 
-```output
-[1] "This new lesson looks good"
+## Now "Reduce" but give the result to all
+all_sum = allreduce( n ) # Sum is default
+
+text = paste( "Hello: n is", n, "sum is", all_sum )
+comm.print( text, all.rank = TRUE )
+
+finalize ()
 ```
 
-:::::::::::::::::::::::::::::::::
+### Calculate Pi
 
+```r
+### Compute pi by simulaton
+library( pbdMPI, quiet = TRUE )
 
-## Challenge 2: how do you nest solutions within challenge blocks?
+comm.set.seed( seed = 1234567, diff = TRUE )
 
-:::::::::::::::::::::::: solution 
+my.N = 1e7 %/% comm.size()
+my.X = matrix( runif( my.N * 2 ), ncol = 2 )
+my.r = sum( rowSums( my.X^2 ) <= 1 )
+r = allreduce( my.r )
+PI = 4*r / ( my.N * comm.size() )
+comm.print( PI )
 
-You can add a line with at least three colons and a `solution` tag.
+finalize()
+```
 
-:::::::::::::::::::::::::::::::::
-::::::::::::::::::::::::::::::::::::::::::::::::
+### Broadcast
+
+```r
+library( pbdMPI, quiet = TRUE )
+
+if ( comm.rank() == 0 ){
+  x = matrix( 1:4, nrow = 2 )
+} else {
+  x = NULL
+}
+
+y = bcast( x )
+
+comm.print( y, all.rank = TRUE )
+comm.print( x, all.rank = TRUE )
+
+finalize()
+```
+
+### Gather
+
+```r
+library( pbdMPI, quiet = TRUE )
+
+comm.set.seed( seed = 1234567, diff = TRUE )
 
-## Figures
+my_rank = comm.rank()
+n = sample( 1:10, size = my_rank + 1 )
+comm.print(n, all.rank = TRUE)
 
-You can also include figures generated from R Markdown:
+gt = gather(n)
 
-```{r pyramid, fig.alt = "pie chart illusion of a pyramid", fig.cap = "Sun arise each and every morning"}
-pie(
-  c(Sky = 78, "Sunny side of pyramid" = 17, "Shady side of pyramid" = 5), 
-  init.angle = 315, 
-  col = c("deepskyblue", "yellow", "yellow3"), 
-  border = FALSE
-)
+obj_len = gather(length(n))
+comm.cat("gathered unequal size objects. lengths =", unlist(obj_len), "\n")
+
+comm.print( unlist( gt ), all.rank = TRUE )
+
+finalize()
+```
+
+### Gather Unequal
+
+```r
+library( pbdMPI, quiet = TRUE )
+
+comm.set.seed( seed = 1234567, diff = TRUE )
+
+my_rank = comm.rank( )
+n = sample( 1:10, size = my_rank + 1 )
+comm.print( n, all.rank = TRUE )
+
+gt = gather( n )
+
+obj_len = gather( length( n ) )
+comm.cat( "gathered unequal size objects. lengths =", obj_len, "\n" )
+
+comm.print( unlist( gt ), all.rank = TRUE )
+
+finalize( )
 ```
 
-Or you can use standard markdown for static figures with the following syntax:
+### Gather Named
+
+```r
+library( pbdMPI, quiet = TRUE )
 
-`![optional caption that appears below the figure](figure url){alt='alt text for
-accessibility purposes'}`
+comm.set.seed( seed = 1234567, diff = TRUE )
 
-![You belong in The Carpentries!](https://raw.githubusercontent.com/carpentries/logo/master/Badge_Carpentries.svg){alt='Blue Carpentries hex person logo with no text.'}
+my_rank = comm.rank()
+n = sample( 1:10, size = my_rank + 1 )
+names(n) = paste0("a", 1:(my_rank + 1))
+comm.print(n, all.rank = TRUE)
 
-## Math
+gt = gather( n )
 
-One of our episodes contains $\LaTeX$ equations when describing how to create
-dynamic reports with {knitr}, so we now use mathjax to describe this:
+comm.print( unlist( gt ), all.rank = TRUE )
 
-`$\alpha = \dfrac{1}{(1 - \beta)^2}$` becomes: $\alpha = \dfrac{1}{(1 - \beta)^2}$
+finalize()
+```
+
+### Chunk
+
+```r
+library( pbdMPI, quiet = TRUE )
+
+my.rank = comm.rank( )
+
+k = comm.chunk( 10 )
+comm.cat( my.rank, ":", k, "\n", all.rank = TRUE, quiet = TRUE)
+
+k = comm.chunk( 10 , form = "vector")
+comm.cat( my.rank, ":", k, "\n", all.rank = TRUE, quiet = TRUE)
+
+k = comm.chunk( 10 , form = "vector", type = "equal")
+comm.cat( my.rank, ":", k, "\n", all.rank = TRUE, quiet = TRUE)
+
+finalize( )
+```
+
+### Timing
+
+```r
+library( pbdMPI, quiet = TRUE )
 
-Cool, right?
+comm.set.seed( seed = 1234567, diff = T )
+
+test = function( timed )
+{
+  ltime = system.time( timed )[ 3 ]
+
+  mintime = allreduce( ltime, op='min' )
+  maxtime = allreduce( ltime, op='max' )
+  meantime = allreduce( ltime, op='sum' ) / comm.size()
+
+  return( data.frame( min = mintime, mean = meantime, max = maxtime ) )
+}
+
+# generate 10,000,000 random normal values (total)
+times = test( rnorm( 1e7/comm.size() ) ) # ~76 MiB of data
+comm.print( times )
+
+finalize()
+```
+Are there bindings to MPI_wtime()?
+
+### Covariance
+
+```r
+library( pbdMPI, quiet = TRUE )
+
+comm.set.seed( seed = 1234567, diff = TRUE )
+
+## Generate 10 rows and 3 columns of data per process
+my.X = matrix( rnorm(10*3), ncol = 3 )
+
+## Compute mean
+N = allreduce( nrow( my.X ), op = "sum" )
+mu = allreduce( colSums( my.X ) / N, op = "sum" )
+
+## Sweep out mean and compute crossproducts sum
+my.X = sweep( my.X, STATS = mu, MARGIN = 2 )
+Cov.X = allreduce( crossprod( my.X ), op = "sum" ) / ( N - 1 )
+
+comm.print( Cov.X )
+
+finalize()
+```
+
+### Matrix reduction
+```r
+library( pbdMPI, quiet = TRUE )
+
+x <- matrix( 10*comm.rank() + (1:6), nrow = 2 )
+
+comm.print( x, all.rank = TRUE )
+
+z <- reduce( x ) # knows it's a matrix
+
+comm.print( z, all.rank = TRUE )
+
+finalize()
+```
+
+### Ordinary Least Squares
+
+```r
+### Least Squares Fit wia Normal Equations (see lm.fit for a better way)
+library( pbdMPI, quiet = TRUE )
+
+comm.set.seed( seed = 12345, diff = TRUE )
+
+## 10 rows and 3 columns of data per process
+my.X = matrix( rnorm(10*3), ncol = 3 )
+my.y = matrix( rnorm(10*1), ncol = 1 )
+
+## Form the Normal Equations components
+my.Xt = t( my.X )
+XtX = allreduce( my.Xt %*% my.X, op = "sum" )
+Xty = allreduce( my.Xt %*% my.y, op = "sum" )
+
+## Everyone solve the Normal Equations
+ols = solve( XtX, Xty )
+
+comm.print( ols )
+
+finalize()
+```
+
+### QR Decomposition
+
+```r
+library(cop, quiet = TRUE)
+
+rank = comm.rank()
+size = comm.size()
+
+rows = 3
+cols = 3
+xb = matrix((1:(rows*cols*size))^2, ncol = cols) # a full matrix
+xa = xb[(1:rows) + rank*rows, ]  # split by row blocks
+
+comm.print(xa, all.rank = TRUE)
+comm.print(xb)
+
+## compute usual QR from full matrix
+rb = qr.R(qr(xb))
+comm.print(rb)
+
+## compute QR from gathered local QRs
+rloc = qr.R(qr(xa))  # compute local QRs
+rra = allgather(rloc)  # gather them into a list
+rra = do.call(rbind, rra)  # rbind list elements
+comm.print(rra)  # print combined local QRs
+ra = qr.R(qr(rra)) # QR the combined local QRs
+comm.print(ra)
+
+## use cop package to do it again via qr_allreduce
+ra = qr_allreduce(xa)
+comm.print(ra)
+
+finalize()
+```
+
+### Collective communication for a one dimensional domain decomposition
+
+```r
+## Splits the world communicator into two sets of smaller communicators and
+## demonstrates how a sum collective works
+library(pbdMPI)
+.pbd_env$SPMD.CT
+comm_world = .pbd_env$SPMD.CT$comm  # default communicator
+my_rank = comm.rank(comm_world) # my default rank in world communicator
+comm_new = 5L       # new communicators can be 5 and up (0-4 are taken)
+
+row_color = my_rank %/% 2L # set new partition colors and split accordingly
+comm.split(comm_world, color = row_color, key = my_rank, newcomm = comm_new)
+barrier()
+my_newrank = comm.rank(comm_new)
+comm.cat("comm_world:", comm_world, "comm_new", comm_new, "row_color:",
+         row_color, "my_rank:", my_rank, "my_newrank", my_newrank, "\n",
+         all.rank = TRUE)
+x = my_rank + 1
+comm.cat("x", x, "\n", all.rank = TRUE, comm = comm_world)
+xa = allreduce(x, comm = comm_world)
+xb = allreduce(x, comm = comm_new)
+
+comm.cat("xa", xa, "xb", xb, "\n", all.rank = TRUE, comm = comm_world)
+comm.free(comm_new)
+
+
+finalize()
+```
+
+### Collective communication for a two dimensional domain decomposition
+
+```r
+## Run with:
+## mpiexec -np 32 Rscript comm_split8x4.R
+##
+## Splits a 32-rank communicator into 4 row-communicators of size 8 and
+## othogonal to them 8 column communicators of size 4. Prints rank assignments,
+## and demonstrates how sum collectives work in each set of communicators.
+##
+## Useful row ooperations or column operations on tile-distrobued matrices. But
+## note there is package pbdDMAT that already has these operations powered by
+## ScaLAPACK.
+## It can also serve for any two levels of distributed parallelism that are
+## nested.
+##
+library(pbdMPI)
+
+ncol = 8
+nrow = 4
+if(comm.size() != ncol*nrow) stop("Error: Must run with -np 32")
+
+## Get world communicator rank
+comm_w = .pbd_env$SPMD.CT$comm # world communicator (normallly assigned 0)
+rank_w = comm.rank(comm_w)  # world rank
+
+## Split comm_w into ncol communicators of size nrow
+comm_c = 5L               # assign them a number
+color_c = rank_w %/% nrow # ranks of same color are in same communicator
+comm.split(comm_w, color = color_c, key = rank_w, newcomm = comm_c)
+
+## Split comm_w into nrow communicators of size ncol
+comm_r = 6L               # assign them a number
+color_r = rank_w %% nrow  # make these orthogonal to the row communicators
+comm.split(comm_w, color = color_r, key = rank_w, newcomm = comm_r)
+
+## Print the resulting communicator colors and ranks
+comm.cat(comm.rank(comm = comm_w),
+         paste0("(", color_r, ":", comm.rank(comm = comm_r), ")"),
+         paste0("(", color_c, ":", comm.rank(comm = comm_c), ")"),
+         "\n", all.rank = TRUE, quiet = TRUE, comm = comm_w)
+
+## Print sums of rank numbers across each communicator to illustrate collectives
+x = comm.rank(comm_w)
+w = allreduce(x, op = "sum", comm = comm_w)
+comm.cat(" ", w, all.rank = TRUE, quiet = TRUE)
+comm.cat("\n", quiet = TRUE)
+
+r = allreduce(x, op = "sum", comm = comm_r)
+comm.cat(" ", r, all.rank = TRUE, quiet = TRUE)
+comm.cat("\n", quiet = TRUE)
+
+c = allreduce(x, op = "sum", comm = comm_c)
+comm.cat(" ", c, all.rank = TRUE, quiet = TRUE)
+comm.cat("\n", quiet = TRUE)
+
+
+#comm.free(comm_c)
+#comm.free(comm_r)
+
+finalize()
+```
 
 ::::::::::::::::::::::::::::::::::::: keypoints 
 
-- Use `.md` files for episodes when you want static content
-- Use `.Rmd` files for episodes when you need to generate output
-- Run `sandpaper::check_lesson()` to identify any issues with your lesson
-- Run `sandpaper::build_lesson()` to preview your lesson locally
+- The message passing interface offers many operations that can be used to
+  efficiently and portably add parallelizm to your program
+- It is possible to use parallel libraries to minimize the amount of parallel
+  programming you need to do for your data exploration and data analysis
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
-[r-markdown]: https://rmarkdown.rstudio.com/
diff --git a/episodes/07-random-forest-mpi.Rmd b/episodes/07-random-forest-mpi.Rmd
index 1e21fea..a3c7823 100644
--- a/episodes/07-random-forest-mpi.Rmd
+++ b/episodes/07-random-forest-mpi.Rmd
@@ -1,113 +1,152 @@
 ---
-title: "Using RMarkdown"
+title: "MPI - Distributed Memory Parallelizm"
 teaching: 10
 exercises: 2
 ---
 
 :::::::::::::::::::::::::::::::::::::: questions 
 
-- How do you write a lesson using R Markdown and `{sandpaper}`?
+- How do you utilize more than one shared memory node?
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: objectives
 
-- Explain how to use markdown with the new lesson template
-- Demonstrate how to include pieces of code, figures, and nested challenge blocks
+- Demonstrate that distributed memory parallelizm is useful for working with large data
+- Demonstrate that distributed memory parallelizm can lead to improved time to solution
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ## Introduction
 
-This is a lesson created via The Carpentries Workbench. It is written in
-[Pandoc-flavored Markdown](https://pandoc.org/MANUAL.txt) for static files and
-[R Markdown][r-markdown] for dynamic files that can render code into output. 
-Please refer to the [Introduction to The Carpentries 
-Workbench](https://carpentries.github.io/sandpaper-docs/) for full documentation.
+### Distributed Memory Random Forest
 
-What you need to know is that there are three sections required for a valid
-Carpentries lesson template:
+#### Digit Recognition
 
- 1. `questions` are displayed at the beginning of the episode to prime the
-    learner for the content.
- 2. `objectives` are the learning objectives for an episode displayed with
-    the questions.
- 3. `keypoints` are displayed at the end of the episode to reinforce the
-    objectives.
-
-:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: instructor
-
-Inline instructor notes can help inform instructors of timing challenges
-associated with the lessons. They appear in the "Instructor View"
-
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-
-::::::::::::::::::::::::::::::::::::: challenge 
-
-## Challenge 1: Can you do it?
+```r
+suppressPackageStartupMessages(library(randomForest))
+data(LetterRecognition, package = "mlbench")
+library(pbdMPI, quiet = TRUE)                #<<
+comm.set.seed(seed = 7654321, diff = FALSE)      #<<
+
+n = nrow(LetterRecognition)
+n_test = floor(0.2 * n)
+i_test = sample.int(n, n_test)
+train = LetterRecognition[-i_test, ]
+test = LetterRecognition[i_test, ][comm.chunk(n_test, form = "vector"), ]    #<<
+
+comm.set.seed(seed  = 1234, diff = TRUE)          #<<
+my.rf = randomForest(lettr ~ ., train, ntree = comm.chunk(500), norm.votes = FALSE) #<<
+rf.all = allgather(my.rf)                  #<<
+rf.all = do.call(combine, rf.all)          #<<
+pred = as.vector(predict(rf.all, test))
+
+correct = allreduce(sum(pred == test$lettr))  #<<
+comm.cat("Proportion Correct:", correct/(n_test), "\n")
+
+finalize()
+```
 
-What is the output of this command?
+#### Diamond Classification
 
 ```r
-paste("This", "new", "lesson", "looks", "good")
+library(randomForest)
+data(diamonds, package = "ggplot2")
+library(pbdMPI)                                  #<<
+comm.set.seed(seed = 7654321, diff = FALSE)      #<<
+
+n = nrow(diamonds)
+n_test = floor(0.5 * n)
+i_test = sample.int(n, n_test)
+train = diamonds[-i_test, ]
+test = diamonds[i_test, ][comm.chunk(n_test, form = "vector"), ]    #<<
+
+comm.set.seed(seed = 1e6 * runif(1), diff = TRUE)          #<<
+my.rf = randomForest(price ~ ., train, ntree = comm.chunk(100), norm.votes = FALSE) #<<
+rf.all = allgather(my.rf)                  #<<
+rf.all = do.call(combine, rf.all)          #<<
+pred = as.vector(predict(rf.all, test))
+
+sse = sum((pred - test$price)^2)
+comm.cat("MSE =", reduce(sse)/n_test, "\n")
+
+finalize()          #<<
 ```
 
 :::::::::::::::::::::::: solution 
 
-## Output
+#### SLURM submission script
  
-```output
-[1] "This new lesson looks good"
+```bash
+#!/bin/bash
+#SBATCH -J rf
+#SBATCH -A CSC143
+#SBATCH -p batch
+#SBATCH --nodes=1
+#SBATCH -t 00:40:00
+#SBATCH --mem=0
+#SBATCH -e ./rf.e
+#SBATCH -o ./rf.o
+#SBATCH --open-mode=truncate
+
+cd ~/R4HPC/code_5
+pwd
+
+## modules are specific to andes.olcf.ornl.gov
+module load openblas/0.3.17-omp
+module load flexiblas
+flexiblas add OpenBLAS $OLCF_OPENBLAS_ROOT/lib/libopenblas.so
+export LD_PRELOAD=$OLCF_FLEXIBLAS_ROOT/lib64/libflexiblas.so
+module load r
+echo -e "loaded R with FlexiBLAS"
+module list
+
+time Rscript ../code_2/rf_serial.R
+time mpirun --map-by ppr:1:node Rscript rf_mpi.R
+time mpirun --map-by ppr:2:node Rscript rf_mpi.R
+time mpirun --map-by ppr:4:node Rscript rf_mpi.R
+time mpirun --map-by ppr:8:node Rscript rf_mpi.R
+time mpirun --map-by ppr:16:node Rscript rf_mpi.R
+time mpirun --map-by ppr:32:node Rscript rf_mpi.R
 ```
 
 :::::::::::::::::::::::::::::::::
 
 
-## Challenge 2: how do you nest solutions within challenge blocks?
-
 :::::::::::::::::::::::: solution 
 
-You can add a line with at least three colons and a `solution` tag.
-
-:::::::::::::::::::::::::::::::::
-::::::::::::::::::::::::::::::::::::::::::::::::
-
-## Figures
-
-You can also include figures generated from R Markdown:
-
-```{r pyramid, fig.alt = "pie chart illusion of a pyramid", fig.cap = "Sun arise each and every morning"}
-pie(
-  c(Sky = 78, "Sunny side of pyramid" = 17, "Shady side of pyramid" = 5), 
-  init.angle = 315, 
-  col = c("deepskyblue", "yellow", "yellow3"), 
-  border = FALSE
-)
+#### PBS Submission Script
+
+```bash
+#!/bin/bash
+#PBS -N rf
+#PBS -l select=1:ncpus=32
+#PBS -l walltime=00:05:00
+#PBS -q qexp
+#PBS -e rf.e
+#PBS -o rf.o
+
+cd ~/R4HPC/code_5
+pwd
+
+module load R
+echo "loaded R"
+
+time Rscript ../code_2/rf_serial.R
+time mpirun --map-by ppr:1:node Rscript rf_mpi.R
+time mpirun --map-by ppr:2:node Rscript rf_mpi.R
+time mpirun --map-by ppr:4:node Rscript rf_mpi.R
+time mpirun --map-by ppr:8:node Rscript rf_mpi.R
+time mpirun --map-by ppr:16:node Rscript rf_mpi.R
+time mpirun --map-by ppr:32:node Rscript rf_mpi.R
 ```
+:::::::::::::::::::::::::::::::::
 
-Or you can use standard markdown for static figures with the following syntax:
-
-`![optional caption that appears below the figure](figure url){alt='alt text for
-accessibility purposes'}`
-
-![You belong in The Carpentries!](https://raw.githubusercontent.com/carpentries/logo/master/Badge_Carpentries.svg){alt='Blue Carpentries hex person logo with no text.'}
-
-## Math
-
-One of our episodes contains $\LaTeX$ equations when describing how to create
-dynamic reports with {knitr}, so we now use mathjax to describe this:
-
-`$\alpha = \dfrac{1}{(1 - \beta)^2}$` becomes: $\alpha = \dfrac{1}{(1 - \beta)^2}$
-
-Cool, right?
 
 ::::::::::::::::::::::::::::::::::::: keypoints 
 
-- Use `.md` files for episodes when you want static content
-- Use `.Rmd` files for episodes when you need to generate output
-- Run `sandpaper::check_lesson()` to identify any issues with your lesson
-- Run `sandpaper::build_lesson()` to preview your lesson locally
+- Classification can be used for data other than digits, such as diamonds
+- Distributed memory parallelizm can speed up training
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
-[r-markdown]: https://rmarkdown.rstudio.com/
diff --git a/episodes/08-randomized-parallel-svd.Rmd b/episodes/08-randomized-parallel-svd.Rmd
index 1e21fea..c1101e1 100644
--- a/episodes/08-randomized-parallel-svd.Rmd
+++ b/episodes/08-randomized-parallel-svd.Rmd
@@ -1,113 +1,243 @@
 ---
-title: "Using RMarkdown"
+title: "Parallel Randomized Singular Value Decomposition for Classification"
 teaching: 10
 exercises: 2
 ---
 
 :::::::::::::::::::::::::::::::::::::: questions 
 
-- How do you write a lesson using R Markdown and `{sandpaper}`?
+- How well can an alternative parallel classification algorithm work?
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: objectives
 
-- Explain how to use markdown with the new lesson template
-- Demonstrate how to include pieces of code, figures, and nested challenge blocks
+- Introduce the Randomized singular value decomposition (RSVD)
+- Use the randomized singular value decomposition to classify digits
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ## Introduction
 
-This is a lesson created via The Carpentries Workbench. It is written in
-[Pandoc-flavored Markdown](https://pandoc.org/MANUAL.txt) for static files and
-[R Markdown][r-markdown] for dynamic files that can render code into output. 
-Please refer to the [Introduction to The Carpentries 
-Workbench](https://carpentries.github.io/sandpaper-docs/) for full documentation.
 
-What you need to know is that there are three sections required for a valid
-Carpentries lesson template:
+### What is the randomized singular value decomposition?
 
- 1. `questions` are displayed at the beginning of the episode to prime the
-    learner for the content.
- 2. `objectives` are the learning objectives for an episode displayed with
-    the questions.
- 3. `keypoints` are displayed at the end of the episode to reinforce the
-    objectives.
-
-:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: instructor
-
-Inline instructor notes can help inform instructors of timing challenges
-associated with the lessons. They appear in the "Instructor View"
+```r
+rsvd <- function(x, k=1, q=3, retu=TRUE, retvt=TRUE) {
+  
+  n <- ncol(x)
+  
+  if (class(x) == "matrix")
+    Omega <- matrix(runif(n*2L*k), nrow=n, ncol=2L*k)
+  else if (class(x) == "ddmatrix")  #<<
+    Omega <- ddmatrix("runif", nrow=n, ncol=2L*k, bldim=x@bldim, ICTXT=x@ICTXT) #<<
+  
+  Y <- x %*% Omega
+  Q <- qr.Q(qr(Y))
+  
+  for (i in 1:q) {
+    Y <- crossprod(x, Q)
+    Q <- qr.Q(qr(Y))
+    Y <- x %*% Q
+    Q <- qr.Q(qr(Y))
+  }
+  
+  B <- crossprod(Q, x)
+  
+  if (!retu) nu <- 0
+  else nu <- min(nrow(B), ncol(B))
+  
+  if (!retvt) nv <- 0
+  else nv <- min(nrow(B), ncol(B))
+  
+  svd.B <- La.svd(x=B, nu=nu, nv=nv)
+  
+  d <- svd.B$d
+  d <- d[1L:k]
+  
+  # Produce u/vt as desired
+  if (retu) {
+    u <- svd.B$u
+    u <- Q %*% u
+    u <- u[, 1L:k, drop=FALSE]
+  }
+  
+  if (retvt) vt <- svd.B$vt[1L:k, , drop=FALSE]
+  
+  # wrangle return
+  if (retu) {
+    if (retvt) svd <- list(d=d, u=u, vt=vt)
+    else svd <- list(d=d, u=u)
+  } else {
+    if (retvt) svd <- list(d=d, vt=vt)
+    else svd <- list(d=d)
+  }
+  
+  return( svd )
+}
+```
 
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
+## HDF5 and reading in data
 
-::::::::::::::::::::::::::::::::::::: challenge 
+```r
+suppressMessages(library(rhdf5))
+suppressMessages(library(pbdMPI))
+file = "/gpfs/alpine/world-shared/gen011/mnist/train.hdf5"
+dat1  = "image"
+dat2  = "label"
+
+## get and broadcast dimensions to all processors
+if (comm.rank() == 0) {
+   h5f = H5Fopen(file, flags="H5F_ACC_RDONLY")
+   h5d = H5Dopen(h5f, dat1)
+   h5s = H5Dget_space(h5d)
+   dims = H5Sget_simple_extent_dims(h5s)$size
+   H5Dclose(h5d)
+   H5Fclose(h5f)
+} else dims = NA
+dims = bcast(dims)
+
+nlast = dims[length(dims)] # last dim moves slowest
+my_ind = comm.chunk(nlast, form = "vector")
+
+## parallel read of data columns
+my_train = as.double(h5read(file, dat1, index = list(NULL, NULL, my_ind)))
+my_train_lab = as.character(h5read(file, dat2, index = list(my_ind)))
+H5close()
+
+dim(my_train) = c(prod(dims[-length(dims)]), length(my_ind))
+my_train = t(my_train)  # row-major write and column-major read
+my_train = rbind(my_train, my_train, my_train, my_train, my_train, my_train)
+comm.cat("Local dim at rank", comm.rank(), ":", dim(my_train), "\n")
+total_rows = allreduce(nrow(my_train))
+comm.cat("Total dim :", total_rows, ncol(my_train), "\n")
+
+## plot for debugging
+# if(comm.rank() == 0) {
+#   ivals = sample(nrow(my_train), 36)
+#   library(ggplot2)
+#   image = rep(ivals, 28*28)
+#   lab = rep(my_train_lab[ivals], 28*28)
+#   image = factor(paste(image, lab, sep = ": "))
+#   col = rep(rep(1:28, 28), each = length(ivals))
+#   row = rep(rep(1:28, each = 28), each = length(ivals))
+#   im = data.frame(image = image, row = row, col = col,
+#                   val = as.numeric(unlist(my_train[ivals, ])))
+#   print(ggplot(im, aes(row, col, fill = val)) + geom_tile() + facet_wrap(~ image))
+# }
+#barrier()
+## remove finalize if sourced in another script
+#finalize()
+```
 
-## Challenge 1: Can you do it?
 
-What is the output of this command?
+## Using the Randomized Singular Value Decomposition for Classification
 
 ```r
-paste("This", "new", "lesson", "looks", "good")
+source("mnist_read_mpi.R") # reads blocks of rows
+suppressMessages(library(pbdDMAT))
+suppressMessages(library(pbdML))
+init.grid()
+
+## construct block-cyclic ddmatrix
+bldim = c(allreduce(nrow(my_train), op = "max"), ncol(my_train))
+gdim = c(allreduce(nrow(my_train), op = "sum"), ncol(my_train))
+dmat_train = new("ddmatrix", Data = my_train, dim = gdim, 
+                 ldim = dim(my_train), bldim = bldim, ICTXT = 2)
+cyclic_train = as.blockcyclic(dmat_train)
+
+comm.print(comm.size())
+t1 = as.numeric(Sys.time())
+rsvd_train = rsvd(cyclic_train, k = 10, q = 3, retu = FALSE, retvt = TRUE)
+t2 = as.numeric(Sys.time())
+t1 = allreduce(t1, op = "min")
+t2 = allreduce(t2, op = "max")
+comm.cat("Time:", t2 - t1, "seconds\n")
+comm.cat("dim(V):", dim(rsvd_train$vt), "\n")
+
+comm.cat("rsvd top 10 singular values:", rsvd_train$d, "\n")
+
+finalize()
 ```
 
 :::::::::::::::::::::::: solution 
 
-## Output
+#### SLURM Submission Script
  
-```output
-[1] "This new lesson looks good"
+```bash
+#!/bin/bash
+#SBATCH -J rsve
+#SBATCH -A gen011
+#SBATCH -p batch
+#SBATCH --nodes=4
+#SBATCH --mem=0
+#SBATCH -t 00:00:10
+#SBATCH -e ./rsve.e
+#SBATCH -o ./rsve.o
+#SBATCH --open-mode=truncate
+
+## assumes this repository was cloned in your home area
+cd ~/R4HPC/rsvd
+pwd
+
+## modules are specific to andes.olcf.ornl.gov
+module load openblas/0.3.17-omp
+module load flexiblas
+flexiblas add OpenBLAS $OLCF_OPENBLAS_ROOT/lib/libopenblas.so
+export LD_PRELOAD=$OLCF_FLEXIBLAS_ROOT/lib64/libflexiblas.so
+export UCX_LOG_LEVEL=error  # no UCX warn messages
+
+module load r
+echo -e "loaded R with FlexiBLAS"
+module list
+
+time mpirun --map-by ppr:1:node Rscript mnist_rsvd.R
+time mpirun --map-by ppr:2:node Rscript mnist_rsvd.R
+time mpirun --map-by ppr:4:node Rscript mnist_rsvd.R
+time mpirun --map-by ppr:8:node Rscript mnist_rsvd.R
 ```
 
 :::::::::::::::::::::::::::::::::
 
 
-## Challenge 2: how do you nest solutions within challenge blocks?
 
 :::::::::::::::::::::::: solution 
 
-You can add a line with at least three colons and a `solution` tag.
-
-:::::::::::::::::::::::::::::::::
-::::::::::::::::::::::::::::::::::::::::::::::::
-
-## Figures
-
-You can also include figures generated from R Markdown:
-
-```{r pyramid, fig.alt = "pie chart illusion of a pyramid", fig.cap = "Sun arise each and every morning"}
-pie(
-  c(Sky = 78, "Sunny side of pyramid" = 17, "Shady side of pyramid" = 5), 
-  init.angle = 315, 
-  col = c("deepskyblue", "yellow", "yellow3"), 
-  border = FALSE
-)
+#### PBS Submission Script
+
+```bash
+#!/bin/bash
+#PBS -N rsvd
+#PBS -l select=1:mpiprocs=64,walltime=00:10:00
+#PBS -q qexp
+#PBS -e rsvd.e
+#PBS -o rsvd.o
+
+cd ~/ROBUST2022/mpi
+pwd
+
+module load R
+echo "loaded R"
+
+## Fix for warnings from libfabric/1.12 bug
+module swap libfabric/1.12.1-GCCcore-10.3.0 libfabric/1.13.2-GCCcore-11.2.0 
+export UCX_LOG_LEVEL=error
+
+time mpirun --map-by ppr:1:node Rscript mnist_rsvd.R
+time mpirun --map-by ppr:2:node Rscript mnist_rsvd.R
+time mpirun --map-by ppr:4:node Rscript mnist_rsvd.R
+time mpirun --map-by ppr:8:node Rscript mnist_rsvd.R
+time mpirun --map-by ppr:16:node Rscript mnist_rsvd.R
+time mpirun --map-by ppr:32:node Rscript mnist_rsvd.R
+time mpirun --map-by ppr:64:node Rscript mnist_rsvd.R
 ```
-
-Or you can use standard markdown for static figures with the following syntax:
-
-`![optional caption that appears below the figure](figure url){alt='alt text for
-accessibility purposes'}`
-
-![You belong in The Carpentries!](https://raw.githubusercontent.com/carpentries/logo/master/Badge_Carpentries.svg){alt='Blue Carpentries hex person logo with no text.'}
-
-## Math
-
-One of our episodes contains $\LaTeX$ equations when describing how to create
-dynamic reports with {knitr}, so we now use mathjax to describe this:
-
-`$\alpha = \dfrac{1}{(1 - \beta)^2}$` becomes: $\alpha = \dfrac{1}{(1 - \beta)^2}$
-
-Cool, right?
+:::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: keypoints 
 
-- Use `.md` files for episodes when you want static content
-- Use `.Rmd` files for episodes when you need to generate output
-- Run `sandpaper::check_lesson()` to identify any issues with your lesson
-- Run `sandpaper::build_lesson()` to preview your lesson locally
+- There are a variety of machine learning algorithms which can be used for classification
+- Some will work better than others for your data
+- The memory and compute requirements will differ, choose your algorithms and their implementations wisely!
 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
-[r-markdown]: https://rmarkdown.rstudio.com/
diff --git a/learners/setup.Rmd b/learners/setup.Rmd
index 1bd99b3..48e9c1d 100644
--- a/learners/setup.Rmd
+++ b/learners/setup.Rmd
@@ -240,7 +240,8 @@ of new versions every once in a while.
 
 During the course we will need a number of R packages. Packages contain useful
 R code written by other people. We will use the packages
-`pbdR`, `pbdML`, `pbdMPI`, `pbdMAT`, `flexiblas`, `parallel`, and `randomForest`.
+`pbdR`, `pbdML`, `pbdMPI`, `pbdMAT`, `flexiblas`, `memuse`, `mlbench`, `parallel`,
+`remotes` and `randomForest`.
 
 To try to install these packages, open RStudio and copy and paste the following
 command into the console window (look for a blinking cursor on the bottom left),
@@ -249,7 +250,8 @@ to execute the command.
 
 ```{r, eval=FALSE}
 install.packages(c("pbdR", "pbdML", "pbdMPI", "pbdMAT",
-		   "flexiblas", "parallel", "randomForest"))
+		   "flexiblas", "memuse", "mlbench", 
+		   "parallel", "remotes", "randomForest"))
 ```
 
 Alternatively, you can install the packages using RStudio's graphical user
@@ -266,8 +268,11 @@ library(pbdML)
 library(pbdMPI)
 library(pbdMAT)
 library(flexiblas)
+library(memuse)
+library(mlbench)
 library(parallel)
 library(randomForest)
+library(remotes)
 ```
 
 If you do not see an error like `there is no package called ‘...'` you are good