", "", sep = "\n"
- )
- )
-
-knitr::knit_engines$set(text_answer = eng_text_answer)
diff --git a/episodes/visualizing-ggplot.Rmd b/episodes/visualizing-ggplot.Rmd
new file mode 100644
index 000000000..153869c91
--- /dev/null
+++ b/episodes/visualizing-ggplot.Rmd
@@ -0,0 +1,603 @@
+---
+title: "Data visualization with ggplot2"
+teaching: 90
+exercises: 4
+---
+
+:::::::::::::::::::::::::::::::::::::: questions
+
+- How do you make plots using R?
+- How do you customize and modify plots?
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::: objectives
+
+- Produce scatter plots and boxplots using `ggplot2`.
+- Represent data variables with plot components.
+- Modify the scales of plot components.
+- Iteratively build and modify `ggplot2` plots by adding layers.
+- Change the appearance of existing `ggplot2` plots using premade and customized themes.
+- Describe what faceting is and apply faceting in `ggplot2`.
+- Save plots as image files.
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Setup
+
+```{r setup, include=FALSE}
+knitr::opts_chunk$set(dpi = 200, out.height = 600, out.width = 600, R.options = list(max.print = 100))
+```
+
+We are going to be using **functions** from the **`ggplot2`** package to create visualizations of data. Functions are predefined bits of code that automate more complicated actions. R itself has many built-in functions, but we can access many more by loading other **packages** of functions and data into R.
+
+If you don't have a blank, untitled script open yet, go ahead and open one with Shift+Cmd+N (Mac) or Shift+Ctrl+N (Windows). Then save the file to your `scripts/` folder, and title it `workshop_code.R`.
+
+Earlier, you had to **install** the `ggplot2` package by running `install.packages("ggplot2")`. That installed the package onto your computer so that R can access it. In order to use it in our current session, we have to **load** the package using the `library()` function.
+
+::::::::::::::::::::::::::::: callout
+
+If you do not have `ggplot2` installed, you can run `install.packages("ggplot2")` in the **console**.
+
+It is a good practice not to put `install.packages()` into a script. This is because every time you run that whole script, the package will be reinstalled, which is typically unnecessary. You want to install the package to your computer once, and then load it with `library()` in each script where you need to use it.
+
+:::::::::::::::::::::::::::::::::::::
+
+```{r load-tidyverse-package, message=FALSE}
+library(ggplot2)
+```
+
+Later we will learn how to read data from external files into R, but for now we are going to use a clean and ready-to-use dataset that is provided by the **`ratdat`** data package. To make our dataset available, we need to load this package too.
+
+```{r load-ratdat-package,message=FALSE}
+library(ratdat)
+```
+
+The `ratdat` package contains data from the [Portal Project](https://github.com/weecology/PortalData), which is a long-term dataset from Portal, Arizona, in the Chihuahuan desert.
+
+We will be using a dataset called `complete_old`, which contains older years of survey data. Let's try to learn a little bit about the data. We can use a `?` in front of the name of the dataset, which will bring up the help page for the data.
+
+```{r data-help}
+?complete_old
+```
+
+Here we can read descriptions of each variable in our data.
+
+To actually take a look at the data, we can use the `View()` function to open an interactive viewer, which behaves like a simplified version of a spreadsheet program. It's a handy function, but somewhat limited when trying to view large datasets.
+
+```{r view-data, eval=FALSE}
+View(complete_old)
+```
+
+If you hover over the tab for the interactive `View()`, you can click the "x" that appears, which will close the tab.
+
+We can find out more about the dataset by using the `str()` function to examine the **str**ucture of the data.
+
+```{r str}
+str(complete_old)
+```
+
+`str()` will tell us how many observations/rows (obs) and variables/columns we have, as well as some information about each of the variables. We see the name of a variable (such as `year`), followed by the kind of variable (**int** for integer, **chr** for character), and the first 10 entries in that variable. We will talk more about different data types and structures later on.
+
+## Plotting with **`ggplot2`**
+
+:::::::::::::::::::::::::::::::::::::::::::: instructor
+
+Probably worth mentioning that people often just say **ggplot** when referring to the package `ggplot2`.
+
+::::::::::::::::::::::::::::::::::::::::::::
+
+**`ggplot2`** is a powerful package that allows you to create complex plots from tabular data (data in a table format with rows and columns). The **`gg`** in **`ggplot2`** stands for "grammar of graphics", and the package uses consistent vocabulary to create plots of widely varying types. Therefore, we only need small changes to our code if the underlying data changes or we decide to make a box plot instead of a scatter plot. This approach helps you create publication-quality plots with minimal adjusting and tweaking.
+
+**`ggplot2`** is part of the **`tidyverse`** series of packages, which tend to like data in the "long" or "tidy" format, which means each column represents a single variable, and each row represents a single observation. Well-structured data will save you lots of time making figures with **`ggplot2`**. For now, we will use data that are already in this format. We start learning R by using **`ggplot2`** because it relies on concepts that we will need when we talk about data transformation in the next lessons.
+
+**`ggplot`** plots are built step by step by adding new layers, which allows for extensive flexibility and customization of plots.
+
+::::::::::::::::::::::::::::: callout
+
+Some languages, like Python, require certain spacing or indentation for code to run properly. This isn't the case in R, so if you see spaces or indentation in the code from this lesson, it is to improve readability.
+
+:::::::::::::::::::::::::::::
+
+To build a plot, we will use a basic template that can be used for different types of plots:
+
+```r
+ggplot(data = , mapping = aes()) + ()
+```
+
+We use the `ggplot()` function to create a plot. In order to tell it what data to use, we need to specify the `data` **argument**. An argument is an input that a function takes, and you set arguments using the `=` sign.
+
+```{r bare-plot}
+ggplot(data = complete_old)
+```
+
+We get a blank plot because we haven't told `ggplot()` which variables we want to correspond to parts of the plot. We can specify the "mapping" of variables to plot elements, such as x/y coordinates, size, or shape, by using the `aes()` function. We'll also add a comment, which is any line starting with a `#`. It's a good idea to use comments to organize your code or clarify what you are doing.
+
+```{r plot-with-axes}
+# adding a mapping to x and y axes
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length))
+```
+
+Now we've got a plot with x and y axes corresponding to variables from `complete_old`. However, we haven't specified how we want the data to be displayed. We do this using `geom_` functions, which specify the type of `geom`etry we want, such as points, lines, or bars. We can add a `geom_point()` layer to our plot by using the `+` sign. We indent onto a new line to make it easier to read, and we have to **end** the first line with the `+` sign.
+
+```{r scatter-plot}
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length)) +
+ geom_point()
+```
+
+You may notice a warning that missing values were removed. If a variable necessary to make the plot is missing from a given row of data (in this case, `hindfoot_length` or `weight`), it can't be plotted. `ggplot2` just uses a warning message to let us know that some rows couldn't be plotted.
+
+
+::::::::::::::::::::::::::::: callout
+
+**Warning** messages are one of a few ways R will communicate with you. Warnings can be thought of as a "heads up". Nothing necessarily went *wrong*, but the author of that function wanted to draw your attention to something. In the above case, it's worth knowing that some of the rows of your data were not plotted because they had missing data.
+
+A more serious type of message is an **error**. Here's an example:
+
+```{r geom-error}
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length)) +
+ geom_poit()
+```
+
+As you can see, we only get the error message, with no plot, because something has actually gone wrong. This particular error message is fairly common, and it happened because we misspelled `point` as `poit`. Because there is no function named `geom_poit()`, R tells us it can't find a function with that name.
+
+:::::::::::::::::::::::::::::
+
+## Changing aesthetics
+
+Building **`ggplot`** plots is often an iterative process, so we'll continue developing the scatter plot we just made. You may have noticed that parts of our scatter plot have many overlapping points, making it difficult to see all the data. We can adjust the transparency of the points using the `alpha` argument, which takes a value between 0 and 1:
+
+```{r change-alpha, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length)) +
+ geom_point(alpha = 0.2)
+```
+
+We can also change the color of the points:
+
+```{r change-color, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length)) +
+ geom_point(alpha = 0.2, color = "blue")
+```
+
+::::::::::::::::::::::::::::: callout
+
+Two common issues you might run into when working in R are forgetting a closing bracket or a closing quote. Let's take a look at what each one does.
+
+Try running the following code:
+
+```{r no-closing-bracket, eval=FALSE, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length)) +
+ geom_point(color = "blue", alpha = 0.2
+```
+
+You will see a `+` appear in your console. This is R telling you that it expects more input in order to finish running the code. It is missing a closing bracket to end the `geom_point` function call. You can hit Esc in the console to reset it.
+
+Something similar will happen if you run the following code:
+
+```{r no-closing-quote, eval=FALSE, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length)) +
+ geom_point(color = "blue, alpha = 0.2)
+```
+
+A missing quote at the end of `blue` means that the rest of the code is treated as part of the quote, which is a bit easier to see since RStudio displays character strings in a different color.
+
+You will get a different error message if you run the following code:
+
+```{r extra-bracket, eval=FALSE, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length)) +
+ geom_point(color = "blue", alpha = 0.2))
+```
+
+This time we have an extra closing `)`, which R doesn't know what to do with. It tells you there is an unexpected `)`, but it doesn't pinpoint exactly where. With enough time working in R, you will get better at spotting mismatched brackets.
+
+:::::::::::::::::::::::::::::
+
+### Adding another variable
+
+Let's try coloring our points according to the sampling plot type (plot here refers to the physical area where rodents were sampled and has nothing to do with making graphs). Since we're now mapping a variable (`plot_type`) to a component of the ggplot2 plot (`color`), we need to put the argument inside `aes()`:
+
+```{r color-plot-type, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length, color = plot_type)) +
+ geom_point(alpha = 0.2)
+```
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 1: Modifying plots
+
+1. Try modifying the plot so that the `shape` of the point varies by `sex`. You will set the `shape` the same way you set the `color`.
+
+Do you think this is a good way to represent `sex` with these data?
+
+:::::::::::::::::::::::: solution
+
+```{r modify-points-challenge-answer, warning=FALSE}
+ggplot(data = complete_old,
+ mapping = aes(x = weight, y = hindfoot_length, shape = sex)) +
+ geom_point(alpha = 0.2)
+```
+
+::::::::::::::::::::::::
+
+2. Now try changing the plot so that the `color` of the points vary by `year`. Do you notice a difference in the color scale compared to changing color by plot type? Why do you think this happened?
+
+:::::::::::::::::::::::: solution
+
+```{r modify-color-challenge-answer, warning=FALSE}
+ggplot(data = complete_old,
+ mapping = aes(x = weight, y = hindfoot_length, color = year)) +
+ geom_point(alpha = 0.2)
+```
+
+- For Part 2, the color scale is different compared to using `color = plot_type` because `plot_type` and `year` are different variable types. `plot_type` is a categorical variable, so `ggplot2` defaults to use a **discrete** color scale, whereas `year` is a numeric variable, so `ggplot2` uses a **continuous** color scale.
+
+::::::::::::::::::::::::
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Changing scales
+
+The default discrete color scale isn't always ideal: it isn't friendly to viewers with colorblindness and it doesn't translate well to grayscale. However, **`ggplot2`** comes with quite a few other color scales, including the fantastic `viridis` scales, which are designed to be colorblind and grayscale friendly. We can change scales by adding `scale_` functions to our plots:
+
+```{r scale-viridis, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length, color = plot_type)) +
+ geom_point(alpha = 0.2) +
+ scale_color_viridis_d()
+```
+
+Scales don't just apply to colors- any plot component that you put inside `aes()` can be modified with `scale_` functions. Just as we modified the scale used to map `plot_type` to `color`, we can modify the way that `weight` is mapped to the `x` axis by using the `scale_x_log10()` function:
+
+```{r scale-log, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = weight, y = hindfoot_length, color = plot_type)) +
+ geom_point(alpha = 0.2) +
+ scale_x_log10()
+```
+
+One nice thing about `ggplot` and the `tidyverse` in general is that groups of functions that do similar things are given similar names. Any function that modifies a `ggplot` scale starts with `scale_`, making it easier to search for the right function.
+
+## Boxplot
+
+Let's try making a different type of plot altogether. We'll start off with our same basic building blocks using `ggplot()` and `aes()`.
+
+```{r blank-boxplot}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length))
+```
+
+This time, let's try making a boxplot, which will have `plot_type` on the x axis and `hindfoot_length` on the y axis. We can do this by adding `geom_boxplot()` to our `ggplot()`:
+
+```{r boxplot}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length)) +
+ geom_boxplot()
+```
+
+Just as we colored the points before, we can color our boxplot by `plot_type` as well:
+
+```{r boxplot-color, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length, color = plot_type)) +
+ geom_boxplot()
+```
+
+It looks like `color` has only affected the outlines of the boxplot, not the rectangular portions. This is because the `color` only impacts 1-dimensional parts of a `ggplot`: points and lines. To change the color of 2-dimensional parts of a plot, we use `fill`:
+
+```{r boxplot-fill, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length, fill = plot_type)) +
+ geom_boxplot()
+```
+
+::::::::::::::::::::::::::::: callout
+
+One thing you may notice is that the axis labels are overlapping each other, depending on how wide your plot viewer is. One way to help make them more legible is to **wrap** the text. We can do that by modifying the **labels** for the `x` axis `scale`.
+
+We use the `scale_x_discrete()` function because we have a discrete axis, and we modify the `labels` argument. The function `label_wrap_gen()` will wrap the text of the labels to make them more legible.
+
+```{r boxplot-label-wrap, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length, fill = plot_type)) +
+ geom_boxplot() +
+ scale_x_discrete(labels = label_wrap_gen(width = 10))
+```
+
+:::::::::::::::::::::::::::::
+
+## Adding geoms
+
+One of the most powerful aspects of **`ggplot`** is the way we can add components to a plot in successive layers. While boxplots can be very useful for summarizing data, it is often helpful to show the raw data as well. With **`ggplot`**, we can easily add another `geom_` to our plot to show the raw data.
+
+Let's add `geom_point()` to visualize the raw data. We will modify the `alpha` argument to help with overplotting.
+
+```{r boxplot-points, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length)) +
+ geom_boxplot() +
+ geom_point(alpha = 0.2)
+```
+
+Uh oh... all our points for a given `x` axis category fall exactly on a line, which isn't very useful. We can shift to using `geom_jitter()`, which will add points with a bit of random noise added to the positions to prevent this from happening.
+
+```{r boxplot-jitter, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length)) +
+ geom_boxplot() +
+ geom_jitter(alpha = 0.2)
+```
+
+You may have noticed that some of our data points are now appearing on our plot twice: the outliers are plotted as black points from `geom_boxplot()`, but they are also plotted with `geom_jitter()`. Since we don't want to represent these data multiple times in the same form (points), we can stop `geom_boxplot()` from plotting them. We do this by setting the `outlier.shape` argument to `NA`, which means the outliers don't have a shape to be plotted.
+
+```{r boxplot-outliers, warning = F}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length)) +
+ geom_boxplot(outlier.shape = NA) +
+ geom_jitter(alpha = 0.2)
+```
+
+Just as before, we can map `plot_type` to `color` by putting it inside `aes()`.
+
+```{r global-color, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length, color = plot_type)) +
+ geom_boxplot(outlier.shape = NA) +
+ geom_jitter(alpha = 0.2)
+```
+
+Notice that both the color of the points and the color of the boxplot lines changed. Any time we specify an `aes()` mapping inside our initial `ggplot()` function, that mapping will apply to all our `geom`s.
+
+If we want to limit the mapping to a single `geom`, we can put the mapping into the specific `geom_` function, like this:
+
+```{r geom-color, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length)) +
+ geom_boxplot(outlier.shape = NA) +
+ geom_jitter(aes(color = plot_type), alpha = 0.2)
+```
+
+Now our points are colored according to `plot_type`, but the boxplots are all the same color. One thing you might notice is that even with `alpha = 0.2`, the points obscure parts of the boxplot. This is because the `geom_point()` layer comes after the `geom_boxplot()` layer, which means the points are plotted on top of the boxes. To put the boxplots on top, we switch the order of the layers:
+
+```{r reverse-layers, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length)) +
+ geom_jitter(aes(color = plot_type), alpha = 0.2) +
+ geom_boxplot(outlier.shape = NA)
+```
+
+Now we have the opposite problem! The white `fill` of the boxplots completely obscures some of the points. To address this problem, we can remove the `fill` from the boxplots altogether, leaving only the black lines. To do this, we set `fill` to `NA`:
+
+```{r fill-na, warning=FALSE}
+ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length)) +
+ geom_jitter(aes(color = plot_type), alpha = 0.2) +
+ geom_boxplot(outlier.shape = NA, fill = NA)
+```
+
+Now we can see all the raw data and our boxplots on top.
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 2: Change `geom`s
+
+Violin plots are similar to boxplots- try making one using `plot_type` and `hindfoot_length` as the x and y variables. Remember that all geom functions start with `geom_`, followed by the type of geom.
+
+This might also be a place to test your search engine skills. It is often useful to search for `R package_name stuff you want to search`. So for this example we might search for `R ggplot2 violin plot`.
+
+:::::::::::::::::::::::: solution
+
+```{r violin-challenge-answer, warning=FALSE}
+ggplot(data = complete_old,
+ mapping = aes(x = plot_type,
+ y = hindfoot_length,
+ color = plot_type)) +
+ geom_jitter(alpha = 0.2) +
+ geom_violin(fill = "white")
+```
+
+::::::::::::::::::::::::
+
+For an *extra challenge*, , make the color of the points and outlines of the violins vary by `plot_type`, and set the fill of the violins to white. Try playing with the order of the layers to see what looks best.
+
+:::::::::::::::::::::::: solution
+
+```{r violin-challenge-answer-2, warning=FALSE}
+ggplot(data = complete_old,
+ mapping = aes(x = plot_type,
+ y = hindfoot_length,
+ color = plot_type)) +
+ geom_jitter(alpha = 0.2) +
+ geom_violin(fill = "white")
+```
+
+::::::::::::::::::::::::
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Changing themes
+
+So far we've been changing the appearance of parts of our plot related to our data and the `geom_` functions, but we can also change many of the non-data components of our plot.
+
+At this point, we are pretty happy with the basic layout of our plot, so we can **assign** it to a plot to a named **object**. We do this using the **assignment arrow** `<-`. What we are doing here is taking the result of the code on the right side of the arrow, and assigning it to an object whose name is on the left side of the arrow.
+
+We will create an object called `myplot`. If you run the name of the `ggplot2` object, it will show the plot, just like if you ran the code itself.
+
+```{r}
+myplot <- ggplot(data = complete_old, mapping = aes(x = plot_type, y = hindfoot_length)) +
+ geom_jitter(aes(color = plot_type), alpha = 0.2) +
+ geom_boxplot(outlier.shape = NA, fill = NA)
+
+myplot
+```
+
+This process of assigning something to an **object** is not specific to `ggplot2`, but rather a general feature of R. We will be using it a lot in the rest of this lesson. We can now work with the `myplot` object as if it was a block of `ggplot2` code, which means we can use `+` to add new components to it.
+
+We can change the overall appearance using `theme_` functions. Let's try a black-and-white theme by adding `theme_bw()` to our plot:
+
+```{r theme-bw, warning=FALSE}
+myplot + theme_bw()
+```
+
+As you can see, a number of parts of the plot have changed. `theme_` functions usually control many aspects of a plot's appearance all at once, for the sake of convenience. To individually change parts of a plot, we can use the `theme()` function, which can take many different arguments to change things about the text, grid lines, background color, and more. Let's try changing the size of the text on our axis titles. We can do this by specifying that the `axis.title` should be an `element_text()` with `size` set to 14.
+
+```{r text-size, warning=FALSE}
+myplot +
+ theme_bw() +
+ theme(axis.title = element_text(size = 14))
+```
+
+Another change we might want to make is to remove the vertical grid lines. Since our x axis is categorical, those grid lines aren't useful. To do this, inside `theme()`, we will change the `panel.grid.major.x` to an `element_blank()`.
+
+```{r element-blank, warning=FALSE}
+myplot +
+ theme_bw() +
+ theme(axis.title = element_text(size = 14),
+ panel.grid.major.x = element_blank())
+```
+
+Another useful change might be to remove the color legend, since that information is already on our x axis. For this one, we will set `legend.position` to "none".
+
+```{r legend-remove, warning=FALSE}
+myplot +
+ theme_bw() +
+ theme(axis.title = element_text(size = 14),
+ panel.grid.major.x = element_blank(),
+ legend.position = "none")
+
+```
+
+::::::::::::::::::::::::::::: callout
+
+Because there are so many possible arguments to the `theme()` function, it can sometimes be hard to find the right one. Here are some tips for figuring out how to modify a plot element:
+
+- type out `theme()`, put your cursor between the parentheses, and hit Tab to bring up a list of arguments
+ - you can scroll through the arguments, or start typing, which will shorten the list of potential matches
+- like many things in the `tidyverse`, similar argument start with similar names
+ - there are `axis`, `legend`, `panel`, `plot`, and `strip` arguments
+- arguments have hierarchy
+ - `text` controls all text in the whole plot
+ - `axis.title` controls the text for the axis titles
+ - `axis.title.x` controls the text for the x axis title
+
+:::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::: callout
+
+You may have noticed that we have used 3 different approaches to getting rid of something in `ggplot`:
+
+- `outlier.shape = NA` to remove the outliers from our boxplot
+- `panel.grid.major.x = element_blank()` to remove the x grid lines
+- `legend.position = "none"` to remove our legend
+
+Why are there so many ways to do what seems like the same thing?? This is a common frustration when working with R, or with any programming language. There are a couple reasons for it:
+
+1. Different people contribute to different packages and functions, and they may choose to do things differently.
+2. Code may *appear* to be doing the same thing, when the details are actually quite different. The inner workings of `ggplot2` are actually quite complex, since it turns out making plots is a very complicated process! Because of this, things that seem the same (removing parts of a plot), may actually be operating on very different components or stages of the final plot.
+3. Developing packages is a highly iterative process, and sometimes things change. However, changing too much stuff can make old code break. Let's say removing the legend was introduced as a feature of `ggplot2`, and then a lot of time passed before someone added the feature letting you remove outliers from `geom_boxplot()`. Changing the way you remove the legend, so that it's the same as the boxplot approach, could break all of the code written in the meantime, so developers may opt to keep the old approach in place.
+
+:::::::::::::::::::::::::::::
+
+## Changing labels
+
+Our plot is really shaping up now. However, we probably want to make our axis titles nicer, and perhaps add a main title to the plot. We can do this using the `labs()` function:
+
+```{r labels, warning=FALSE}
+myplot +
+ theme_bw() +
+ theme(axis.title = element_text(size = 14),
+ legend.position = "none") +
+ labs(title = "Rodent size by plot type",
+ x = "Plot type",
+ y = "Hindfoot length (mm)")
+```
+
+We removed our legend from this plot, but you can also change the titles of various legends using `labs()`. For example, `labs(color = "Plot type")` would change the title of a color scale legend to "Plot type".
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 3: Customizing a plot
+
+Modify the previous plot by adding a descriptive subtitle. Increase the font size of the plot title and make it bold.
+
+**Hint**: "bold" is referred to as a font "face"
+
+:::::::::::::::::::::::: solution
+
+```{r customizing-challenge-answer, warning=FALSE}
+myplot +
+ theme_bw() +
+ theme(axis.title = element_text(size = 14), legend.position = "none",
+ plot.title = element_text(face = "bold", size = 20)) +
+ labs(title = "Rodent size by plot type",
+ subtitle = "Long-term dataset from Portal, AZ",
+ x = "Plot type",
+ y = "Hindfoot length (mm)")
+```
+
+::::::::::::::::::::::::
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Faceting
+
+One of the most powerful features of **`ggplot`** is the ability to quickly split a plot into multiple smaller plots based on a categorical variable, which is called **faceting**.
+
+So far we've mapped variables to the x axis, the y axis, and color, but trying to add a 4th variable becomes difficult. Changing the shape of a point might work, but only for very few categories, and even then, it can be hard to tell the differences between the shapes of small points.
+
+Instead of cramming one more variable into a single plot, we will use the `facet_wrap()` function to generate a series of smaller plots, split out by `sex`. We also use `ncol` to specify that we want them arranged in a single column:
+
+```{r facet-wrap, warning=FALSE}
+myplot +
+ theme_bw() +
+ theme(axis.title = element_text(size = 14),
+ legend.position = "none",
+ panel.grid.major.x = element_blank()) +
+ labs(title = "Rodent size by plot type",
+ x = "Plot type",
+ y = "Hindfoot length (mm)",
+ color = "Plot type") +
+ facet_wrap(vars(sex), ncol = 1)
+```
+
+::::::::::::::::::::::::::::: callout
+
+Faceting comes in handy in many scenarios. It can be useful when:
+
+- a categorical variable has too many levels to differentiate by color (such as a dataset with 20 countries)
+- your data overlap heavily, obscuring categories
+- you want to show more than 3 variables at once
+- you want to see each category in isolation while allowing for general comparisons between categories
+
+:::::::::::::::::::::::::::::::::::::
+
+## Exporting plots
+
+Once we are happy with our final plot, we can assign the whole thing to a new object, which we can call `finalplot`.
+
+```{r assign-finalplot}
+finalplot <- myplot +
+ theme_bw() +
+ theme(axis.title = element_text(size = 14),
+ legend.position = "none",
+ panel.grid.major.x = element_blank()) +
+ labs(title = "Rodent size by plot type",
+ x = "Plot type",
+ y = "Hindfoot length (mm)",
+ color = "Plot type") +
+ facet_wrap(vars(sex), ncol = 1)
+```
+
+After this, we can run `ggsave()` to save our plot. The first argument we give is the path to the file we want to save, including the correct file extension. This code will make an image called `rodent_size_plots.jpg` in the `images/` folder of our current project. We are making a `.jpg`, but you can save `.pdf`, `.tiff`, and other file formats. Next, we tell it the name of the plot object we want to save. We can also specify things like the width and height of the plot in inches.
+
+```{r save-plot, eval=FALSE}
+ggsave(filename = "images/rodent_size_plots.jpg", plot = finalplot,
+ height = 6, width = 8)
+```
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 4: Make your own plot
+
+Try making your own plot! You can run `str(complete_old)` or `?complete_old` to explore variables you might use in your new plot. Feel free to use variables we have already seen, or some we haven't explored yet.
+
+Here are a couple ideas to get you started:
+
+ - make a histogram of one of the numeric variables
+ - try using a different color `scale_`
+ - try changing the size of points or thickness of lines in a `geom`
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+::::::::::::::::::::::::::::::::::::: keypoints
+
+- the `ggplot()` function initiates a plot, and `geom_` functions add representations of your data
+- use `aes()` when mapping a variable from the data to a part of the plot
+- use `scale_` functions to modify the scales used to represent variables
+- use premade `theme_` functions to broadly change appearance, and the `theme()` function to fine-tune
+- start simple and build your plots iteratively
+
+::::::::::::::::::::::::::::::::::::::::::::::::
diff --git a/episodes/working-with-data.Rmd b/episodes/working-with-data.Rmd
new file mode 100644
index 000000000..6194d7915
--- /dev/null
+++ b/episodes/working-with-data.Rmd
@@ -0,0 +1,638 @@
+---
+title: "Working with data"
+teaching: 120
+exercises: 4
+---
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+:::::::::::::::::::::::::::::::::::::: questions
+
+- How do you manipulate tabular data in R?
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::: objectives
+
+- Import CSV data into R.
+- Understand the difference between base R and `tidyverse` approaches.
+- Subset rows and columns of data.frames.
+- Use pipes to link steps together into pipelines.
+- Create new data.frame columns using existing columns.
+- Utilize the concept of split-apply-combine data analysis.
+- Reshape data between wide and long formats.
+- Export data to a CSV file.
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+```{r setup, include=FALSE}
+knitr::opts_chunk$set(dpi = 200, out.height = 600, out.width = 600, R.options = list(max.print = 100))
+```
+
+```{r load-tidyverse-package, message=FALSE}
+library(tidyverse)
+```
+
+## Importing data
+
+Up until this point, we have been working with the `complete_old` dataframe contained in the `ratdat` package. However, you typically won't access data from an R package; it is much more common to access data files stored somewhere on your computer. We are going to download a CSV file containing the surveys data to our computer, which we will then read into R.
+
+Click this link to download the file: .
+
+You will be prompted to save the file on your computer somewhere. Save it inside the `cleaned` data folder, which is in the `data` folder in your `R-Ecology-Workshop` folder. Once it's inside our project, we will be able to point R towards it.
+
+#### File paths
+
+When we reference other files from an R script, we need to give R precise instructions on where those files are. We do that using something called a **file path**. It looks something like this: `"Documents/Manuscripts/Chapter_2.txt"`. This path would tell your computer how to get from whatever folder contains the `Documents` folder all the way to the `.txt` file.
+
+There are two kinds of paths: **absolute** and **relative**. Absolute paths are specific to a particular computer, whereas relative paths are relative to a certain folder. Because we are keeping all of our work in the `R-Ecology-Workshop` folder, all of our paths can be relative to this folder.
+
+Now, let's read our CSV file into R and store it in an object named `surveys`. We will use the `read_csv` function from the `tidyverse`'s `readr` package, and the argument we give will be the **relative path** to the CSV file.
+
+```{r read-csv}
+surveys <- read_csv("data/cleaned/surveys_complete_77_89.csv")
+```
+
+:::::::::::::::::::::::::::::::::::::::::: callout
+
+Typing out paths can be error prone, so we can utilize a keyboard shortcut. Inside the parentheses of `read_csv()`, type out a pair of quotes and put your cursor between them. Then hit Tab. A small menu showing your folders and files should show up. You can use the ↑ and ↓ keys to move through the options, or start typing to narrow them down. You can hit Enter to select a file or folder, and hit Tab again to continue building the file path. This might take a bit of getting used to, but once you get the hang of it, it will speed up writing file paths and reduce the number of mistakes you make.
+
+::::::::::::::::::::::::::::::::::::::::::
+
+You may have noticed a bit of feedback from R when you ran the last line of code. We got some useful information about the CSV file we read in. We can see:
+
+- the number of rows and columns
+- the **delimiter** of the file, which is how values are separated, a comma `","`
+- a set of columns that were **parsed** as various vector types
+ - the file has `r surveys %>% select(where(is.character)) %>% ncol()` character columns and `r surveys %>% select(where(is.numeric)) %>% ncol()` numeric columns
+ - we can see the names of the columns for each type
+
+
+When working with the output of a new function, it's often a good idea to check the `class()`:
+
+```{r class-tibble}
+class(surveys)
+```
+
+Whoa! What is this thing? It has multiple classes? Well, it's called a `tibble`, and it is the `tidyverse` version of a data.frame. It *is* a data.frame, but with some added perks. It prints out a little more nicely, it highlights `NA` values and negative values in red, and it will generally communicate with you more (in terms of warnings and errors, which is a good thing).
+
+:::::::::::::::::::::::::::::::::::::::::: callout
+
+**`tidyverse` vs. base R**
+
+As we begin to delve more deeply into the `tidyverse`, we should briefly pause to mention some of the reasons for focusing on the `tidyverse` set of tools. In R, there are often many ways to get a job done, and there are other approaches that can accomplish tasks similar to the `tidyverse`.
+
+The phrase **base R** is used to refer to approaches that utilize functions contained in R's default packages. We have already used some base R functions, such as `str()`, `head()`, and `mean()`, and we will be using more scattered throughout this lesson. However, there are some key base R approaches we will not be teaching. These include square bracket subsetting and base plotting. You may come across code written by other people that looks like `surveys[1:10, 2]` or `plot(surveys$weight, surveys$hindfoot_length)`, which are base R commands. If you're interested in learning more about these approaches, you can check out other Carpentries lessons like the [Software Carpentry Programming with R](https://swcarpentry.github.io/r-novice-inflammation/) lesson.
+
+We choose to teach the `tidyverse` set of packages because they share a similar syntax and philosophy, making them consistent and producing highly readable code. They are also very flexible and powerful, with a growing number of packages designed according to similar principles and to work well with the rest of the packages. The `tidyverse` packages tend to have very clear documentation and wide array of learning materials that tend to be written with novice users in mind. Finally, the `tidyverse` has only continued to grow, and has strong support from RStudio, which implies that these approaches will be relevant into the future.
+
+::::::::::::::::::::::::::::::::::::::::::
+
+## Manipulating data
+
+One of the most important skills for working with data in R is the ability to manipulate, modify, and reshape data. The `dplyr` and `tidyr` packages in the `tidyverse` provide a series of powerful functions for many common data manipulation tasks.
+
+We'll start off with two of the most commonly used `dplyr` functions: `select()`, which selects certain columns of a data.frame, and `filter()`, which filters out rows according to certain criteria.
+
+:::::::::::::::::::::::::::::::::::::::::: callout
+
+Between `select()` and `filter()`, it can be hard to remember which operates on columns and which operates on rows. `sele`**`c`**`t()` has a **c** for **c**olumns and `filte`**`r`**`()` has an **r** for **r**ows.
+
+::::::::::::::::::::::::::::::::::::::::::
+
+#### `select()`
+
+To use the `select()` function, the first argument is the name of the data.frame, and the rest of the arguments are *unquoted* names of the columns you want:
+
+```{r select}
+select(surveys, plot_id, species_id, hindfoot_length)
+```
+
+The columns are arranged in the order we specified inside `select()`.
+
+To select all columns except specific columns, put a `-` in front of the column you want to exclude:
+
+```{r select-minus}
+select(surveys, -record_id, -year)
+```
+
+`select()` also works with numeric vectors for the order of the columns. To select the 3rd, 4th, 5th, and 10th columns, we could run the following code:
+
+```{r select-vector}
+select(surveys, c(3:5, 10))
+```
+
+You should be careful when using this method, since you are being less explicit about which columns you want. However, it can be useful if you have a data.frame with many columns and you don't want to type out too many names.
+
+Finally, you can select columns based on whether they match a certain criteria by using the `where()` function. If we want all numeric columns, we can ask to `select` all the columns `where` the class `is numeric`:
+
+```{r select-where}
+select(surveys, where(is.numeric))
+```
+
+Instead of giving names or positions of columns, we instead pass the `where()` function with the name of another function inside it, in this case `is.numeric()`, and we get all the columns for which that function returns `TRUE`.
+
+We can use this to select any columns that have any `NA` values in them:
+
+```{r select-anyNA}
+select(surveys, where(anyNA))
+```
+
+#### `filter()`
+
+The `filter()` function is used to select rows that meet certain criteria. To get all the rows where the value of `year` is equal to 1985, we would run the following:
+
+```{r filter}
+filter(surveys, year == 1985)
+```
+
+The `==` sign means "is equal to". There are several other operators we can use: >, >=, <, <=, and != (not equal to). Another useful operator is `%in%`, which asks if the value on the lefthand side is found anywhere in the vector on the righthand side. For example, to get rows with specific `species_id` values, we could run:
+
+```{r filter-in}
+filter(surveys, species_id %in% c("RM", "DO"))
+```
+
+We can also use multiple conditions in one `filter()` statement. Here we will get rows with a year less than or equal to 1988 and whose hindfoot length values are not `NA`. The `!` before the `is.na()` function means "not".
+
+```{r filter-multiple}
+filter(surveys, year <= 1988 & !is.na(hindfoot_length))
+```
+
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 1: Filtering and selecting
+
+1. Use the surveys data to make a data.frame that has only data with years from 1980 to 1985.
+
+:::::::::::::::::::::::: solution
+
+```{r filter-challenge-answer}
+surveys_filtered <- filter(surveys, year >= 1980 & year <= 1985)
+```
+
+::::::::::::::::::::::::
+
+2. Use the surveys data to make a data.frame that has only the following columns, in order: `year`, `month`, `species_id`, `plot_id`.
+
+:::::::::::::::::::::::: solution
+
+```{r selectfilter-challenge-answer}
+surveys_selected <- select(surveys, year, month, species_id, plot_id)
+```
+
+::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+## The pipe: `%>%`
+
+What happens if we want to both `select()` and `filter()` our data? We have a couple options. First, we could use **nested** functions:
+
+```{r filter-select-nested}
+filter(select(surveys, -day), month >= 7)
+```
+
+R will evaluate statements from the inside out. First, `select()` will operate on the `surveys` data.frame, removing the column `day`. The resulting data.frame is then used as the first argument for `filter()`, which selects rows with a month greater than or equal to 7.
+
+Nested functions can be very difficult to read with only a few functions, and nearly impossible when many functions are done at once. An alternative approach is to create **intermediate** objects:
+
+```{r filter-select-intermediate}
+surveys_noday <- select(surveys, -day)
+filter(surveys_noday, month >= 7)
+```
+
+This approach is easier to read, since we can see the steps in order, but after enough steps, we are left with a cluttered mess of intermediate objects, often with confusing names.
+
+An elegant solution to this problem is an operator called the **pipe**, which looks like `%>%`. You can insert it by using the keyboard shortcut Shift+Cmd+M (Mac) or Shift+Ctrl+M (Windows). Here's how you could use a pipe to select and filter in one step:
+
+```{r filter-select-pipe}
+surveys %>%
+ select(-day) %>%
+ filter(month >= 7)
+```
+
+What it does is take the thing on the lefthand side and insert it as the first argument of the function on the righthand side. By putting each of our functions onto a new line, we can build a nice, readable **pipeline**. It can be useful to think of this as a little assembly line for our data. It starts at the top and gets piped into a `select()` function, and it comes out modified somewhat. It then gets sent into the `filter()` function, where it is further modified, and then the final product gets printed out to our console. It can also be helpful to think of `%>%` as meaning "and then". Since many `tidyverse` functions have verbs for names, a pipeline can be read like a sentence.
+
+
+:::::::::::::::::::::::::::::::::::::::::::: instructor
+It's worth showing the learners that you can run a **pipeline** without highlighting the whole thing. If your cursor is on any line of a pipeline, running that line will run the whole thing.
+
+You can also show that by highlighting a section of a pipeline, you can run only the first X steps of it.
+::::::::::::::::::::::::::::::::::::::::::::
+
+If we want to store this final product as an object, we use an assignment arrow at the start:
+
+```{r pipeline-assignment}
+surveys_sub <- surveys %>%
+ select(-day) %>%
+ filter(month >= 7)
+```
+
+A good approach is to build a pipeline step by step prior to assignment. You add functions to the pipeline as you go, with the results printing in the console for you to view. Once you're satisfied with your final result, go back and add the assignment arrow statement at the start. This approach is very interactive, allowing you to see the results of each step as you build the pipeline, and produces nicely readable code.
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 2: Using pipes
+
+Use the surveys data to make a data.frame that has the columns `record_id`, `month`, and `species_id`, with data from the year 1988. Use a pipe between the function calls.
+
+:::::::::::::::::::::::: solution
+
+```{r filter-select-challenge-answer}
+surveys_1988 <- surveys %>%
+ filter(year == 1988) %>%
+ select(record_id, month, species_id)
+```
+
+Make sure to `filter()` before you `select()`. You need to use the `year` column for filtering rows, but it is discarded in the `select()` step. You also need to make sure to use `==` instead of `=` when you are filtering rows where `year` is equal to 1988.
+
+::::::::::::::::::::::::
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+## Making new columns with `mutate()`
+
+Another common task is creating a new column based on values in existing columns. For example, we could add a new column that has the weight in kilograms instead of grams:
+
+```{r mutate}
+surveys %>%
+ mutate(weight_kg = weight / 1000)
+```
+
+You can create multiple columns in one `mutate()` call, and they will get created in the order you write them. This means you can even reference the first new column in the second new column:
+
+```{r mutate-multiple}
+surveys %>%
+ mutate(weight_kg = weight / 1000,
+ weight_lbs = weight_kg * 2.2)
+```
+
+
+
+We can also use multiple columns to create a single column. For example, it's often good practice to keep the components of a date in separate columns until necessary, as we've done here. This is because programs like Excel can do automatic things with dates in a way that is not reproducible and sometimes hard to notice. However, now that we are working in R, we can safely put together a date column.
+
+To put together the columns into something that looks like a date, we can use the `paste()` function, which takes arguments of the items to paste together, as well as the argument `sep`, which is the character used to separate the items.
+
+```{r paste-date}
+surveys %>%
+ mutate(date = paste(year, month, day, sep = "-"))
+```
+
+Since our new column gets moved all the way to the end, it doesn't end up printing out. We can use the `relocate()` function to put it after our `year` column:
+
+```{r relocate}
+surveys %>%
+ mutate(date = paste(year, month, day, sep = "-")) %>%
+ relocate(date, .after = year)
+```
+
+Now we can see that we have a character column that contains our date string. However, it's not truly a date column. Dates are a type of numeric variable with a defined, ordered scale. To turn this column into a proper date, we will use a function from the `tidyverse`'s `lubridate` package, which has lots of useful functions for working with dates. The function `ymd()` will parse a date string that has the order year-month-day. Let's load the package and use `ymd()`.
+
+```{r lubridate}
+library(lubridate)
+
+surveys %>%
+ mutate(date = paste(year, month, day, sep = "-"),
+ date = ymd(date)) %>%
+ relocate(date, .after = year)
+
+surveys %>%
+ mutate(date = paste(year, month, day, sep = "-"),
+ date = as.Date(date)) %>%
+ relocate(date, .after = year)
+```
+
+
+
+Now we can see that our `date` column has the type `date` as well. In this example, we created our column with two separate lines in `mutate()`, but we can combine them into one:
+
+```{r making-date}
+# using nested functions
+surveys %>%
+ mutate(date = ymd(paste(year, month, day, sep = "-"))) %>%
+ relocate(date, .after = year)
+
+# using a pipe *inside* mutate()
+surveys %>%
+ mutate(date = paste(year, month, day,
+ sep = "-") %>% ymd()) %>%
+ relocate(date, .after = year)
+```
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 3: Plotting date
+
+Because the `ggplot()` function takes the data as its first argument, you can actually pipe data straight into `ggplot()`. Try building a pipeline that creates the date column and plots weight across date.
+
+:::::::::::::::::::::::: solution
+
+```{r date-plot-challenge-answer}
+surveys %>%
+ mutate(date = ymd(paste(year, month, day, sep = "-"))) %>%
+ ggplot(aes(x = date, y = weight)) +
+ geom_jitter(alpha = 0.1)
+```
+
+This isn't necessarily the most useful plot, but we will learn some techniques that will help produce nice time series plots
+
+::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+## The split-apply-combine approach
+
+Many data analysis tasks can be achieved using the split-apply-combine approach: you split the data into groups, apply some analysis to each group, and combine the results in some way. `dplyr` has a few convenient functions to enable this approach, the main two being `group_by()` and `summarize()`.
+
+`group_by()` takes a data.frame and the name of one or more columns with categorical values that define the groups. `summarize()` then collapses each group into a one-row summary of the group, giving you back a data.frame with one row per group. The syntax for `summarize()` is similar to `mutate()`, where you define new columns based on values of other columns. Let's try calculating the mean weight of all our animals by sex.
+
+```{r group-by-summarize}
+surveys %>%
+ group_by(sex) %>%
+ summarize(mean_weight = mean(weight, na.rm = T))
+```
+
+You can see that the mean weight for males is slightly higher than for females, but that animals whose sex is unknown have much higher weights. This is probably due to small sample size, but we should check to be sure. Like `mutate()`, we can define multiple columns in one `summarize()` call. The function `n()` will count the number of rows in each group.
+
+```{r summarize-multiple}
+surveys %>%
+ group_by(sex) %>%
+ summarize(mean_weight = mean(weight, na.rm = T),
+ n = n())
+```
+
+You will often want to create groups based on multiple columns. For example, we might be interested in the mean weight of every species + sex combination. All we have to do is add another column to our `group_by()` call.
+
+```{r group-by-multiple}
+surveys %>%
+ group_by(species_id, sex) %>%
+ summarize(mean_weight = mean(weight, na.rm = T),
+ n = n())
+```
+
+Our resulting data.frame is much larger, since we have a greater number of groups. We also see a strange value showing up in our `mean_weight` column: `NaN`. This stands for "Not a Number", and it often results from trying to do an operation a vector with zero entries. How can a vector have zero entries? Well, if a particular group (like the AB species ID + `NA` sex group) has **only** `NA` values for weight, then the `na.rm = T` argument in `mean()` will remove **all** the values prior to calculating the mean. The result will be a value of `NaN`. Since we are not particularly interested in these values, let's add a step to our pipeline to remove rows where weight is `NA` **before** doing any other steps. This means that any groups with only `NA` values will disappear from our data.frame before we formally create the groups with `group_by()`.
+
+```{r filter-group-by}
+surveys %>%
+ filter(!is.na(weight)) %>%
+ group_by(species_id, sex) %>%
+ summarize(mean_weight = mean(weight),
+ n = n())
+```
+
+That looks better! It's often useful to take a look at the results in some order, like the lowest mean weight to highest. We can use the `arrange()` function for that:
+
+```{r arrange}
+surveys %>%
+ filter(!is.na(weight)) %>%
+ group_by(species_id, sex) %>%
+ summarize(mean_weight = mean(weight),
+ n = n()) %>%
+ arrange(mean_weight)
+```
+
+If we want to reverse the order, we can wrap the column name in `desc()`:
+
+```{r arrange-desc}
+surveys %>%
+ filter(!is.na(weight)) %>%
+ group_by(species_id, sex) %>%
+ summarize(mean_weight = mean(weight),
+ n = n()) %>%
+ arrange(desc(mean_weight))
+```
+
+You may have seen several messages saying `summarise() has grouped output by 'species_id'. You can override using the .groups argument.` These are warning you that your resulting data.frame has retained some group structure, which means any subsequent operations on that data.frame will happen at the group level. If you look at the resulting data.frame printed out in your console, you will see these lines:
+
+```
+# A tibble: 46 × 4
+# Groups: species_id [18]
+```
+
+They tell us we have a data.frame with 46 rows, 4 columns, and a group variable `species_id`, for which there are 18 groups. We will see something similar if we use `group_by()` alone:
+
+```{r group-by-alone}
+surveys %>%
+ group_by(species_id, sex)
+```
+
+What we get back is the entire `surveys` data.frame, but with the grouping variables added: 67 groups of `species_id` + `sex` combinations. Groups are often maintained throughout a pipeline, and if you assign the resulting data.frame to a new object, it will also have those groups. This can lead to confusing results if you forget about the grouping and want to carry out operations on the whole data.frame, not by group. Therefore, it is a good habit to remove the groups at the end of a pipeline containing `group_by()`:
+
+```{r ungroup}
+surveys %>%
+ filter(!is.na(weight)) %>%
+ group_by(species_id, sex) %>%
+ summarize(mean_weight = mean(weight),
+ n = n()) %>%
+ arrange(desc(mean_weight)) %>%
+ ungroup()
+```
+
+Now our data.frame just says `# A tibble: 46 × 4` at the top, with no groups.
+
+While it is common that you will want to get the one-row-per-group summary that `summarise()` provides, there are times where you want to calculate a per-group value but keep all the rows in your data.frame. For example, we might want to know the mean weight for each species ID + sex combination, and then we might want to know how far from that mean value each observation in the group is. For this, we can use `group_by()` and `mutate()` together:
+
+```{r group-by-mutate}
+surveys %>%
+ filter(!is.na(weight)) %>%
+ group_by(species_id, sex) %>%
+ mutate(mean_weight = mean(weight),
+ weight_diff = weight - mean_weight)
+```
+
+Since we get all our columns back, the new columns are at the very end and don't print out in the console. Let's use `select()` to just look at the columns of interest. Inside `select()` we can use the `contains()` function to get any column containing the word "weight" in the name:
+
+```{r select-contains}
+surveys %>%
+ filter(!is.na(weight)) %>%
+ group_by(species_id, sex) %>%
+ mutate(mean_weight = mean(weight),
+ weight_diff = weight - mean_weight) %>%
+ select(species_id, sex, contains("weight"))
+```
+
+What happens with the `group_by()` + `mutate()` combination is similar to using `summarize()`: for each group, the mean weight is calculated. However, instead of reporting only one row per group, the mean weight for each group is added to each row in that group. For each row in a group (like DM species ID + M sex), you will see the same value in `mean_weight`.
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 4: Making a time series
+
+1. Use the split-apply-combine approach to make a `data.frame` that counts the total number of animals of each sex caught on each day in the `surveys` data.
+
+:::::::::::::::::::::::: solution
+
+```{r sex-counts-challenge-answer}
+
+surveys_daily_counts <- surveys %>%
+ mutate(date = ymd(paste(year, month, day, sep = "-"))) %>%
+ group_by(date, sex) %>%
+ summarize(n = n())
+
+# shorter approach using count()
+surveys_daily_counts <- surveys %>%
+ mutate(date = ymd(paste(year, month, day, sep = "-"))) %>%
+ count(date, sex)
+```
+
+::::::::::::::::::::::::
+
+2. Now use the data.frame you just made to plot the daily number of animals of each sex caught over time. It's up to you what `geom` to use, but a `line` plot might be a good choice. You should also think about how to differentiate which data corresponds to which sex.
+
+:::::::::::::::::::::::: solution
+
+```{r time-series-challenge-answer}
+surveys_daily_counts %>%
+ ggplot(aes(x = date, y = n, color = sex)) +
+ geom_line()
+```
+
+::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Reshaping data with `tidyr`
+
+Let's say we are interested in comparing the mean weights of each species across our different plots. We can begin this process using the `group_by()` + `summarize()` approach:
+
+```{r mean-weight-by-plot}
+sp_by_plot <- surveys %>%
+ filter(!is.na(weight)) %>%
+ group_by(species_id, plot_id) %>%
+ summarise(mean_weight = mean(weight)) %>%
+ arrange(species_id, plot_id)
+
+sp_by_plot
+```
+
+That looks great, but it is a bit difficult to compare values across plots. It would be nice if we could reshape this data.frame to make those comparisons easier. Well, the `tidyr` package from the `tidyverse` has a pair of functions that allow you to reshape data by pivoting it: `pivot_wider()` and `pivot_longer()`. `pivot_wider()` will make the data wider, which means increasing the number of columns and reducing the number of rows. `pivot_longer()` will do the opposite, reducing the number of columns and increasing the number of rows.
+
+In this case, it might be nice to create a data.frame where each species has its own row, and each plot has its own column containing the mean weight for a given species. We will use `pivot_wider()` to reshape our data in this way. It takes 3 arguments:
+
+1. the name of the data.frame
+2. `names_from`: which column should be used to generate the names of the new columns?
+3. `values_from`: which column should be used to fill in the values of the new columns?
+
+Any columns not used for `names_from` or `values_from` will not be pivoted.
+
+{alt='Diagram depicting the behavior of `pivot_wider()` on a small tabular dataset.'}
+
+In our case, we want the new columns to be named from our `plot_id` column, with the values coming from the `mean_weight` column. We can pipe our data.frame right into `pivot_wider()` and add those two arguments:
+
+```{r pivot-wider}
+sp_by_plot_wide <- sp_by_plot %>%
+ pivot_wider(names_from = plot_id,
+ values_from = mean_weight)
+
+sp_by_plot_wide
+```
+
+Now we've got our reshaped data.frame. There are a few things to notice. First, we have a new column for each `plot_id` value. There is one old column left in the data.frame: `species_id`. It wasn't used in `pivot_wider()`, so it stays, and now contains a single entry for each unique `species_id` value.
+
+Finally, a lot of `NA`s have appeared. Some species aren't found in every plot, but because a data.frame has to have a value in every row and every column, an `NA` is inserted. We can double-check this to verify what is going on.
+
+Looking in our new pivoted data.frame, we can see that there is an `NA` value for the species `BA` in plot `1`. Let's take our `sp_by_plot` data.frame and look for the `mean_weight` of that species + plot combination.
+
+```{r pivot-wider-check}
+sp_by_plot %>%
+ filter(species_id == "BA" & plot_id == 1)
+```
+
+We get back 0 rows. There is no `mean_weight` for the species `BA` in plot `1`. This either happened because no `BA` were ever caught in plot `1`, or because every `BA` caught in plot `1` had an `NA` weight value and all the rows got removed when we used `filter(!is.na(weight))` in the process of making `sp_by_plot`. Because there are no rows with that species + plot combination, in our pivoted data.frame, the value gets filled with `NA`.
+
+There is another `pivot_` function that does the opposite, moving data from a wide to long format, called `pivot_longer()`. It takes 3 arguments: `cols` for the columns you want to pivot, `names_to` for the name of the new column which will contain the old column names, and `values_to` for the name of the new column which will contain the old values.
+
+{alt='Diagram depicting the behavior of `pivot_longer()` on a small tabular dataset.'}
+
+We can pivot our new wide data.frame to a long format using `pivot_longer()`. We want to pivot all the columns except `species_id`, and we will use `PLOT` for the new column of plot IDs, and `MEAN_WT` for the new column of mean weight values.
+
+```{r pivot-longer}
+sp_by_plot_wide %>%
+ pivot_longer(cols = -species_id, names_to = "PLOT", values_to = "MEAN_WT")
+```
+
+One thing you will notice is that all those `NA` values that got generated when we pivoted wider. However, we can filter those out, which gets us back to the same data as `sp_by_plot`, before we pivoted it wider.
+
+```{r pivot-longer-filter}
+sp_by_plot_wide %>%
+ pivot_longer(cols = -species_id, names_to = "PLOT", values_to = "MEAN_WT") %>%
+ filter(!is.na(MEAN_WT))
+```
+
+Data are often recorded in spreadsheets in a wider format, but lots of `tidyverse` tools, especially `ggplot2`, like data in a longer format, so `pivot_longer()` is often very useful.
+
+## Exporting data
+
+Let's say we want to send the wide version of our `sb_by_plot` data.frame to a colleague who doesn't use R. In this case, we might want to save it as a CSV file.
+
+First, we might want to modify the names of the columns, since right now they are bare numbers, which aren't very informative. Luckily, `pivot_wider()` has an argument `names_prefix` which will allow us to add "plot_" to the start of each column.
+
+```{r pivot-wider-prefix}
+sp_by_plot %>%
+ pivot_wider(names_from = plot_id, values_from = mean_weight,
+ names_prefix = "plot_")
+```
+
+That looks better! Let's save this data.frame as a new object.
+
+```{r assign-pivot-wider}
+surveys_sp <- sp_by_plot %>%
+ pivot_wider(names_from = plot_id, values_from = mean_weight,
+ names_prefix = "plot_")
+
+surveys_sp
+```
+
+Now we can save this data.frame to a CSV using the `write_csv()` function from the `readr` package. The first argument is the name of the data.frame, and the second is the path to the new file we want to create, including the file extension `.csv`.
+
+```{r write-csv}
+write_csv(surveys_sp, "data/cleaned/surveys_meanweight_species_plot.csv")
+```
+
+If we go look into our `data/cleaned_data` folder, we will see this new CSV file.
+
+::::::::::::::::::::::::::::::::::::: keypoints
+
+- use `filter()` to subset rows and `select()` to subset columns
+- build up pipelines one step at a time before assigning the result
+- it is often best to keep components of dates separate until needed, then use `mutate()` to make a date column
+- `group_by()` can be used with `summarize()` to collapse rows or `mutate()` to keep the same number of rows
+- `pivot_wider()` and `pivot_longer()` are powerful for reshaping data, but you should plan out how to use them thoughtfully
+
+::::::::::::::::::::::::::::::::::::::::::::::::
diff --git a/index.Rmd b/index.Rmd
deleted file mode 100644
index a9c789b30..000000000
--- a/index.Rmd
+++ /dev/null
@@ -1,47 +0,0 @@
----
-title: "Data Analysis and Visualization in R for Ecologists"
-author: François Michonneau & Auriel Fournier (Lesson Maintainers)
-site: sandpaper::sandpaper_site
----
-
-::::::::::::::::::::::::::: callout
-
-### This version of the lesson will soon be replaced
-
-[This lesson will be replaced with a redesigned version on 8th July 2024](https://carpentries.org/blog/2024/03/dc-r-ecology-adoption/).
-If you plan to teach this lesson before that date, we recommend that you consider using the redesigned version instead.
-Feedback on the new version of the lesson and your experience teaching it is very welcome as issues on [the source repository](https://github.com/datacarpentry/R-ecology-lesson-alternative/).
-
-::::::::::::::::::::::::::::::::::::::
-
-
-
-
-
-
-
-Data Carpentry's aim is to teach researchers basic concepts, skills,
-and tools for working with data so that they can get more done in less
-time, and with less pain. The lessons below were designed for those interested
-in working with ecology data in R.
-
-This is an introduction to R designed for participants with no programming
-experience. These lessons can be taught in a day (~ 6 hours). They start with
-some basic information about R syntax, the RStudio interface, and move through
-how to import CSV files, the structure of data frames, how to deal with factors,
-how to add/remove rows and columns, how to calculate summary statistics from a
-data frame, and a brief introduction to plotting. The last lesson demonstrates
-how to work with databases directly from R.
-
-This lesson assumes no prior knowledge of R or RStudio and no programming
-experience.
-
-## Contributors
-
-The list of contributors to this lesson is available [in the citation page](https://datacarpentry.org/R-ecology-lesson/CITATION).
-
-
diff --git a/index.md b/index.md
new file mode 100644
index 000000000..817540014
--- /dev/null
+++ b/index.md
@@ -0,0 +1,20 @@
+---
+site: sandpaper::sandpaper_site
+---
+
+:::::::::::::::::::::::::: callout
+
+## This lesson changed a lot recently!
+
+You may notice some big changes in the content of this lesson since you last visited.
+
+[The Data Carpentry Ecology Curriculum Advisory Committee recently approved this redesigned version of the lesson for adoption into the curriculum](https://carpentries.org/blog/2024/03/dc-r-ecology-adoption/)! It replaced [the previous version](https://doi.org/10.5281/zenodo.12684301) in the curriculum in July 2024.
+
+::::::::::::::::::::::::::::::::::
+
+Data Carpentry’s aim is to teach researchers basic concepts, skills, and tools for working with data so that they can get more done in less time, and with less pain. The lessons below were designed for those interested in working with ecology data in R.
+
+This is an introduction to R designed for participants with no programming experience. This lesson can be taught in a day (~ 6 hours). It starts with information about the R programming language and the RStudio interface. It then moves to loading in data and exploring how to visualise it with ggplot2. The next episode takes learners through an exploration of data frames and some common data cleaning operations, before discussing vectors and factors. The final episode introduces the flow of data in R, and how to combine operations to select, filter, and mutate a data frame.
+
+This lesson assumes no prior knowledge of R or RStudio and no programming experience.
+
diff --git a/instructors/instructor-notes.md b/instructors/instructor-notes.md
index f58e1e0f7..4a9ce1452 100644
--- a/instructors/instructor-notes.md
+++ b/instructors/instructor-notes.md
@@ -4,6 +4,18 @@ title: Instructor notes
root: .
---
+## Workshop Timing
+
+In a two half-day format, you could probably complete the "Intro to R and RStudio" and "ggplot2" lessons, and make it about halfway through the "Exploring and understanding data" episode in the first half-day.
+
+We do not recommend splitting the workshop into episodes 1 and 2 on the first half-day and 3 and 4 on the second half-day, as we have found this leaves excess time on the first day and requires rushing or skipping content on the second day.
+
+**Instructor Notes below this line were written for
+[the original Data Carpentry lesson](https://datacarpentry.org/R-ecology-lesson),
+and may not be relevant to this redesigned version.**
+
+---
+
## Dataset
The data used for this lesson are in the figshare repository at:
@@ -94,8 +106,8 @@ font, R will try to load a different font. This can be done with
Save yourself some aggrevation, and have everyone check and see if they can
install all these packages before you start the first day.
-See the "Preparations" section on the homepage of the course website for
-package installation instructions.
+See the "Install required R packages" section on the homepage of the course
+website for package installation instructions.
Sometimes learners are unable to install the **`tidyverse`** package.
In that case, they can try to install the individual packages that are actually
@@ -243,5 +255,3 @@ maintainers by email or
For a more in-depth coverage of topics of the workshops, you may want to read
"[R for Data Science](https://r4ds.had.co.nz/)" by Hadley Wickham and Garrett
Grolemund.
-
-
diff --git a/learners/extra-challenges.Rmd b/learners/extra-challenges.Rmd
new file mode 100644
index 000000000..c24709f50
--- /dev/null
+++ b/learners/extra-challenges.Rmd
@@ -0,0 +1,83 @@
+---
+title: "Extra Challenges"
+teaching: 45
+exercises: 3
+---
+
+
+```{r setup, include=FALSE}
+knitr::opts_chunk$set(dpi = 200, out.height = 600, out.width = 600, R.options = list(max.print = 100))
+```
+
+```{r load-tidyverse-package, message=FALSE}
+library(tidyverse)
+surveys <- read_csv("data/cleaned/surveys_complete_77_89.csv")
+```
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge: `ggplot2` syntax
+
+There are some issues with these `ggplot2` examples. Can you figure out what is wrong with each one?
+
+```{r, eval=FALSE}
+ggplot(data = surveys,
+ mapping = aes(x = weight, y = hindfoot_length, color = "blue")) +
+ geom_point()
+```
+
+
+:::::::::::::::::::::::: solution
+
+Our points don't actually turn out blue, because we defined the color inside of `aes()`. `aes()` is used for translating variables from the data into plot elements, like color. There is no variable in the data called "blue".
+
+::::::::::::::::::::::::
+
+```{r, eval=FALSE}
+ggplot(data = surveys,
+ mapping = aes(x = "weight", y = "hindfoot_length")) +
+ geom_point()
+```
+
+:::::::::::::::::::::::: solution
+
+Variable names inside `aes()` should not be wrapped in quotes.
+
+::::::::::::::::::::::::
+
+```{r, eval=FALSE}
+ggplot(data = surveys,
+ mapping = aes(x = weight, y = hindfoot_length))
+ + geom_point()
+```
+
+:::::::::::::::::::::::: solution
+
+When adding things like `geom_` or `scale_` functions to a `ggplot()`, you have to end a line with `+`, not begin a line with it.
+
+::::::::::::::::::::::::
+
+```{r, eval=FALSE}
+ggplot(data = surveys, x = weight, y = hindfoot_length) +
+ geom_point()
+```
+
+:::::::::::::::::::::::: solution
+
+When translating variables from the data, like `weight` and `hindfoot_length`, to elements of the plot, like `x` and `y`, you must put them inside `aes()`.
+
+::::::::::::::::::::::::
+
+```{r, eval=FALSE}
+ggplot(data = surveys,
+ mapping = aes(x = weight, y = hindfoot_length, color = species_id)) +
+ geom_point() +
+ scale_color_continuous(type = "viridis")
+```
+
+:::::::::::::::::::::::: solution
+
+`species_id` is a categorical variable, but `scale_color_continuous()` supplies a continuous color scale. `scale_color_discrete()` would give a discrete/categorical scale.
+
+::::::::::::::::::::::::
+::::::::::::::::::::::::::::::::::::::::::::::::
diff --git a/learners/setup.md b/learners/setup.md
index 703c609f2..b4db35bd8 100644
--- a/learners/setup.md
+++ b/learners/setup.md
@@ -10,165 +10,121 @@ to be able to install a number of R packages, create directories, and download
files.
To avoid troubleshooting during the lesson, learners should follow the
-instruction below to download and install everything beforehand.
-If they are using their own computers this should be no problem,
-but if the computer is managed by their organization's IT department
+instructions below to download and install everything beforehand.
+If the computer is managed by their organization's IT department
they might need help from an IT administrator.
-## Install R and RStudio
+### Install R and RStudio
-R and RStudio are two separate pieces of software:
+R and RStudio are two separate pieces of software:
-- **R** is a programming language that is especially powerful for data
- exploration, visualization, and statistical analysis
-- **RStudio** is an integrated development environment (IDE) that makes using
- R easier. In this course we use RStudio to interact with R.
+* **R** is a programming language and software used to run code written in R.
+* **RStudio** is an integrated development environment (IDE) that makes using R easier. In this course we use RStudio to interact with R.
+
+If you don't already have R and RStudio installed, follow the instructions for your operating system below.
+You have to install R before you install RStudio.
-::::::::::::: discussion
+::::::: spoiler
-### Installation Instructions
-
-If you don't already have R and RStudio installed, follow the instructions for
-your operating system below. You have to install R before you install RStudio.
+## For Windows
+* Download R from the [CRAN website](https://cran.r-project.org/bin/windows/base/release.htm).
+* Run the `.exe` file that was just downloaded
+* Go to the [RStudio download page](https://www.rstudio.com/products/rstudio/download/#download)
+* Under *Installers* select **RStudio x.yy.zzz - Windows Vista/7/8/10** (where x, y, and z represent version numbers)
+* Double click the file to install it
+* Once it's installed, open RStudio to make sure it works and you don't get any error messages.
+
:::::::::::::::::::::::::
-::::::: solution
+:::::::::::::::: spoiler
-### Windows
+## For MacOS
-- Download R from the
- [CRAN website](https://cran.r-project.org/bin/windows/base/release.htm).
-- Run the `.exe` file that was just downloaded
-- Go to the [RStudio download page](https://www.rstudio.com/products/rstudio/download/#download)
-- Under *All Installers*, download the RStudio Installer for Windows.
-- Double click the file to install it
-- Once it's installed, open RStudio to make sure it works and you don't get any
- error messages.
+* Download R from the [CRAN website](https://cran.r-project.org/bin/macosx/).
+* Select the `.pkg` file for the latest R version
+* Double click on the downloaded file to install R
+* It is also a good idea to install [XQuartz](https://www.xquartz.org/) (needed by some packages)
+* Go to the [RStudio download page](https://www.rstudio.com/products/rstudio/download/#download)
+* Under *Installers* select **RStudio x.yy.zzz - Mac OS X 10.6+ (64-bit)** (where x, y, and z represent version numbers)
+* Double click the file to install RStudio
+* Once it's installed, open RStudio to make sure it works and you don't get any error messages.
::::::::::::::::
-::::::: solution
+::::::: spoiler
-### MacOS
+## For Linux
-- Download R from
- the [CRAN website](https://cran.r-project.org/bin/macosx/).
-- Select the `.pkg` file for the latest R version
-- Double click on the downloaded file to install R
-- It is also a good idea to install [XQuartz](https://www.xquartz.org/) (needed
- by some packages)
-- Go to the [RStudio download page](https://www.rstudio.com/products/rstudio/download/#download)
-- Under *All Installers*, download the RStudio Installer for MacOS.
-- Double click the file to install RStudio
-- Once it's installed, open RStudio to make sure it works and you don't get any
- error messages.
+* Download R from the [CRAN website](https://cran.r-project.org/bin/macosx/).
+* Select the `.pkg` file for the latest R version
+* Double click on the downloaded file to install R
+* It is also a good idea to install [XQuartz](https://www.xquartz.org/) (needed by some packages)
+* Go to the [RStudio download page](https://www.rstudio.com/products/rstudio/download/#download)
+* Under *Installers* select **RStudio x.yy.zzz - Mac OS X 10.6+ (64-bit)** (where x, y, and z represent version numbers)
+* Double click the file to install RStudio
+* Once it's installed, open RStudio to make sure it works and you don't get any error messages.
::::::::::::::::
-::::::: solution
+### Update R and RStudio
+If you already have R and RStudio installed, first check if your R version is up to date:
-### Linux
+* When you open RStudio your R version will be printed in the console on the bottom left. Alternatively, you can type `sessionInfo()` into the console. If your R version is 4.0.0 or later, you don't need to update R for this lesson. If your version of R is older than that, download and install the latest version of R from the R project website [for Windows](https://cran.r-project.org/bin/windows/base/), [for MacOS](https://cran.r-project.org/bin/macosx/), or [for Linux](https://cran.r-project.org/bin/linux/)
+* It is not necessary to remove old versions of R from your system, but if you wish to do so you can check [How do I uninstall R?](https://cran.r-project.org/bin/windows/base/rw-FAQ.html#How-do-I-UNinstall-R_003f)
+* After installing a new version of R, you will have to reinstall all your packages with the new version. For Windows, there is a package called `installr` that can help you with upgrading your R version and migrate your package library. A similar package called `pacman` can help with updating R packages across
+To update RStudio to the latest version, open RStudio and click on
+`Help > Check for Updates`. If a new version is available follow the
+instruction on screen. By default, RStudio will also automatically notify you
+of new versions every once in a while.
-- Follow the instructions for your distribution
- from [CRAN](https://cloud.r-project.org/bin/linux), they provide information
- to get the most recent version of R for common distributions. For most
- distributions, you could use your package manager (e.g., for Debian/Ubuntu run
- `sudo apt-get install r-base`, and for Fedora `sudo yum install R`), but we
- don't recommend this approach as the versions provided by this are
- usually out of date. In any case, make sure you have at least R 3.3.1.
-- Go to the
- [RStudio download page](https://www.rstudio.com/products/rstudio/download/#download)
-- Under *All Installers*, select the version that matches your distribution and
- install it with your preferred method (e.g., with Debian/Ubuntu `sudo dpkg -i rstudio-YYYY.MM.X-ZZZ-amd64.deb` at the terminal).
-- Once it's installed, open RStudio to make sure it works and you don't get any
- error messages.
+::::::::::::::::::::::::::::: callout
-::::::::::::::::
+The changes introduced by new R versions are usually backwards-compatible. That is, your old code should still work after updating your R version. However, if breaking changes happen, it is useful to know that you can have multiple versions of R installed in parallel and that you can switch between them in RStudio by going to `Tools > Global Options > General > Basic`.
-## Update R and RStudio
-
-If you already have R and RStudio installed, first check if your R version is
-up to date:
-
-- When you open RStudio your R version will be printed in the console on
- the bottom left. Alternatively, you can type `sessionInfo()` into the console.
- If your R version is 4.0.0 or later, you don't need to update R for this
- lesson. If your version of R is older than that, download and install the
- latest version of R from the R project website
- [for Windows](https://cran.r-project.org/bin/windows/base/),
- [for MacOS](https://cran.r-project.org/bin/macosx/),
- or [for Linux](https://cran.r-project.org/bin/linux/)
-- It is not necessary to remove old versions of R from your system,
- but if you wish to do so you can check
- [How do I uninstall R?](https://cran.r-project.org/bin/windows/base/rw-FAQ.html#How-do-I-UNinstall-R_003f)
-- Note: The changes introduced by new R versions are usually backwards-compatible.
- That is, your old code should still work after updating your R version.
- However, if breaking changes happen, it is useful to know that you can have
- multiple versions of R installed in parallel and that you can switch between
- them in RStudio by going to `Tools > Global Options > General > Basic`.
-- After installing a new version of R, you will have to reinstall all your packages
- with the new version. For Windows, there is a package called `installr` that can
- help you with upgrading your R version and migrate your package library.
-
-To update RStudio to the latest version, open RStudio and click on
-`Help > Check for Updates`. If a new version is available follow the
-instruction on screen. By default, RStudio will also automatically notify you
-of new versions every once in a while.
+While this may sound scary, it is **far more common** to run into issues due to using out-of-date versions of R or R packages. Keeping up with the latest versions of R, RStudio, and any packages you regularly use is a good practice.
+
+:::::::::::::::::::::::::::::
-## Install required R packages
+### Install required R packages
-During the course we will need a number of R packages. Packages contain useful
-R code written by other people. We will use the packages
-`tidyverse`, `hexbin`, `patchwork`, and `RSQLite`.
+During the course we will need a number of R packages. Packages contain useful R code written by other people. We will use the packages `tidyverse`, `lubridate`, and `ratdat`.
-To try to install these packages, open RStudio and copy and paste the following
-command into the console window (look for a blinking cursor on the bottom left),
-then press the Enter (Windows and Linux) or Return (MacOS)
-to execute the command.
+To try to install these packages, open RStudio and copy and paste the following command into the console window (look for a blinking cursor on the bottom left), then press the Enter (Windows and Linux) or Return (MacOS) to execute the command.
```r
-install.packages(c("tidyverse", "hexbin", "patchwork", "RSQLite"))
+install.packages(c("tidyverse", "lubridate", "ratdat"))
```
-Alternatively, you can install the packages using RStudio's graphical user
-interface by going to `Tools > Install Packages` and typing the names of the
-packages separated by a comma.
+Alternatively, you can install the packages using RStudio's graphical user interface by going to `Tools > Install Packages` and typing the names of the packages separated by a comma.
-R tries to download and install the packages on your machine.
-When the installation has finished, you can try to load the
-packages by pasting the following code into the console:
+R tries to download and install the packages on your machine.
+
+When the installation has finished, you can try to load the packages by pasting the following code into the console:
```r
library(tidyverse)
-library(hexbin)
-library(patchwork)
-library(RSQLite)
+library(lubridate)
+library(ratdat)
```
-If you do not see an error like `there is no package called ‘...'` you are good
-to go!
+If you do not see an error like `there is no package called ‘...’` you are good to go!
+
+### Updating R packages
+
+Generally, it is recommended to keep your R version and all packages up to date, because new versions bring improvements and important bugfixes. To update the packages that you have installed, click `Update` in the `Packages` tab in the bottom right panel of RStudio, or go to `Tools > Check for Package Updates...`
+
+You should update **all of the packages** required for the lesson, even if you installed them relatively recently.
-## Updating R packages
+Sometimes, package updates introduce changes that break your old code, which can be very frustrating. To avoid this problem, you can use a package called `renv`. It locks the package versions you have used for a given project and makes it straightforward to reinstall those exact package version in a new environment, for example after updating your R version or on another computer. However, the details are outside of the scope of this lesson.
-Generally, it is recommended to keep your R version and all packages
-up to date, because new versions bring improvements and important bugfixes.
-To update the packages that you have installed, click `Update` in the
-`Packages` tab in the bottom right panel of RStudio, or go to
-`Tools > Check for Package Updates...`.
+### Download the data
-Sometimes, package updates introduce changes that break your old code,
-which can be very frustrating. To avoid this problem, you can use a package
-called `renv`. It locks the package versions you have used for a given project
-and makes it straightforward to reinstall those exact package version in a
-new environment, for example after updating your R version or on another
-computer. However, the details are outside of the scope of this lesson.
+We will download the data directly from R during the lessons. However, if you are expecting problems with the network, it may be better to download the data beforehand and store it on your machine.
-## Download the data
+The data files for the lesson can be downloaded manually:
-We will download the data directly from R during the lessons. However, if you
-are expecting problems with the network, it may be better to download the data
-beforehand and store it on your machine.
+ - [cleaned data](../episodes/data/cleaned/surveys_complete_77_89.csv) and
+ - [zip file of raw data](../episodes/data/new_data.zip).
-The data files for the lesson can be downloaded manually here: [https://doi.org/10.6084/m9.figshare.1314459](https://doi.org/10.6084/m9.figshare.1314459)
diff --git a/renv/profiles/lesson-requirements/renv.lock b/renv/profiles/lesson-requirements/renv.lock
index 0c3407f49..a383702c6 100644
--- a/renv/profiles/lesson-requirements/renv.lock
+++ b/renv/profiles/lesson-requirements/renv.lock
@@ -2,17 +2,9 @@
"R": {
"Version": "4.4.0",
"Repositories": [
- {
- "Name": "carpentries",
- "URL": "https://carpentries.r-universe.dev"
- },
- {
- "Name": "carpentries_archive",
- "URL": "https://carpentries.github.io/drat"
- },
{
"Name": "CRAN",
- "URL": "https://cran.rstudio.com"
+ "URL": "https://cloud.r-project.org"
}
]
},
@@ -30,7 +22,7 @@
},
"MASS": {
"Package": "MASS",
- "Version": "7.3-60.2",
+ "Version": "7.3-61",
"Source": "Repository",
"Repository": "CRAN",
"Requirements": [
@@ -41,13 +33,13 @@
"stats",
"utils"
],
- "Hash": "2f342c46163b0b54d7b64d1f798e2c78"
+ "Hash": "0cafd6f0500e5deba33be22c46bf6055"
},
"Matrix": {
"Package": "Matrix",
"Version": "1.7-0",
"Source": "Repository",
- "Repository": "CRAN",
+ "Repository": "RSPM",
"Requirements": [
"R",
"grDevices",
@@ -64,7 +56,7 @@
"Package": "R6",
"Version": "2.5.1",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"R"
],
@@ -80,25 +72,6 @@
],
"Hash": "45f0398006e83a5b10b72a90663d8d8c"
},
- "RSQLite": {
- "Package": "RSQLite",
- "Version": "2.3.7",
- "Source": "Repository",
- "Repository": "CRAN",
- "Requirements": [
- "DBI",
- "R",
- "bit64",
- "blob",
- "cpp11",
- "memoise",
- "methods",
- "pkgconfig",
- "plogr",
- "rlang"
- ],
- "Hash": "46b45a4dd7bb0e0f4e3fc22245817240"
- },
"Rcpp": {
"Package": "Rcpp",
"Version": "1.0.12",
@@ -114,7 +87,7 @@
"Package": "askpass",
"Version": "1.2.0",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"sys"
],
@@ -134,7 +107,7 @@
"Package": "base64enc",
"Version": "0.1-3",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"R"
],
@@ -256,20 +229,20 @@
},
"cli": {
"Package": "cli",
- "Version": "3.6.2",
+ "Version": "3.6.3",
"Source": "Repository",
"Repository": "CRAN",
"Requirements": [
"R",
"utils"
],
- "Hash": "1216ac65ac55ec0058a6f75d7ca0fd52"
+ "Hash": "b21916dd77a27642b447374a5d30ecf3"
},
"clipr": {
"Package": "clipr",
"Version": "0.8.0",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"utils"
],
@@ -314,21 +287,21 @@
},
"crayon": {
"Package": "crayon",
- "Version": "1.5.2",
+ "Version": "1.5.3",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"grDevices",
"methods",
"utils"
],
- "Hash": "e8a1e41acf02548751f45c718d55aa6a"
+ "Hash": "859d96e65ef198fd43e82b9628d593ef"
},
"curl": {
"Package": "curl",
"Version": "5.2.1",
"Source": "Repository",
- "Repository": "carpentries",
+ "Repository": "https://carpentries.r-universe.dev",
"Requirements": [
"R"
],
@@ -375,14 +348,14 @@
},
"digest": {
"Package": "digest",
- "Version": "0.6.35",
+ "Version": "0.6.36",
"Source": "Repository",
"Repository": "CRAN",
"Requirements": [
"R",
"utils"
],
- "Hash": "698ece7ba5a4fa4559e3d537e7ec3d31"
+ "Hash": "fd6824ad91ede64151e93af67df6376b"
},
"dplyr": {
"Package": "dplyr",
@@ -428,14 +401,14 @@
},
"evaluate": {
"Package": "evaluate",
- "Version": "0.23",
+ "Version": "0.24.0",
"Source": "Repository",
"Repository": "CRAN",
"Requirements": [
"R",
"methods"
],
- "Hash": "daf4a1246be12c1fa8c7705a0935c1a0"
+ "Hash": "a1066cbc05caee9a4bf6d90f194ff4da"
},
"fansi": {
"Package": "fansi",
@@ -528,7 +501,7 @@
"Package": "generics",
"Version": "0.1.3",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"R",
"methods"
@@ -575,7 +548,7 @@
"Package": "googledrive",
"Version": "2.1.1",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"R",
"cli",
@@ -600,7 +573,7 @@
"Package": "googlesheets4",
"Version": "1.1.1",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"R",
"cellranger",
@@ -754,7 +727,7 @@
"Package": "jquerylib",
"Version": "0.1.4",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"htmltools"
],
@@ -790,7 +763,7 @@
"Package": "labeling",
"Version": "0.4.3",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"graphics",
"stats"
@@ -801,7 +774,7 @@
"Package": "lattice",
"Version": "0.22-6",
"Source": "Repository",
- "Repository": "CRAN",
+ "Repository": "RSPM",
"Requirements": [
"R",
"grDevices",
@@ -842,7 +815,7 @@
"Package": "magrittr",
"Version": "2.0.3",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"R"
],
@@ -852,7 +825,7 @@
"Package": "memoise",
"Version": "2.0.1",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"cachem",
"rlang"
@@ -880,7 +853,7 @@
"Package": "mime",
"Version": "0.12",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"tools"
],
@@ -917,9 +890,9 @@
},
"nlme": {
"Package": "nlme",
- "Version": "3.1-164",
+ "Version": "3.1-165",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"R",
"graphics",
@@ -927,41 +900,23 @@
"stats",
"utils"
],
- "Hash": "a623a2239e642806158bc4dc3f51565d"
+ "Hash": "2769a88be217841b1f33ed469675c3cc"
},
"openssl": {
"Package": "openssl",
"Version": "2.2.0",
"Source": "Repository",
- "Repository": "carpentries",
+ "Repository": "https://carpentries.r-universe.dev",
"Requirements": [
"askpass"
],
"Hash": "2bcca3848e4734eb3b16103bc9aa4b8e"
},
- "patchwork": {
- "Package": "patchwork",
- "Version": "1.2.0",
- "Source": "Repository",
- "Repository": "CRAN",
- "Requirements": [
- "cli",
- "ggplot2",
- "grDevices",
- "graphics",
- "grid",
- "gtable",
- "rlang",
- "stats",
- "utils"
- ],
- "Hash": "9c8ab14c00ac07e9e04d1664c0b74486"
- },
"pillar": {
"Package": "pillar",
"Version": "1.9.0",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"cli",
"fansi",
@@ -978,19 +933,12 @@
"Package": "pkgconfig",
"Version": "2.0.3",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"utils"
],
"Hash": "01f28d4278f15c76cddbea05899c5d6f"
},
- "plogr": {
- "Package": "plogr",
- "Version": "0.2.0",
- "Source": "Repository",
- "Repository": "CRAN",
- "Hash": "09eb987710984fc2905c7129c7d85e65"
- },
"prettyunits": {
"Package": "prettyunits",
"Version": "1.2.0",
@@ -1058,7 +1006,7 @@
"Package": "ragg",
"Version": "1.3.2",
"Source": "Repository",
- "Repository": "carpentries",
+ "Repository": "https://carpentries.r-universe.dev",
"Requirements": [
"systemfonts",
"textshaping"
@@ -1069,12 +1017,22 @@
"Package": "rappdirs",
"Version": "0.3.3",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"R"
],
"Hash": "5e3c5dc0b071b21fa128676560dbe94d"
},
+ "ratdat": {
+ "Package": "ratdat",
+ "Version": "1.1.0",
+ "Source": "Repository",
+ "Repository": "CRAN",
+ "Requirements": [
+ "R"
+ ],
+ "Hash": "5764aa379c01dc65f275059dd11e4556"
+ },
"readr": {
"Package": "readr",
"Version": "2.1.5",
@@ -1117,7 +1075,7 @@
"Package": "rematch",
"Version": "2.0.0",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Hash": "cbff1b666c6fa6d21202f07e2318d4f1"
},
"rematch2": {
@@ -1164,14 +1122,14 @@
},
"rlang": {
"Package": "rlang",
- "Version": "1.1.3",
+ "Version": "1.1.4",
"Source": "Repository",
"Repository": "CRAN",
"Requirements": [
"R",
"utils"
],
- "Hash": "42548638fae05fd9a9b5f3f437fbbbe2"
+ "Hash": "3eec01f8b1dee337674b2e34ab1f9bc1"
},
"rmarkdown": {
"Package": "rmarkdown",
@@ -1273,7 +1231,7 @@
"Package": "stringi",
"Version": "1.8.4",
"Source": "Repository",
- "Repository": "carpentries",
+ "Repository": "https://carpentries.r-universe.dev",
"Requirements": [
"R",
"stats",
@@ -1310,7 +1268,7 @@
"Package": "systemfonts",
"Version": "1.1.0",
"Source": "Repository",
- "Repository": "carpentries",
+ "Repository": "https://carpentries.r-universe.dev",
"Requirements": [
"R",
"cpp11",
@@ -1322,7 +1280,7 @@
"Package": "textshaping",
"Version": "0.4.0",
"Source": "Repository",
- "Repository": "carpentries",
+ "Repository": "https://carpentries.r-universe.dev",
"Requirements": [
"R",
"cpp11",
@@ -1335,7 +1293,7 @@
"Package": "tibble",
"Version": "3.2.1",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"R",
"fansi",
@@ -1454,7 +1412,7 @@
"Package": "tzdb",
"Version": "0.4.0",
"Source": "Repository",
- "Repository": "RSPM",
+ "Repository": "CRAN",
"Requirements": [
"R",
"cpp11"
@@ -1545,7 +1503,7 @@
},
"xfun": {
"Package": "xfun",
- "Version": "0.44",
+ "Version": "0.45",
"Source": "Repository",
"Repository": "CRAN",
"Requirements": [
@@ -1553,7 +1511,7 @@
"stats",
"tools"
],
- "Hash": "317a0538d32f4a009658bcedb7923f4b"
+ "Hash": "ca59c87fe305b16a9141a5874c3a7889"
},
"xml2": {
"Package": "xml2",