Week9-Lecture2.qmd

---
date: "`r (lubridate::ymd('20241002')+lubridate::dweeks(8))`"
title: "Class 18 Natural Experiment I: Regression Discontinuity Design"
execute: 
  cache: false
---


# Natural Experiment

## Class Objectives

  -   Concept of regression discontinuity design

  -   Estimation of causal effects using regression discontinuity design

  -   Application of regression discontinuity design in the business field


## From RCTs to Secondary Data

-   RCTs are the gold standard of causal inference: In an RCT, the treatment is randomized and hence uncorrelated with any confounding factors, i.e., $cov(X,\epsilon)=0$

-   In practice, however, it can be challenging to implement a perfect RCT.

    1.  Crossover and spillover effects;

    2.  Costly in terms of time and money

-   Therefore, we may want to exploit causal effects from existing secondary data. Besides the **instrumental variable** method, we can also investigate **natural experiments**.

## Comparison: RCT & Natural Experiment

::: callout-note
## Natural Experiment

A **natural experiment** is an event in which individuals are exposed to the experimental conditions that are determined by **nature** or **exogenous factors beyond researchers' control**. The process governing the exposures arguably **resembles randomized experiments**.
:::

\vspace{1cm}

::: columns
::: {.column width="50%"}
**RCT**

1.  Assignment of treatment is randomized by us

2.  Treatment is under control by us

3.  Primary data
:::

::: {.column width="50%"}
**Natural Experiment**

1.  Assignment of treatment is randomized by nature

2.  Treatment is not controlled by us

3.  Secondary data
:::
:::

# Regression Discontinuity Design

## What is an RDD

-   A **regression discontinuity design (RDD)** is a natural-experimental design that aims to determine the causal effects of interventions by identifying a **cutoff** around which an intervention is as if randomized across individuals.

```{r}
#| echo: false
#| fig-align: 'center'
#| fig-cap: 'Visual illustration of RDD'
knitr::include_graphics("images/Week 9/RDD.png")
```

## Motivating Example

**Business objective**: What is the causal effect of receiving a Master's degree with Distinction versus Non-Distinction on students' future salary?

::: {.content-visible when-format="beamer"}
-   Can we run the following simple linear regression and obtain the causal effects?

$$
Salary_i = \alpha + \beta Distinction_i + \epsilon_i
$$

-   Can we use RCT?

-   Can we use instrumental variables?
:::

::: {.content-hidden when-format="beamer"}
-   Can we run the following simple linear regression and obtain the causal effects?
    -   No, because the regression suffers from endogeneity issue, such as omitted variable bias. Individual's ability is correlated with their average grades and also affects individual salary, which is omitted from the regression.

$$
Salary_i = \alpha + \beta Distinction_i + \epsilon_i
$$

-   Can we use RCT?
    -   No, RCTs would be unethical to run for this research question.
-   Can we use instrumental variables?
    -   Theoretically yes, but in practice, it is difficult to find valid instrumental variables.
:::

## A Natural Experiment in the UK

-   In the UK Education system, students receiving 70% or above final average grades will receive Distinction while students below 70% will receive Merit.

-   The above setting gives us a nice natural experiment:

    -   Students may improve their average grades significantly, such as moving from 60% to 69% by working harder, but they cannot perfectly control their average grades around the cutoff, say, from 69.9% to 70%.

## Visual Illustration of RDD: An Example of Distinction on Salary

```{r}
#| echo: false
#| fig-align: 'center'
knitr::include_graphics("images/Week 9/RDDExample.png")
```

## Why RDD Gives Causal Effects?

-   For students just above 70%, to measure the treatment effects of receiving Distinction, we would need their *counterfactual salaries* *if they had not received Distinction.*

-   At the same time, because the "running variable" **cannot be perfectly controlled** by the individuals **around the cutoff point**, it's as if the treatment was randomized near the cutoff. Thus, individuals near the cutoff should be very similar, such that there should be no systematic differences across the treatment and control group.

    -   Similar to RCT, we overcome the fundamental problem of causal inference using students just below 70 as the control group.

-   All else being equal, a sudden change in the outcome variable at the cutoff can only be attributed to the treatment effect.

## Conditions for Using an RDD

-   An RDD design arises when treatment is assigned based on whether an underlying **continuous variable** crosses a cutoff.
    -   The continuous variable is often referred to as the **running variable**.
-   **AND** the characteristic cannot be perfectly manipulated by individuals
    -   We should only focus on individuals close to the cutoff point.

**Exercise**: eBay endorses sellers with 10,000 orders as Gold Seller. Can we use RDD to identify the causal effect of receiving Gold Seller endorsement on seller sales?

::: {.content-hidden when-format="beamer"}
No, because sellers can have perfect control over their sales around the cutoff point.
:::

# Implementation of RDD

## Step 1: Select Sample of Analysis

1.  Determine the bandwidth above and below the cutoff and select the subset of individuals within the bandwidth
    -   e.g., if we choose a bandwidth of 0.5, we need to filter out students with average scores between 69.5 and 70.5

-   We face a trade-off when selecting the bandwidth**:** If we choose a smaller bandwidth around the cut-off
    -   Pros: Individuals should be more similar around the cutoff, thus it is more likely the control group and treatment group are "as-if randomized", thus higher internal validity.
    -   Cons: We have a smaller subset of subjects which may not be representative of remaining individuals, thus lower external validity; We have a smaller sample size due to fewer individuals selected
-   In practice, there is no specific rule how to determine the bandwidth. We need to run a set of different bandwidths as **robustness checks**.

## Step 2: Examine Continuity of Observed Characteristics

2.  Examine if other characteristics of the treatment group and control group are continuous at the cut-off point.
    -   The idea is similar to "randomization check" in an RCT.

## Step 3: Data Analysis

3.  Regress the outcome variable on the treatment indicator to obtain the causal effect.

$$
Y_i = \beta_0 + \beta_1 Treated + \beta_2 running\_variable + \epsilon_i
$$

-   $Treated$ is a binary variable for whether or not the running variable is above the cutoff.

-   We may also want to control the running variable in the regression to mitigate its confounding effects.

## The Causal Effect of Distinction on Salary

-  We collect a dataset of 1000 graduates with their MSc final grade and salary.

```{r}
pacman::p_load(dplyr, fixest, modelsummary, ggplot2)
data_rdd <- read.csv("https://www.dropbox.com/scl/fi/z4rgm15cmp19m3i65il3a/data_rdd.csv?rlkey=wnb5ypssg79whq2x4iiov6vte&dl=1") %>%
    mutate(Distinction = ifelse(score >= 70, 1, 0))

data_rdd %>%
    slice(1:5)
```


## Linear Regression Analyses

-   Run a linear regression: `salary ~ Distinction`

\footnotesize
```{r}
#| echo: true
feols(
    fml  = salary ~ Distinction,
    data = data_rdd
) %>%
    modelsummary(
        stars = T,
        gof_map = c("nobs", "r.squared")
    )
```

\normalsize
-   The result suggests that Distinction can increase the salary by 5.5k, which is likely over-estimated due to omitted variable bias.

## RDD Analysis

-   Step 1: Select a bandwidth around the cutoff, between 68% to 72%
-   Step 2: Examine discontinuity of other variables (randomization check).
-   Step 3: Run a linear regression on the subsample.

```{r}
result_RDD <- feols(
    fml = salary ~ Distinction,
    data = data_rdd %>%
        filter(score > 68 & score < 72)
)
```

## RDD Results

```{r}
result_RDD %>%
    modelsummary(
        stars = T,
        gof_map = c("nobs", "r.squared")
    )
```

- The result suggests that Distinction can increase the salary by 2.898k, which is likely a more accurate estimate of the causal effect than the OLS estimate.

## Visualization of RDD

::: {.columns}

::: {.column width='50%'}

\tiny
```{r}
#| echo: true
#| eval: false
pacman::p_load(ggplot2, ggthemes)
data_rdd %>%
    ggplot(aes(x = score, y = salary)) +
    geom_point() +
    geom_vline(xintercept = 70, linetype = "dashed") +
    labs(
        title = "RDD: Distinction on Salary",
        x = "Final Score",
        y = "Salary"
    ) +
    geom_smooth(
        data = subset(data_rdd, score < 70),
        method = "lm", se = FALSE,
        color = "red"
    ) +
    geom_smooth(
        data = subset(data_rdd, score >= 70),
        method = "lm", se = FALSE,
        color = "blue"
    ) +
    theme_minimal()
```

:::

::: {.column width='50%'}

```{r}
pacman::p_load(ggplot2, ggthemes)
data_rdd %>%
    ggplot(aes(x = score, y = salary)) +
    geom_point() +
    geom_vline(xintercept = 70, linetype = "dashed") +
    labs(
        title = "RDD: Distinction on Salary",
        x = "Final Score",
        y = "Salary"
    ) +
    geom_smooth(
        data = subset(data_rdd, score < 70),
        method = "lm", se = FALSE,
        color = "red"
    ) +
    geom_smooth(
        data = subset(data_rdd, score >= 70),
        method = "lm", se = FALSE,
        color = "blue"
    ) +
    theme_minimal()
```

:::

:::

## Application of RDD in Marketing and Business Context

- [Scores vs. stars: A regression discontinuity study of online consumer reviews](https://www.sciencedirect.com/science/article/abs/pii/S037872061730054X): 4.49 rating vs. 4.5 rating result in a significant difference in sales.

- [The causal effect of Uber's surge pricing on demand and driver labor supply](https://www.nber.org/papers/w22627): On the Uber platform, surge multiplier is rounded to the nearest 0.1, which creates a natural experiment for RDD.


## After-class Reading

-   (recommended) [Quasi-experiment](https://www.econometrics-with-r.org/13-4-quasi-experiments.html) (Econometrics with R)