Regression discontinuity estimates a programme effect by comparing people just above and just below an eligibility cutoff. This guide explains the idea in plain language, the checks that make it credible, and how to run and read it in AnalyZ Solutions.
Regression discontinuity design estimates the effect of a programme by comparing people just either side of an eligibility cutoff. If everyone who scores 60 or above gets a scholarship and everyone below does not, then students who scored 61 and students who scored 59 are almost the same kind of student. What happened to them afterwards can be compared fairly, because the only systematic difference between them is the scholarship.
The name describes what the method looks for. Regression means a line fitted to the data. Discontinuity means a sudden break. The method fits a line through the outcomes on each side of the cutoff and measures the break between the two lines where they meet.
Take a vocational scholarship awarded to every student who scores at least 60 on an entrance test. You follow 1,600 students and record their monthly income two years later. Income tends to rise with the test score, because stronger students do better anyway, so you expect a gentle upward slope. If the scholarship helps, there will also be a step up at 60.
Use regression discontinuity when a programme's eligibility is decided by a threshold on a continuous measure. Typical cases include a poverty score below which households receive a cash transfer, an age at which a benefit starts, a test score for a scholarship, or an income limit for a subsidy.
It works well when:
| Design | When it applies | What the module estimates |
|---|---|---|
| Sharp | The cutoff decides treatment completely. Everyone at or above it gets the programme and nobody below does. | The jump in the outcome at the cutoff. |
| Fuzzy | Crossing the cutoff raises the chance of getting the programme but does not settle it. Some eligible people do not take it up, or some ineligible people get it anyway. | The jump in the outcome divided by the jump in the share who actually received the programme. |
Whether a student scores 59.5 or 60.5 depends partly on chance: a question that happened to suit them, a good night's sleep, a marker's judgement. So students on either side of the line differ little in ability, motivation or family background. If incomes jump at the line, the best explanation is the scholarship.
Figure 1 shows the example. Each dot is the average income of students in a two-point band of scores. Incomes rise slowly with the score, then step up at 60. The two green lines are fitted to students within 8 points of the cutoff, and the gap between them at 60 is the estimate, about $29.
The estimate applies to students close to the cutoff. It says what the scholarship did for students scoring around 60. It does not say what it would do for a student who scored 95 or one who scored 30. This limitation is the price of a very credible comparison.
The coefficient called effect is the gap between the lines at the cutoff. Within the window, every student counts equally.
A narrow window gives a fairer comparison, because the students are more alike, but it leaves fewer students and a noisier estimate. A wide window gives more precision, but it brings in students who are less alike and whose incomes may follow a curve that a straight line misses. The module offers two ways to choose.
Whatever you choose, the module also re-runs the estimate at half, three-quarters, one and a half times and twice the width, so you can see whether the answer depends on the choice.
Suppose crossing the cutoff raises the share who actually receive the programme from 10% to 90%. The outcome jump then reflects the programme in only 80% of cases, so it is divided by 0.8. The module does this with a two-stage method, the same one used for instrumental variables, and reports the first-stage jump so you can see how strong the link between the cutoff and the programme is.
Each estimate comes with a robust standard error, a t statistic, a p-value and a 95% confidence interval. Because only the students inside the window are used, the standard error is usually larger than in other methods. That is the cost of a comparison this clean.
The design, the number of students inside the bandwidth and on each side of the cutoff, the bandwidth and the cutoff. If one side has very few students, the estimate rests on little data.
The estimate in the units of the outcome, with its standard error, t statistic, p-value and 95% confidence interval. For a fuzzy design this is labelled as a local average treatment effect.
R-squared and an F test for the local regression inside the window. As with any design that compares groups, a modest R-squared does not weaken the estimate.
The jump in the probability of receiving the programme at the cutoff. A jump near zero means the cutoff barely changes who gets the programme, and the fuzzy estimate will be unreliable.
| Check | What it asks | What you want to see |
|---|---|---|
| Manipulation check (density) | Are there more or fewer people just above the cutoff than just below? | No significant difference. A jump in the number of people at the line suggests that scores were moved. |
| Sorting check (counts) | Are the numbers of observations on the two sides of the window similar? | No significant difference. |
| Covariate balance | Do characteristics measured before the programme, such as age, jump at the cutoff? | No significant jump. A jump suggests something other than the programme changes at the line. |
| Bandwidth sensitivity | Does the estimate hold across a range of window widths? | Estimates of similar size and sign. |
| Placebo cutoffs | Does the method find an effect at fake cutoffs, one below and one above the real one? | No significant effect. A significant placebo result is a warning sign. |
Open Effect Analysis and choose Regression Discontinuity.
The figures below are invented to show the steps, and they come from running the module on a made-up dataset of 1,600 students. The running variable is the entrance test score, the cutoff is 60, and the outcome is monthly income in dollars two years later. The design is sharp, and the covariate is age. For this example the window was set by hand to 8 points either side of the cutoff.
| Design | N in window | N left | N right | Bandwidth | Cutoff |
|---|---|---|---|---|---|
| Sharp | 636 | 313 | 323 | 8.0 (manual) | 60 |
| Estimand | Estimate | SE | t | p-value | 95% CI lower | 95% CI upper |
|---|---|---|---|---|---|---|
| RD Treatment Effect | 29.13 | 4.39 | 6.63 | < 0.001 | 20.52 | 37.74 |
| Check | Result | p-value |
|---|---|---|
| Density just left of cutoff | 0.0632 | |
| Density just right of cutoff | 0.0672 | 0.7989 |
| Counts in the window | 313 left, 323 right | 0.721 |
| Age: jump at cutoff | 0.724 | 0.1224 |
| Bandwidth | Window width | Estimate | SE | p-value |
|---|---|---|---|---|
| 0.5x | 4 | 26.30 | 5.96 | < 0.001 |
| 0.75x | 6 | 31.20 | 5.01 | < 0.001 |
| 1.0x | 8 | 29.13 | 4.39 | < 0.001 |
| 1.5x | 12 | 32.38 | 3.69 | < 0.001 |
| 2.0x | 16 | 32.81 | 3.24 | < 0.001 |
| Placebo cutoff | Cutoff value | Estimate | p-value |
|---|---|---|---|
| Below true cutoff | 49.0 | -8.76 | 0.1000 |
| Above true cutoff | 70.1 | -2.18 | 0.6621 |
Two years after the entrance test, students who scored just above the scholarship cutoff earned about $29 more per month than students who scored just below it (95% confidence interval $21 to $38). The checks found no sign that scores were manipulated or that the two sides differed in age, and the result held across windows from 4 to 16 points.
Leaving the default cutoff. The module starts with the median of the running variable. If you forget to enter the real threshold you will be estimating a jump where nothing happened.
Ignoring signs of manipulation. If staff round scores up to help someone pass, or applicants report an income just under the limit, the two sides of the cutoff are no longer comparable. A jump in the density is the warning.
Quoting the effect for everyone. The estimate holds near the cutoff. Students far above it may respond quite differently.
Choosing the window to get the answer you want. Decide the bandwidth first, and report the sensitivity table whichever way it comes out.
Using a running variable with few values. A score from 1 to 10 gives ten points to fit lines to. Many distinct values near the cutoff are needed.
Mixing up sharp and fuzzy. If some people above the cutoff did not take the programme, a sharp analysis estimates the effect of being offered it, not the effect of receiving it.
Open Effect Analysis in AnalyZ Solutions. Free to sign up, free to analyze.
Get started