Three kinds of causal claim

Three kinds of causal claim • It works somewhere:T contributes to R somewhere (e.g. in some study population). • It works in general: T contributes to R everywhere, or ‘widely’. • It will work here:T will contribute to R in our population.

Duflo and Kremer: ‘Use of Randomization in the Evaluation of Development Effectiveness’ ‘‘The benefits of knowing which programs work …extend far beyond any program or agency, and credible impact evaluations… can offer reliable guidance to international organizations, governments, donors, and…NGO’s beyond national borders.” • Which programs work = it works ‘in general’ • Impact evaluation = it works somewhere • Reliable guidance = it will work here.

It is a long tortuous road from ‘It works somewhere’ to ‘It will work here’.

T contributes to R under the causal principles for the study population. • ??? Therefore, T will contribute to R under the principles that hold here.

T contributes to R under the causal principles for the study population. • T contributes everywhere (‘everywhere enough’) Therefore, T will contribute to R under the principles that hold here.

T contributes to R under the causal principles for the study population. (From RCT) • T contributes everywhere (‘everywhere enough’) (From ???) Therefore, T will contribute to R under the principles that hold here.

Pressing a lever….

R = α + bT + cW Average effect size = Exp(b) [T – T’] T = value of the treatment in the treatment wing T’ =value in the control wing.

Rent here?

R = α + bT + cW Average effect size = Exp(b) [T – T’] T = value of the treatment in the treatment wing T’ =value in the control wing.

You can’t pave the way to the conclusions you need in EBM with gold bricks

Three kinds of causal claim