Elevator pitch
Good policy requires reliable scientific knowledge, but there are many obstacles. Most econometric estimates lack adequate statistical power; some estimates cannot be replicated; publication selection bias (the selective reporting of results) is common; and there is wide variation in the evidence base on most policy issues. Meta-regression analysis offers a way to increase statistical power, correct the evidence base for a range of biases, and make sense of the unceasing flow of contradictory econometric estimates. It enables policymakers to develop evidence-based policies even when the initial evidence base lacks credibility.
Key findings
Pros
Meta-regression offers a systematic scientific approach to research synthesis and quantitative review of the evidence base.
Meta-regression methods can be used to formally test hypotheses and rival theories.
Meta-regression is particularly useful for explaining sources of the wide variation in reported econometric estimates.
Meta-regression can correct the evidence base for specification bias and reduce the effects of publication selection bias.
Involving simple regression analysis and standard software, with well-established guidelines and protocols, meta-regression is easy to implement.
Cons
Identifying relevant studies and extracting and coding the data needed to conduct meta-regression analysis is very labor intensive.
Inexperienced coders can make coding errors, thereby biasing estimates.
Researcher bias in making modeling choices can negatively affect meta-regression analysis just as it can affect primary data analysis.
Meta-regression methods are less effective for research areas in which there are only a handful of econometric studies.
