Methods

  • Using linear regression to establish empirical relationships

    Linear regression is a powerful tool for estimating the relationship between one variable and a set of other variables

    Marno Verbeek, February 2017
    Linear regression is a powerful tool for investigating the relationships between multiple variables by relating one variable to a set of variables. It can identify the effect of one variable while adjusting for other observable differences. For example, it can analyze how wages relate to gender, after controlling for differences in background characteristics such as education and experience. A linear regression model is typically estimated by ordinary least squares, which minimizes the differences between the observed sample values and the fitted values from the model. Multiple tools are available to evaluate the model.
    MoreLess
  • Using instrumental variables to establish causality

    Even with observational data, causality can be recovered with the help of instrumental variables estimation

    Sascha O. Becker, April 2016
    Randomized control trials are often considered the gold standard to establish causality. However, in many policy-relevant situations, these trials are not possible. Instrumental variables affect the outcome only via a specific treatment; as such, they allow for the estimation of a causal effect. However, finding valid instruments is difficult. Moreover, instrumental variables estimates recover a causal effect only for a specific part of the population. While those limitations are important, the objective of establishing causality remains; and instrumental variables are an important econometric tool to achieve this objective.
    MoreLess
  • Transparency in empirical economic research

    Open science can enhance research credibility, but only with the correct incentives

    The open science and research transparency movement aims to make the research process more visible and to strengthen the credibility of results. Examples of open research practices include open data, pre-registration, and replication. Open science proponents argue that making data and codes publicly available enables researchers to evaluate the truth of a claim and improve its credibility. Opponents often counter that replications are costly and that open science efforts are not always rewarded with publication of results.
    MoreLess
  • The usefulness of experiments

    Are experiments the gold standard or just over-hyped?

    Jeffrey A. Smith, May 2018
    Non-experimental evaluations of programs compare individuals who choose to participate in a program to individuals who do not. Such comparisons run the risk of conflating non-random selection into the program with its causal effects. By randomly assigning individuals to participate in the program or not, experimental evaluations remove the potential for non-random selection to bias comparisons of participants and non-participants. In so doing, they provide compelling causal evidence of program effects. At the same time, experiments are not a panacea, and require careful design and interpretation.
    MoreLess
  • The use of natural experiments in migration research

    Data on rapid, unexpected refugee flows can credibly identify the impact of migration on native workers’ labor market outcomes

    Semih Tumen, October 2015
    Estimating the causal effect of immigration on the labor market outcomes of native workers has been a major concern in the literature. Because immigrants decide whether and where to migrate, immigrant populations generally consist of individuals with characteristics that differ from those of a randomly selected sample. One solution is to focus on events such as civil wars and natural catastrophes that generate rapid and unexpected flows of refugees into a country unrelated to their personal characteristics, location, and employment preferences. These “natural experiments” yield estimates that find small negative effects on native workers’ employment but not on wages.
    MoreLess
  • Relative deprivation in the labor market

    The choice of reference group crucially determines subjective deprivation and thus affects labor market behavior

    Paolo Verme, June 2017
    Why do different population groups (e.g. rural vs. urban, youth vs. elderly and men vs. women) experience the same objective labor status differently? One hypothesis is that people are more concerned with relative deprivation than objective deprivation and they value their own status relative to the status of their peers—the reference group. One way to test this hypothesis in the labor market is to measure individual differences in labor status while controlling for characteristics that define population groups. This measure is called “relative labor deprivation” and can help policymakers to better understand how labor claims are generated.
    MoreLess
  • Randomized control trials in an imperfect world

    How can we assess the policy effectiveness of randomized control trials when people don’t comply?

    Zahra Siddique, December 2014
    Randomized control trials (RCTs) have become increasingly important as an evidence-based method to evaluate interventions such as government programs and policy initiatives. Frequently, however, RCTs are characterized by “imperfect compliance,” in that not all the subjects who are randomly assigned to take a treatment choose to do so. This could result in a failure to identify the treatment effect, or the impact of the treatment on the population. However, useful information on treatment effectiveness can still be recovered by estimating “bounds,” or a range of values in which treatment effectiveness can lie.
    MoreLess
  • Meta-regression analysis: Producing credible estimates from diverse evidence

    Meta-regression methods can be used to develop evidence-based policies when the evidence base lacks credibility

    Chris Doucouliagos, November 2016
    Good policy requires reliable scientific knowledge, but there are many obstacles. Most econometric estimates lack adequate statistical power; some estimates cannot be replicated; publication selection bias (the selective reporting of results) is common; and there is wide variation in the evidence base on most policy issues. Meta-regression analysis offers a way to increase statistical power, correct the evidence base for a range of biases, and make sense of the unceasing flow of contradictory econometric estimates. It enables policymakers to develop evidence-based policies even when the initial evidence base lacks credibility.
    MoreLess
  • Measuring the cost of children

    Knowing the real cost of children is important for crafting better
 economic policy

    Olivier Donni, March 2015
    The cost of children is a critical parameter used in determining many economic policies. For instance, correctly setting the tax deduction for families with children requires assessing the true household cost of children. Evaluating child poverty at the individual level requires making a clear distinction between the share of family resources received by children and that received by parents. The standard ad hoc measures (equivalence scales) used in official publications to measure the cost of children are arbitrary and are not informed by any economic theory. However, economists have developed methods that are grounded in economic theory and can replace ad hoc measures.
    MoreLess
  • Measuring individual risk preferences

    Incentivized measures are considered to be the gold standard in measuring individuals’ risk preferences, but is that correct?

    Catherine C. Eckel, June 2019
    Risk aversion is an important factor in many settings, including individual decisions about investment or occupational choice, and government choices about policies affecting environmental, industrial, or health risks. Risk preferences are measured using surveys or incentivized games with real consequences. Reviewing the different approaches to measuring individual risk aversion shows that the best approach will depend on the question being asked and the study's target population. In particular, economists’ gold standard of incentivized games may not be superior to surveys in all settings.
    MoreLess
show more