Data and methods

Data, and the methods used to analyze them, are the foundation for evidence-based research. Articles in this subject area discuss the value of different types of data collection, and explain important statistical and econometric methods that provide ways to summarize and present information, and to identify and quantify correlation or causality.

  • Counting on count data models

    Quantitative policy evaluation can benefit from a rich set of econometric methods for analyzing count data

    Rainer Winkelmann, May 2015
    Often, economic policies are directed toward outcomes that are measured as counts. Examples of economic variables that use a basic counting scale are number of children as an indicator of fertility, number of doctor visits as an indicator of health care demand, and number of days absent from work as an indicator of employee shirking. Several econometric methods are available for analyzing such data, including the Poisson and negative binomial models. They can provide useful insights that cannot be obtained from standard linear regression models. Estimation and interpretation are illustrated in two empirical examples.
  • Measuring disincentives to formal work

    Does formal work pay? Synthetic measurements of taxes and benefits can help identify incentives and disincentives to formal work

    Michael Weber, December 2015
    Evidence from transition economies shows that formal work may not pay, particularly for low-wage earners. Synthetic measurements of work disincentives, such as the formalization tax rate or the marginal effective tax rate, confirm a significant positive correlation between these measurements and the probability of informal work. These measures are especially informative for impacts at lower wage levels, where informality is highest. Policymakers who want to increase formal work can use these measurements to determine optimal labor taxation rates for low-wage earners and reform benefit design.
  • Can lab experiments help design personnel policies?

    Employers can use laboratory experiments to structure payment policies and incentive schemes

    Marie Claire Villeval, November 2016
    Can a company attract a different type of employee by changing its compensation scheme? Is it sufficient to pay more to increase employees’ motivation? Should a firm provide evaluation feedback to employees based on their absolute or their relative performance? Laboratory experiments can help address these questions by identifying the causal impact of variations in personnel policy on employees’ productivity and mobility. Although they are collected in an artificial environment, the qualitative external validity of findings from the lab is now well recognized.
  • Relative deprivation in the labor market

    The choice of reference group crucially determines subjective deprivation and thus affects labor market behavior

    Paolo Verme, June 2017
    Why do different population groups (e.g. rural vs. urban, youth vs. elderly and men vs. women) experience the same objective labor status differently? One hypothesis is that people are more concerned with relative deprivation than objective deprivation and they value their own status relative to the status of their peers—the reference group. One way to test this hypothesis in the labor market is to measure individual differences in labor status while controlling for characteristics that define population groups. This measure is called “relative labor deprivation” and can help policymakers to better understand how labor claims are generated.
  • Using linear regression to establish empirical relationships

    Linear regression is a powerful tool for estimating the relationship between one variable and a set of other variables

    Marno Verbeek, February 2017
    Linear regression is a powerful tool for investigating the relationships between multiple variables by relating one variable to a set of variables. It can identify the effect of one variable while adjusting for other observable differences. For example, it can analyze how wages relate to gender, after controlling for differences in background characteristics such as education and experience. A linear regression model is typically estimated by ordinary least squares, which minimizes the differences between the observed sample values and the fitted values from the model. Multiple tools are available to evaluate the model.
  • The use of natural experiments in migration research

    Data on rapid, unexpected refugee flows can credibly identify the impact of migration on native workers’ labor market outcomes

    Semih Tumen, October 2015
    Estimating the causal effect of immigration on the labor market outcomes of native workers has been a major concern in the literature. Because immigrants decide whether and where to migrate, immigrant populations generally consist of individuals with characteristics that differ from those of a randomly selected sample. One solution is to focus on events such as civil wars and natural catastrophes that generate rapid and unexpected flows of refugees into a country unrelated to their personal characteristics, location, and employment preferences. These “natural experiments” yield estimates that find small negative effects on native workers’ employment but not on wages.
  • The importance of measuring dispersion in firm-level outcomes

    Ignoring the large variation in firm-level outcomes can create misunderstandings about the consequences of many policies

    Chad Syverson, May 2014
    Recent research has revealed enormous variation in performance and growth among firms, which both drives and is driven by large reallocations of inputs and outputs across firms (churning) within industries and markets. These differences in firm-level outcomes and the associated turnover of firms affect many economic policies (both labor- and non-labor-oriented), on both a microeconomic and a macroeconomic scale, and are affected by them. Properly evaluating these policies requires familiarity with the sources and consequences of firm-level variation and within-industry reallocation.
  • The importance and challenges of measuring work hours

    Measuring hours worked is important, but different surveys can tell different stories

    Jay Stewart, November 2014
    Work hours are key components in estimating productivity growth and hourly wages as well as being a useful cyclical indicator in their own right, so measuring them correctly is important. The US Bureau of Labor Statistics (BLS) collects data on work hours in several surveys and publishes three widely-used series that measure average weekly hours. The series tell different stories about average weekly hours and trends in those hours but qualitatively similar stories about the cyclical behavior of work hours. The research summarized here explains the differences in levels, but only some of the differences in trends.
  • Randomized control trials in an imperfect world

    How can we assess the policy effectiveness of randomized control trials when people don’t comply?

    Zahra Siddique, December 2014
    Randomized control trials (RCTs) have become increasingly important as an evidence-based method to evaluate interventions such as government programs and policy initiatives. Frequently, however, RCTs are characterized by “imperfect compliance,” in that not all the subjects who are randomly assigned to take a treatment choose to do so. This could result in a failure to identify the treatment effect, or the impact of the treatment on the population. However, useful information on treatment effectiveness can still be recovered by estimating “bounds,” or a range of values in which treatment effectiveness can lie.
  • Performance measures and worker productivity

    Choosing the right performance measures can inform and improve decision-making in policy and management

    Jan Sauermann, May 2016
    Measuring workers’ productivity is important for public policy and private-sector decision-making. Due to a lack of reliable methods to determine workers’ productivity, firms often use specific performance measures, such as how different incentives affect employees’ behavior. The public sector also uses these measures to monitor and evaluate personnel, such as teachers. To select the right performance measures, and as a result design better employment contracts and improve productivity, policymakers and managers need to understand the advantages and disadvantages of the available metrics.
show more