Problems Using Aggregate Data to Infer Individual Behavior

Clifford G. Holderness is Professor of Finance at the Carroll School of Management at Boston College. This post is based on a forthcoming article by Professor Holderness.

Many studies in finance and beyond compare firms and markets across countries. These studies have been influential, especially in the area of corporate governance. There is a rarely discussed—indeed hardly noticed—split in how researchers seek to explain differences in firms or individuals across countries. Some papers form country averages of a particular characteristic, such as the use of internal rather than external financing. These country averages are then used as the dependent variable in any empirical analysis. Other papers use the underlying firm observations as the unit of analysis. None of these papers discuss their decision to go one route or the other.

Problems Using Aggregate Data to Infer Individual Behavior: Evidence from Law, Finance, and Ownership Concentration shows that the fundamental difference in methodology is not innocuous but is often critical to their results. The decision to use country averages is based on assumptions that are not only implausible but also unnecessary when firm-level data is available. This calls into question the findings of many individual-firm hypotheses that have been studied with country averages. Among these topics are capital-structure choice, earnings management, and stock-price movements. The findings from these many diverse studies may not be wrong. They do, however, require re-analysis with individual-firm data.

I illustrate the fundamental differences between individual data analysis and aggregate data analysis with an influential finding from the law-and-finance literature, the inverse relation between legal protections for public market investors and the ownership concentration of public corporations. Understanding why ownership concentration varies around the world is central to many influential papers. All of the existing papers use country averages to find an inverse relation between investors’ legal protections and ownership concentration. These papers provide the empirical foundation for the now-influential proposition that large-percentage shareholders are a response to weak legal protections for public market investors.

My article illustrates how country averages and the underlying individual observations can produce very different results. It analyzes three measures of investors’ protection that are central to a broad literature (not just the law-and-ownership literature): (A) the rights of shareholders to sue corporate directors (the Anti-Director Rights Index of LLSV); (B) a common-law legal origin; and (C) legal prohibitions on self-dealing by corporate insiders (the Anti-Self-Dealing Index of DLLS). I confirm the existing-literature finding that there is a statistically significant inverse relation between each of these legal protections and country-averaged ownership concentration. But when the same data is used on an individual-firm basis, the Anti-Director Rights Index reverses sign, and both Legal Origins and the Anti-Self-Dealing Index become completely insignificant. These analyses use the same ownership data and regression specifications that were originally used in the literature to establish the claimed inverse relation between investors’ legal protections and ownership concentration.

My article shows that inferences change when the unit of analysis is the underlying individual-firm observations rather than country averages. There are three good reasons not to aggregate:

  1. Country averages cannot control for firm-level determinants.
  2. Country averages weight firms differently depending on the composition of the database used.
  3. Country averages distort standard errors by eliminating all within-country variation in ownership, creating a misleading impression with artificial clustering.

The wide use of country averages is surprising because there have been many warnings by statisticians over the years that aggregate data analysis can produce misleading inferences about individual units. These warnings started as early as Pearson et al. (1899) and Yule (1903) and have continued through the distinguished statistician David Freedman (2006a, p. 4028) who warns that “it is all too easy to draw incorrect conclusions from aggregate data.” These warnings triggered a decline in the use of averages as the unit of analysis in many other fields. In contrast, in finance the analysis of country averages has accelerated for reasons that are not articulated and probably not known by their practitioners.

The full article is available for download here.

Both comments and trackbacks are currently closed.