Power and Statistical Significance in Securities Fraud Litigation

The event study—a statistical tool borrowed from financial economics—has become a critical tool in securities fraud litigation. In litigation, event studies are used to measure the extent to which market prices react to the release of new information. Their results are introduced as evidence on the efficiency the market in which the securities trade, the impact of the fraudulent disclosures on market prices, the causal relationship between the fraud and plaintiff’s economic harm, and the appropriate calculation of damages. Courts vary both in the extent to which they require the use of an event study and the degree to which they accept other evidence with respect to these issues, but a properly-conducted event study is often a key factor.

The event study methodology is used to distinguish between normal fluctuations in stock price and returns so large that their magnitude is deemed to be explained by the release of material information. This is done through a process known in statistics as null hypothesis significance testing. An event study analyzes—on the basis of the size of the stock price reaction — whether the price movements in question are within normal limits or are sufficiently unusual that they are likely to have been caused by the disclosure. A stock price movement that is sufficiently unusual that it can be used to infer a causal relationship is termed “statistically significant.”

The social science literature from which the event study methodology is drawn relies on a confidence level of 95% for a finding of statistical significance. That is, the academic standard would permit the conclusion that there is a causal relationship if the study establishes that a stock price movement as large as the observed one would not occur more than 5% of the time if only chance variations were at play. This requirement of a 95% confidence level has to a great extent been imported into the law. Courts have repeatedly held that a “properly conducted” event study demonstrating a statistically significant price effect is necessary to establish or rebut key elements of a securities fraud claim. Similarly, courts have rejected efforts to establish those elements that fail to establish a causal relationship at the 95% confidence level. The courts have not, however, interrogated either the theoretical basis for using a 95% confidence level or the extent to which it reflects an appropriate standard for legal purposes.

In our paper, Power and Significance in Securities Fraud Litigation, forthcoming in the Harvard Business Law Review, we challenge the use of the 95% confidence level. The primary basis for our objection is an issue that the courts have largely overlooked, the relationship between confidence level and power. As we explain, confidence level addresses the possibility of mistakenly allowing non-meritorious claims to proceed—cases in which the disclosure in question did not, in fact, cause a stock price reaction. Power addresses a complementary concern—the erroneous rejection of meritorious claims. Importantly, however, the structure of an event study requires a tradeoff between confidence level and power. Specifically, in many situations, requiring that event studies establish a causal relationship at the 95% confidence level leads those studies to have low power. As a result, event studies that must satisfy the standard of a 95% confidence level, are likely to reject a substantial number of cases of true fraud.

We explain this trade-off in language accessible to lawyers, judges and regulators, with numerical examples. We demonstrate, for example, the impact of stock price and volatility on the difficulty of meeting the 95% threshold. We also show that, under reasonable assumptions, requiring an event study to meet this threshold can result in a high probability that meritorious cases will be rejected.

The tradeoff between confidence level and power means that deciding what confidence level to require for securities fraud event studies is not a matter of objective scientific truth but instead a normative judgment about the appropriate level of difficulty required to establish a securities fraud claim. The concept of statistical significance involves a fundamental policy choice between reducing the risk of imposing liability for a disclosure that did not affect stock prices and increasing the likelihood that defendants will be held accountable for fraudulent disclosures. In their use of the event study methodology and reliance on the social science literature, the courts have not confronted this policy choice directly.

Having revealed the policy choice inherent in the use of statistical significance, it becomes clear that securities litigation should not borrow unthinkingly from empirical practice in the social sciences but should instead determine the appropriate threshold of statistical significance based on the tradeoff between these competing concerns. We therefore consider, from the perspective of comparative institutional competence, which part of our system of securities regulation is best suited to make this policy choice. Although we recognize that Article III courts currently handle this decision implicitly by the standards they impose on the admissibility and persuasiveness of expert testimony, we argue that they are not well situated to do so. Federal judges are poorly positioned to weigh the policy considerations reflected by the tradeoff between confidence level and power and to consider the impact that shifting the extent of the tradeoff will have on the deterrence of fraud and the promotion of market integrity. Although the policy analysis we describe could be conducted by Congress, we argue that Congress lacks both the expertise and the political will to make this determination, and that monitoring the market’s response is also likely to require a level of flexibility and adjustment that one-off legislation is poorly suited to provide.

As a result, we argue that the Securities & Exchange Commission (SEC) should decide the appropriate level of statistical significance to be used in securities litigation event studies. We argue that balancing litigation costs against fraud costs is precisely the type of determination that expert administrative agencies were designed to make. The SEC is well suited to make this determination based both on the technical expertise of its staff of economists and its familiarity with the role of private enforcement in serving the objectives of the federal securities laws. The SEC also has unique access to data that allows it to evaluate the costs involved and the flexibility to adjust its rule in response to changes in market conditions or the behavior of market participants.

Our paper therefore proposes that the SEC engage in formal rulemaking to set the level of statistical significance necessary for an acceptable event study in securities fraud litigation. Although the SEC could use a variety of approaches to analyze the appropriate policy considerations that should inform the choice of confidence level, we conclude by offering one possible approach. Our framework is based on ensuring that a minimum level of power is obtained for a benchmark fraud magnitude, based on the SEC’s judgment about the level of enforcement necessary to provide sufficient deterrence of fraud. We show that, given knowledge of the defendant firm’s market capitalization and abnormal returns distribution, it is straightforward to determine the maximum confidence level (minimum significance level) that is consistent with the minimum required power of detecting a fraud of the benchmark magnitude.

Our proposal has the twin virtues of flexibility and objectivity. The approach is flexible because it allows the required confidence level to vary with the characteristics of the defendant firm. It is objective because it relies on two readily-ascertain pieces of information: the defendant firm’s market value and the structure of the defendant firm’s abnormal return distribution. Finally, the SEC is well-positioned to observe the impact of its determination on both market integrity and litigation quality and to adjust its standard in order attain a cost-effective litigation level to deter securities fraud.

The complete paper is available for download here.