Sense and Nonsense in ESG Ratings

Interactions between firms subject to market discipline and the environmental, social and governance (ESG) context within which they operate have proliferated to the point of influencing board and management behavior, stock prices and corporate valuations—as well as the creation of target investment funds tied to specific ESG outcomes.  In this paper we propose a heuristic to frame ESG objectives, extending from classic regulation to fuzzy signals purporting to reflect collective values—and giving rise to corporate behavior that ranges from compliance failure to conduct deemed “irresponsible” or “unethical,” with implications along the way for revenues, costs and exposure to enterprise risk.  We suggest that this heuristic is broadly consistent with the concerns of investors and the fiduciary obligations of asset managers, and forms  a constructive framework for considering interactions with an array of (often overlapping and conflicted) stakeholders.

It was inevitable that a vigorous market would develop for ESG scoring and rating services that would identify key normative targets and calibrate the  degree to which individual firms achieve those targets in the form of metrics that are transparent and defensible and that can be weighted and aggregated to generate composite scores and displayed in alphanumeric form—perhaps adjusted for industry and incorporating secondary and tertiary impacts up the supply chain. This is a heroic task compared, for example, to the mandate facing credit rating agencies (CRAs)—the likelihood of debt service “on time and in full.” The paper examines the key ESG rating issues in some detail as well as the structure, conduct and performance of the ESG rating industry itself.

Properly executed, and subject to problems in specifying normative ESG targets in the presence of significant utility differences, creating metrics can be helpful in focusing on key issues and signaling ways of dealing with them. But identification and causality problems, indicator-aggregation—as well as transparency and replicability—raise serious conceptual and empirical issues. These include self-reporting and “greenwashing,” the quality of the primary data collection process, indicator mismatch in the use of secondary data, factor aggregation and weighting, setting scoring breaks, and reporting formats and comparability across rating firms. This is a formidable list, which can frustrate transparency, replicability, and therefore credibility. In a competitive “investor-pays” ratings marketplace with no shortage of competitors—and perhaps misplaced user confidence in scoring—caution about the contemporary state of play seems warranted.  The paper suggests an array of issues that must be addressed to reinforce ESG assessments and make them more defensible in capital allocation and enterprise management.

Among them, the identification problem must be resolved—what are the normative outcomes that are to be achieved, and how well anchored are they in consensus and public legitimacy? Inter-community and interpersonal utility differences hang over this issue and cannot be fully resolved, so solutions have to be framed within the theory of second-best. Moreover, data inputs must be mapped specifically onto the normative outcomes that motivate the ESG exercise. What inputs can be captured, and are they internally coherent?  “Good enough” data are not good enough when serious capital-allocation decisions and fiduciary obligations are involved. So, plausibility and notional sufficiency—and the massive practical chore of generating and collating hundreds of inputs on thousands of rated entities by armies of analysts and statisticians of unknown competence—remain to be overcome.

In addition, the weighting process used in creating composite ESG indexes is inherently vulnerable, since there are few objective sources of weights (unlike market capitalization or price volatilities in stock market indexes) in the world of ESG. So weights too have to be judgmental, subjective, and subject to challenge. This is unlikely to change in the short term, although cumulative data on stock-price sensitivity to ESG events may in the future make it possible to generate empirically defensible approaches to factor weighting. At the same time, the unobserved-variable issue needs to be addressed. This “blow- by” problem is commonly encountered in areas like sovereign risk ratings. Information that is not captured in data, and  may be largely qualitative and insufficiently granular, represents a perennial problem.

Transparency is likewise critical. Data, ESG target specification, assumptions, adjustments for missing data, quantification of qualitative information, factor weighting, and index construction should be available in the public domain. It ought to be possible for researchers to replicate ESG calibration and subject it to stress testing using alternative inputs and assumptions. But ESG rating follows a subscription (user-pays) business model based at least in part on proprietary information and rating models, so product differentiation may be marketed as the “secret sauce” in a competitive marketplace. Consequently, transparency may be a long time coming. There is also the need for performance measurement and reporting on the part of investment funds marketed as ESG-driven. Investors are entitled to forensic audits confirming the extent to which progress toward the ESG goals has actually been achieved, alongside the conventional fund reporting. This will not be easy, and is likewise dependent on progress on the issue of defensible ESG performance metrics.

Maturity and commonality are credibility factors that will come only with time. Credit ratings derived in more or less the same way by different CRAs may not be identical, but they will tend not to show extreme differences. Rating convergence has long been common among the three leading agencies, and any rating differences and watch-lists can be examined for remaining differences of view among the respective rating committees. The ESG rating business may well follow the same path, which implies a global oligopolistic market structure. But that will take time and could create a whole different set of problems.

Finally, there is the issue of certification. The CRA path offers a model in terms of recognition by the SEC in 1975 of Nationally Recognized Securities Rating Organizations (NRSROs). Recognition by a serious, politically empowered regulator is important, as are the standards applied by that regulator. It is doubtful that NGO rules like the 2006 UN Principles for Responsible Investment (PRI) will suffice. Regulatory oversight carries both costs and benefits. The costs of aligning with regulatory mandates are clear, both in obtaining operating licenses and ongoing compliance. The benefits derive from certification that is awarded in the public interest. The closer the ESG rating industry comes to the NRSRO model, the greater is likely to be its credibility and impact—but at the same time the higher the likelihood of oligopoly among surviving ESG rating entities.

The complete paper is available for download here.