The 95% confidence fallacy

While a 95% confidence intervalⁱ in data is the target, banks will really have only 80-90% confidence in their data today. In a 2024 study of sell-side reference data operations, over 90% reported that poor data quality caused issues in clearing and settlement, risk management, and regulatory reporting, with 80% citing challenges in automated trading and market connectivity emanating from inaccurate data.ⁱⁱ Moreover, that 80-90% is a bit of an illusion. Here's the reality. Say, I am a bank CTO or chief data scientist, and I have 80% confidence in the data that's coming to me via any type of transaction. I then push that data into the clearing or matching process. Then, I push it into the settlement process; and there's cash movement that goes along with this. That data keeps getting pushed from one process to the next, to the next, and the next, which means there's a little bit of degeneration that happens all the way through. By the time I get to the end of my processes, I have 50% confidence in my data, and that little anomaly from the first process becomes a serious data problem 10 steps later. However, this is an inscrutable problem to recognize, much less solve. It depends on the robustness of the institution's existing data and operational infrastructure, the stage of its data transformation journey, and the asset classes and structures involved.

Meanwhile, the risk of getting it wrong is high. On the undesirable end of the 95% spectrum, Citi shelled out about a billion dollars in fines in the last 5 years for irregularities in its regulatory reporting data and governance failures; and responded by spending millions modernizing its technology.ⁱⁱⁱ Deutsche Bank, Wells Fargo, and Mitsubishi Bank are examples of institutions that have worked through confidential supervisory findings called Matters Requiring Attention (MRAs) and Matters Requiring Immediate Attention (MRIAs). Many of these have been rooted in data processes. In this context, even 95% (and even if it were a true 95%) isn’t enough for global banks — UBS, for instance, has a balance sheet larger than the Swiss economy. A Swiss bailout of such a bank is challenging. The risk needs to be near-zero, which means confidence needs to be near-perfect.

A prime example: private credit’s unstructured data problem

There is probably no better example of how this plays out than in private credit, where a lack of a uniform standard for data coming from private credit loan documents, originators, term sheets, mortgage servicers, and administrators is creating major challenges for financial institutions across capital markets, including banks.

When policymakers put the clamp on banks after the 2008 crisis, private credit was not on anyone’s radar. It soon became the thing, and now it is never going away. With much less regulation and transparency in private markets, regulators have now awakened to a bigger problem that they can't get their arms around. While regulators focus on private credit’s risks as retail investors gain greater access, banks are partnering with private credit in ways that allow them to do the things they do well as traditional advisories. In the past couple of years, Wall Street has made inroads in reclaiming lost credit business by modernizing systems and embracing a new hybrid public-private credit world.^iv