Interpreting The Stress Tests

Evidently there is now a considerable debate out there about the famous (or should that be infamous) CEBS stress tests. Methodologically all sorts of weaknesses have been identified, but in many cases these are decidedly beside the point. It is important to be aware what the tests were (and weren’t) designed to show. They were, it seems to me, essentially designed to free up lending in the short term European interbank market, nothing more, nothing less. This would be useful since it would enable the ECB to step out of playing this particular role. And it may well happen, since if everyone can agree that no European bank is going to fail tomorrow, or be allowed to fail tomorrow, then there should be no difficulty for one bank to lend to another for 24 hours, and so on.

But this issue is a quite separate one from the longer term funding needs of the Spanish banking system, for example, or from the longer term solvency of Greek sovereign debt, where a large quantity of asset backed securities of one kind or another need to be re-financed in the months and years to come. This is a much more complicated issue, since no one has a really very clear idea of the longer term value of the securities which back the pieces of paper, either in the Spanish or the Greek cases, and the stress tests have done nothing to resolve this issue. And that is not surprising, since they were never intended to serve that purpose.

What the tests have I think made reasonably clear is that no EU bank or sovereign will be allowed to fail between now and the end of 2011. They will not be allowed to fail, quite simply because the ECB and the European Financial Stability Facility are there to guarantee that they don’t. So something is something. The Eurozone was created without due care and attention being paid to the kind of institutional backdrop which would be required to support it if things went wrong. Now things certainly have gone wrong – in ways which I think were perfectly foreseeable, but let’s not push that one too hard right now – and we are in the process of putting some of the institutional support in place. Like Gaudi’s famous Sagrada Familia, what we have is a work in progress, with no evident end-date in sight, and no detailed blueprint of what the thing will finally look like. Debate about the future is, as they say, “ongoing”, and the situation is “fluid”.

At the same time issues about the per se usefulness of the kinds of tests the CEBS have just carried out remain, and continue to be legitimate areas of discussion. Some people have spoken about the likelihood of tail risk events (not foreseen in the stress tests). But to some extent even this argument misses the point, since I am not sure that what we are talking about are “tail risks” in a situation like the Spanish or Greek one. The kind of cascading scenario (debt snowball, for example) we could see actually forms part of what anyone with a solid grasp of the underlying macro should actually expect to happen if no one does something to ensure it doesn’t, for the simple reason that all the various economic agents are effectively “inter-linked”, so when one part of the system goes down, then the rest can come crashing down behind it. And this is what will almost inevitably happen if someone, somewhere doesn’t find a way to revert the Spanish and Greek economies to a sustainable growth path.

With this in mind, and with due regard to the fact that most of the models conventional economists and financial analysts work with make all kinds of “linearity” assumptions when in fact many of the processes involved are decidedly non-linear, and subject to various kinds of interconnectedness issues and feedback loops, I though it might be useful to reproduce here an argument to this effect recently made by my friend, the Catalan economist Jordi Molins. So without more fuss or flourish, here its is.

Stressing the European Stress Tests

by Jordi Molins Coronado

The stress tests corresponding to the European financial system have been already published by CEBS. The total recapitalization needs for the whole 91 banks included in the tests, accounting approximately two thirds of the European financial system, is about €3.5bn.

I urge the reader to read again the last statement. Yes, €3.5bn. Not €35bn or €350bn. To put figures into perspective, €3.5bn is the bonus pool of a few big European banks.

Can anybody believe that with an additional recapitalization of €3.5bn, the European financial system would be sound again? That the blockade we have had in the wholesale and money markets could have been avoided by a recapitalization of €3.5bn?

One cannot say the calculations performed by the CEBS, with the support of the ECB and the EC, are wrong. I have been able to follow some of the presentations of the results, and they are quite sound. One can see that a lot of great work has been performed, and brilliant minds are behind the calculations.

How can it be then that the final result is an absurd one?

And definitely one can state the result is nonsensical. I understand that European governments already recapitalized their domestic financial systems with about €200bn. I understand that these stress tests only consider solvency risks, not liquidity risks. And from solvency risks, some of them are discarded (eg, sovereign risk in the banking book, even though officially policymakers argue on the contrary).

One can criticize other aspects of the stress tests: tier I capital has been used instead of core tier I, the usual measure of capital among financial analysts, with the argument that there is no common, accepted definition of core tier I capital across European nations. I prefer the argument that Commerzbank, for example, has a core tier I capital of about 3.5% and a tier I capital of 10.5%.

I have seen all those criticisms. However, I do not think they are the core of the problem. In fact, policymakers have been diverting the problem towards a sovereign one, when in reality, it is a banking book one. Well, that is not true in all cases. For example, Greece has a serious problem of sovereign debt. French and German banks are loaded with periphery sovereign bonds, whose haircuts would imply heavy losses for them. But the key of these stress tests is not Greece, but Spain.

Spain is at the core of the problems within the European financial system. Greece, Portugal and Ireland are small economies. Spain is a big European economy, and highly interconnected with its Central European counterparts. And definitely, the main problem of Spain is not a sovereign debt one, but a private debt one.

My main criticism of the stress tests is the treatment of the loan book. To make a long story short, losses in the banking book are computed in the standard way in the credit arena: exposure at default times probability of default times loss given default. The crux of the matter is how PDs and LGDs are computed: in the stress tests, one takes an estimation of PDs and LGDs as per their realized value on 2009, and then computes a regression that determines the macroeconomic relationship between PDs and LGDs on one hand, and on the other hand, macroeconomic variables like GDP, interest rates or unemployment rate.

Once this relationship has been entertained, one can ’stress’ the macroeconomic variables (-2.6% GDP, shift upwards the interest rates by 75/125bp …) and find, through the macroeconomic relationship, which impact there would be on PDs and LGDs if such a macroeconomic shock were realized.

Then, using those PDs and LGDs, calculating expected losses in the loan book is just a question of doing the arithmetic right.

What is the problem with this approach?

The problem is that in order to compute a regression, one needs historical data. As such, the results are dependent on the previous relationships among the variables, in our case, PDs and LGDs vs GDP, interest rates and unemployment. A regression gives us the ‘best’ (in a technically defined sense) relationship among those variables in the past.

However, in practice theory is challenged through the so-called ‘outliers’. Outliers are data points that are not well represented by the regressed relationships. An outlier is a PD or an LGD that is very far away from the predicted value, assuming a set of macroeconomic data. And clearly there are outliers all the time, especially when the economic environment changes, for example when there is a financial crisis.

I have not done the math, but I would be extremely surprised if the 30%-40% loss rates in US subprime, or the 47% haircut in commercial real estate in the Irish NAMA, could have been explained with a regression using macroeconomic data. These data points are outliers because once a financial crisis bursts, the dynamics linking the economic variables (PDs and LGDs vs macroeconomic data) changes dramatically, and the past relationships break down completely during the crisis.

Which are these new dynamics? This is a very hard question, but I would like to stress at least a part responsible of the change. Non-linear network effects. For example, when Detroit lost approximately about half of its population, it was not (at least, not only) because of the unemployment rate of that city spiking up. It was also because if your brother or your friend left the city, why should you stay there? Definitely, macro effects continue playing a role. But the dramatic outliers are caused by positive/negative feedback loops, created by network effects leading to non-linearities.

If you allow me to be a bit pedantic, I would like to present an analysis from physics I like very much, and it could shed a bit of light why a macroeconomic regression between PDs and LGDs vs GDP, interest rates and unemployment rate could not be the meaningful way to forecast PDs and LGDs in a stressed scenario.

The analogy is a piece of iron. A piece of iron can be modeled as many atoms of iron next to each other. Simplifying as much as possible, each atom can be in two ‘magnetic’ states, up or down. The analogy in the credit world would be that an economy can be understood as a series of ‘atoms’ (households and / or corporations) that can be in two states: default or non-default.

Things start to become more interesting when a magnetic field is applied to the piece of iron. The magnetic field makes that most atoms take a preferred direction, say 95% of them are in the up state (and 5% in the down state). The analogy is that in an economy, say 95% of households and corporations are in a non-default state, and about 5% are in default.

This magnetic field, in the case of the economy, would be the result of the regression: given an array of macroeconomic data, say GDP, interest rates and unemployment rate, PD is given, and as such, the percentage of households and corporations in default is fixed.

In fact, if one uses a well-known model from statistical physics under this same situation that most physics undergraduates learn to compute, the final function is the same as one of the formulas for PD for credit risk in Basel II.

Until now, we have not done anything new: we have just renamed things from our framework (credit risk) into a new framework (the physics of solid state). However, a solid state physicist would soon point out that this model is not interesting at all, and that interesting models include not only magnetic fields, but most importantly, network effects among the iron atoms.

This is easy to understand in an economic environment: a company may have a probability of default given the economic environment, ie if the economy is going great, the probability of default is very low. Instead, if the economy is tanking, the probability that it defaults increases. However, companies depend on other variables than macroeconomic ones: even though the economy may be running at full speed, if an important provider defaults (for any reason) the company may be on the verge of bankruptcy.

And in the same way, even though the economy is in recession, if a given corporation receives a new important contract (or equivalently, an important competitor defaults) its probability of default decreases.

As a consequence, a default in a given company can lead to a domino effect for other corporations that are linked through a supplier chain with the first one.

My intuition here is banks are the most important nodes in an economy. Banks have very strong links with many corporations, many more links than a usual corporation has with other counterparts, and more intense at this. If a bank defaults (or stops providing financing) these effects spread out to many of its clients. At the same time, if corporations fail to get new financing, this may affect other corporations, which will have an influence on other banks, and so on.

This kind of strong chain of relationships is well-known among condensed matter physicists: the Ising model (and variants). The Ising model includes not only a magnetic field (a macroeconomic variable that affects all the agents in the economy) but also direct relationships among the agents. To describe the overall behaviour of the Ising model is out of the scope of this note, but just let me state than once one includes direct relationships among the agents, the simple behaviour of the model under a magnetic field changes dramatically.

Under only a magnetic field (and no direct relationships) the behaviour is simple and predictable. As we have discussed above, undergraduates in physics may compute that relationship easily. However, once one includes direct relationships, everything changes. The relationship between the variables (ie, the dependence of PD or LGD vs the macroeconomic variables) stops being a linear one, and they become highly non-linear. In particular, there may have phase transitions: a small change in a macroeconomic variable may lead of an abrupt change in PD. This is not unlike to the case when a small change in temperature leads to water to freeze.

A study on these lines, that relates these physics models with credit portfolio models, was analyzed by Eduard Vives, a physics professor and myself, and can be found here. I do not argue this is the last word on the relationships between PD and macroeconomic variables, but apart from the appealing theoretical framework around it, one can see that it introduces the possibility to deal with outliers, and to try and understand how is it possible that loss rates in the US subprime and the Irish commercial real estate were so large, when no macroeconomic model could have forecasted such jumps.

The final suggestion, using the physics analogy, is the following: once network effects are present (mainly through the strong relationship lender – borrower) the traditional macroeconomic relationship between PDs and macroeconomic variables breaks down, and a more complex dynamics is required to model such a new reality.

To sum up:

a macroeconomic model like the ECB one relating PDs and LGDs with macroeconomic variables will never be able to model big jumps in PDs and LGDs, as they have occurred in the past with the US subprime or the Irish commercial real estate experience. These models are too slow to accommodate abrupt regime changes. However, there is a series of models, coming from condensed matter physics, that have the potential to accommodate such changes, and that comprise the regression macroeconomic models as particular cases.

This is essential due to the fact that recapitalizations coming from the stress tests, even though very small (€3.5bn) are the difference of two very big numbers. For example, the Bank of Spain has computed recapitalization needs as (simplifying) the difference between gross impairments and available resources. Available resources are more or less well-known (provisions, both specific and general, net operating income, capital gains, tax impact …) but gross impairments are highly dependent on the PDs and LGDs chosen to perform the exercise.

For example, the most toxic exposure in Spain, developer loans, assumes in the BoS calculations a 17% haircut. However, other estimations, like Luis Garicano’s one, with a PD of 70% and LGD of 70% for that portfolio (as such, total losses around 50%), would immediately increase (without taking into account any other changes in other parts of the loan book, that could also add to the recapitalization needs) the recapitalization easily to the €100-€150bn mark.

As such, this strong dependency of gross impairments on the estimated PDs and LGDs in the adverse stress scenario leads us to suggest that a better model than a simple macroeconomic regression like the one used by CEBS should be entertained, especially models that could accommodate outliers like the US subprime or Irish commercial real estate ones. But for that, one would need non-linear models that could explain and predict abrupt regime changes.

Edward Hugh also blogs at A Fistful of Euros and Global Economy Matters.