Podcast Transcript - Moving forward with Backtesting: Unravelling European requirements for IM model validation

Podcast Transcript - listen to the podcast here.

Moving forward with Backtesting: Unravelling European requirements for IM model validation

‍

Jon Cronin (JC)

Hello and welcome to Ahead of the Curve, the podcast fromAcadia, where we take time to get under the skin of the risk margin and collateral industry to dig deep and present topical perspectives and insights on this hugely important sector.

Now, for some time, we've been waiting for European Union regulators to update and further specify the requirements for uncleared margin rules or UMR regulations for firms across Europe in particular, with respect to the initial margin model validation requirements. With UMR coming into effect for Phase 6 firms this September, it's been seen as a particularly important moment for those smaller Phase 5 and 6 firms. Well, the European Banking Authority or EBA, has now published for consultation its proposed specification of existing regulation in the form of its latest regulatory technical standards, or RTS, on initial margin model validation. The update provides clarity on a number of key issues, in particular, the way firms will need to validate the model they use to calculate their initial margin.

So, what do Phase 5and 6 firms need to be thinking about here? And how should they be preparing?To answer these and other questions on the EBA latest announcements I'm delighted to be joined by Robert Kirchner Partner and Head of European Quant Services at Acadia, and Deepak Sitlani Partner at Linklaters.

So, let's get back to basics. First, talk me through the EBA new regulatory technical standards for uncleared margin rules. Robert, let me come to you first.

Robert Kirchner  (RK)

 The RTS on initial margin model validation is the latest proposal in a larger body of regulation that is concerned with the risk mitigation of OTC derivatives.  And relevant for this RTS, is EMIR the European market infrastructure regulation, which is the umbrella regulation.

So, among many other things, those regulations say with respect to model validation, that internal initial margin models, so non grid methodologies, need to be validated at least annually ,and in particular backtested.

However, this existing body of regulation is not very prescriptive about how the validation and the backtesting should take place, and what kind of documentation should be submitted to the regulators for an initial model approval. So, the RTS is a long-awaited specification on how the validation and in particular the backtesting should or can take place, and what a firm that is regulated under EMIR will be required to submit in its application for model approval.

Now, to be clear, this applies to firms who fall under EMIR and are using or are planning to use an initial margin model such as ISDA SIMM. In the RTS, there are two processes defined for model validation depending on your average aggregate notional amount (AANA). For the smaller ones for Phase 5 and Phase 6 firms that have come into scope or will come into scope in September, there's a simplified process for validation of the initial margin model. Now, and under this simplified process, firms will need to demonstrate that they have a proper risk governance and operational structure in place within which they are monitoring the performance or the adequacy of the operations of the model. And the key technical or quantitative requirements for performance monitoring for those firms is that they have to perform a quarterly dynamic backtest.

Now, to ease the operational and technical burden for all firms, the RTS explicitly allows for the outsourcing of virtually all key tasks such as the design, the calibration, the implementation, and also the internal validation of the model. Now, that said, however, an important point is that the regulated entity is still responsible for the adequacy of the model for its portfolio. So that's very briefly, but in a nutshell, what the RTS says and what the context is.

JC

You've painted the picture well there Robert, Deepak anything that you would add to that?

Deepak Sitlani (DS)

So, I think you're right Robert has done a good job of explaining all the different pieces of the jigsaw as to what makes up the EU margin rules. But this latest piece of the jigsaw, the RTS, I think it's worth flagging that it's not yet live. So, where we are in the process is the EBA putout a consultation, and draft RTS, that consultation closed on the 4th February2022. And the next step is just a process pursuant to which comments will be addressed. And I do know that substantive comments were fed in. So, the draft is by no means final. When you fast forward, I think the expectation is that we would see a publication later this year, you then have to go through all of the EU approvals process. So, by the time this comes into force, it will definitely be after the Phase 6 date of September 2022. And even after that there's a phased application of these rules.

So currently, for Phase 5 entities, you've got to get your approvals in two years after the RTS come into force. And for Phase 6 firms three years after the RTS come into force. And then the local regulators have two years in which to object to the models used. So, you can see that actually we're looking quite a way into the future. But I think it is important to be talking about these draft RTS now. Because the temptation is just to worry about the here and now and kind of kick the can down the road in terms of things to worry about for another day. But, the decisions and steps that people take in setting up their initial margin infrastructure today, I think, are really important to actually just factor in what may be coming down the track.

JC

So despite the perhaps slightly longer time horizon, Deepak there is still a sense of urgency here.

DS

I think it's a case of don't ignore these draft rules, just because they are draft and because there are these long transitional provisions, because now is the time particularly for those Phase 6 firms that they're establishing their setup, and it would just be a bit of a pain for them to have to change things in a few years time and possibly revise the approach that they've taken.

JC

Robert, will the requirements outlined in the RTS largely remain or can firms expect a higher degree of relief?

RK

The short answer is probably not much.  Even though many, especially the smaller firms that are very unhappy with what seemed to be quite a burdensome set of requirements here. Particularly the backtesting, and they're also looking over to the US where smaller firms,  that are not prudentially regulated, are exempt from individual backtesting and monitoring requirements.  But we don't see any indication that a substantial relief will come here, particularly with respect to the backtesting, so that is most probably here to stay with us. Maybe something on the documentation here and there. But setting up a good governance infrastructure, and then the technical part of the backtesting that that will probably remain in our estimation.

JC

So what regulatory requirement is in place now when it comes to initial margin model validation - Deepak, can you answer that?

DS

It's a good question, actually. And it's a point that I think sometimes people gloss over in a way so the rules currently do contemplate a number of requirements or criteria that your initial margin models have to satisfy including for example, backtesting. But it's a bit like when you're at school, when you've been asked to do your homework, and then in class you're basically allowed to mark your own homework. Whereas I think once these RTS come into force, you actually have to give your homework into the teacher to mark. And so, what I think is likely to happen is that I think firms will just pay a bit more attention to the requirements that are currently there, including setting out in their risk management procedures, all the steps that they would take in relation to their compliance of the model with their rules.

JC

And Deepak you've already talked about the impact on firms in the midst of Phase 5 and Phase 6, you mentioned, September later this year. But thinking about in particular Phase 5 and Phase 6 firms, what do they need to be doing now or thinking about now?

DS

Again, I think with an eye on what these drafts RTS say, I would be thinking about going through the draft RTS and asking yourself how are you going to demonstrate if you were to put in a request validation today, how would you put together the supporting information for that submission? And, equally, what kind of governance processes are you going to put around your operation of the model, in particular, if you are, as a number of firms will be doing outsourcing the models? So, what are you going to require of the entity to whom for example, you're outsourcing your model requirements. So those are the kinds of things that I think firms should be thinking about, possibly in a way that they haven't kind of thought through for prior phases.

JC

And, Robert, what would you add to that?

RK

If you're moving margin already or you will move margin within the next 12 to 15 months, there is the requirement right now to validate and to backtest your model. You have the RTS that specifies how you might do that so you might as well get your set-up right, consistent with the RTS, start doing that now. And like Deepak said, get your monitoring and internal approval processes up and running, define and delineate staff roles and responsibilities, get your technical implementation work that is needed for running and back testing the model, get that ready, and start to compile the documentation that is needed for model application.

And there's also a nice twist that you can make use of in the in the RTS, maybe another point of relief here with respect to other internal model approval processes. So, in the RTS, it is stated that as soon as you have submitted the application to your local regulator for using your IM model, you can start to use the model and daily operations. You do not need to wait for explicit model approval. So set everything up. And if you're ready, in your estimation, that you're fit for purpose, compile your documents, submit the application, and you're good to go.

JC

Just looking more broadly, the new RTS relates only to Europe in what is of course, a global industry. So why is this the case? And is there any potential impact for firms based in the UK? Deepak? What are your thoughts on this?

DS

So I think when the initial margin rules first came in to play, obviously, the aim was to have as consistent a set of rules across the various jurisdictions as possible. But in a number of respects, that that wasn't achieved. And this is one of those areas. So, for example, in the US, there was a requirement for model approval, but we didn't have that in the EU. So basically, this is, I guess, belatedly the EU kind of catching upon the requirement for regulators to approve models. But then you also have the interesting interaction with Brexit so when the UK left the European Union basically what happened was there was a kind of copy and paste, if you like, of European law into UK law. Now that copy and paste doesn't apply to these draft regulatory technical standards because they didn't exist at the time that the UK left the EU. So that then leaves open the question as to whether the UK will independently be introducing similar requirements. And at the moment, I haven't heard that this will be happening in the UK. That's not to say it won't. But that means that these rules currently will only apply to EU entities rather than rather than UK entities.

JC

So in respect to the UK then Deepak it's more a question of watch this space.

DS

Definitely, I think there's a lot of watch this space, watch the space how the RTS develop. Let's wait and see if UK regulators adopt a similar approach.

JC

Now, Robert, you mentioned earlier the backtest is key to much of what we're discussing here. So, what should a firm do if the worst happens, and the back test fails?

RK

Backtesting is a statistical procedure. So the backtest will fail at some point almost surely. And that's to be expected. However, what's important is that it does not fail significantly or too often, when this level is reached, it's specified in the in the RTS. And if that happens, if that level is reached, the backtest is basically telling you that the initial margin calculated by the model used does not cover the realized P&l moves with a confidence level that the model has been designed for originally. Simply put, the model seems to fail significantly in predicting your exposure. Now, in case of a failing backtest, or instances in which the initial margin does not cover the P&l moves, what firms need to analyze is the root cause of this failure, and they need to explain that to the regulator.

It could, for example, be the case that the model does not cover a risk factor that the portfolio is exposed to, or that certain assumptions that have been made in the model methodology or approximations, they might not be valid. And if you're using a proprietary model, then you might want to recalibrate the model or even revise some of its structure to fit the risk profile of your portfolio better. If you're using a third-party solution, like SIMM for example, then individual revision of the model might not be possible. However, you can still solve the problem, for example, by using a multiplier to your initial margin at the level appropriate for the exceptions encountered. For example, the aggregate portfolio level, you can do an asset specific, per risk class or risk factor level or any combination of that. Now, if a risk factor is not covered by SIMM or not sufficiently covered, or your third-party model, you could also use a fixed add on for example, or you could fall back to standard schedule for certain parts of your portfolio. So there are different possibilities or different ways, different routes that you can take, whatever you do, however, all these remediation actions they need to be agreed by both counterparties and they need to be specified in the CSA, to avoid any conflicts in the reconciliation.

JC

Deepak anything you'd add to that.

DS

So just to say that, you know, we’re talking about the new or the forthcoming RTS, which relates to regulator validation. As I mentioned that the current rules already contemplate things that you have to do to deal with the model. And one of the things is to have risk management procedures in place that basically upfront say, what you might do to address a backtesting failure so that the rules contemplate a model change recalibration or other remediation actions. So again, it's documenting ahead of time what you might do if there is a failure.

RK

Deepak that's a that's a good point. This is one of the most important points that you will need to cover in your documentation and in your internal processes, that you prescribe precisely what you do. How do you react to a backtest failure?

JC

Just time for one more question to you both, now that we have at least some clarity from the EBA on its new regulatory technical standards, what are the next steps for those firms that will be affected by the new RTS, Deepak let me come to you first for a round up on this.

DS

So one thing I would say is watch this space. So do keep an eye on how the draft RTS evolve. And I guess the only other thing I would say is almost pretend that they are enforced before they apply to you - before they actually do so that you are laying the groundwork in terms of model adoption, governance, outsourcing, at the right time, rather than have to come back and revisit in a few years’ time, so it goes back to that don't kick the can down the road line.

JC

I think it's important thing to remember that. And, Robert, the final comment to you.

RK

I think that it is especially important for firms that are moving margin already using a model or are planning to that in the next 12 months or so. And even if you're not planning to do that within the next 12 or 15 months, then communicate with your regulator. Tell them about your plans, tell them how you think you will get ready when the time comes that you move margin, how you get your approvals or your internal structure and so on and so forth. And it's especially if you're a smaller buy side firm then you should think about this because chances are you do not have a risk governance infrastructure in place like most banks do that are used to these levels of regulations. Get an overview of what you need to do and start doing it because to set these things up, that's not something that you do in a couple of weeks - that takes time.

JC

Well, it's a fascinating and important subject, but I'm afraid we are out of time now. Robert Kirchner, Deepak Sitlani, thank you both very much indeed for your time. And thank you for listening to Ahead of the Curve. We'd like to know what you think so please do get in touch and share your thoughts. And you can find out more about Acadia by going to acadia.inc but until next time, it's goodbye.

‍