Model Validation

A model validator independently evaluates the underlying model and identifies model risk issues. A model validator raises findings corresponding to the model risk issues. This post outlines a high-level framework for the identification of model risk issues. For providing practical insights into the high-level framework, take an example of an EAD (Exposure at Default) model meant for the CCAR (Comprehensive Capital Analysis and Review) purposes. In other words, the objective of the illustrative EAD model is to project the EAD under baseline and severely adverse scenarios for the next nine quarters from a given jump-off date. Further, assume that one of the model segments having 5% of the portfolio size has the only macroeconomic variable in the model as statistically insignificant with a p-value of 75%. Why a model risk issue is an issue in the first place? On this point, a model validator must think through holistically. For the illustrative EAD model, using a model with the only macro...

In this post, I will not write about technical concepts on model development or model validation per se. I would write about  elementary aspects  that most folks overlook while writing technical reports, e.g., model validation reports. I have seen even seasoned professionals ignoring these aspects! If you are writing a validation report, try to be in the shoes of someone who relies on the validation report to understand the model and corresponding model risk issues. One might have spent weeks understanding the model and model risk issues. As such, while writing, you are more likely to  presume  that a reader will know it all. Don't err on that side.  Agree on a writing style , e.g., using passive voice, present perfect tense, etc., and use it consistently throughout the report. Agree on terminologies and use them consistently throughout the report. For example, do you want to write model validation, model validati...