Measuring the model risk-adjusted performance of machine learning algorithms in credit default prediction

作     者：Andrés Alonso Robisco JoséManuel CarbóMartínez 

作者机构：Banco de EspañaCalle de Alcalá4828014 MadridSpain 

出 版 物：《Financial Innovation》 (金融创新（英文）)

年 卷 期：2022年第8卷第1期

页      面：1930-1964页

核心收录：

学科分类：02[经济学] 0202[经济学-应用经济学] 1202[管理学-工商管理] 0701[理学-数学] 

基　　金：Carolina Toloba and Arancha Gutiérrez Supervision Department IV of Banco de España 

主　　题：Artificial intelligence Machine learning Credit risk Interpretability Bias Internal ratings based model IRB model Natural language processing NLP 

摘      要：Implementing new machine learning(ML)algorithms for credit default prediction is associated with better predictive performance;however,it also generates new model risks,particularly concerning the supervisory validation process.Recent industry surveys often mention that uncertainty about how supervisors might assess these risks could be a barrier to innovation.In this study,we propose a new framework to quantify model risk-adjustments to compare the performance of several ML methods.To address this challenge,we first harness the internal ratings-based approach to identify up to 13 risk components that we classify into 3 main categories—statistics,technology,and market conduct.Second,to evaluate the importance of each risk category,we collect a series of regulatory documents related to three potential use cases—regulatory capital,credit scoring,or provisioning—and we compute the weight of each category according to the intensity of their mentions,using natural language processing and a risk terminology based on expert knowledge.Finally,we test our framework using popular ML models in credit risk,and a publicly available database,to quantify some proxies of a subset of risk factors that we deem representative.We measure the statistical risk according to the number of hyperparameters and the stability of the predictions.The technological risk is assessed through the transparency of the algorithm and the latency of the ML training method,while the market conduct risk is quantified by the time it takes to run a post hoc technique(SHapley Additive exPlanations)to interpret the output.