Moody's Corporation · NYSE:MCO
Moody's — Moody's Analytics — Webinar: The Changing Landscape of Model Risk Management (March 2019)
28 slides from Moody's's conference presentation, Mar 26, 2019
Slide 1: Thank you for joining us: Changing Landscape of Model Risk Management
Thank you for joining us. The Changing Landscape of Model Risk Management webinar will begin at 10:00am PT | 1:00pm ET.
Slide 2: Webinar: The Changing Landscape of Model Risk Management
Webinar: The Changing Landscape of Model Risk Management March 2019
Slide 3: Speakers: Mark Feeley, Anamaria Pieschacon, Michael J. Denton
Mark Feeley Research Director, Chartis Mark has over 30 years’ experience in global capital markets, consulting and associated technologies, focusing on risk management, front- and middle-office platforms and data management. He brings to Chartis a holistic view of business, technology and regulator
Slide 4: Agenda
Agenda 1. Model validation primer and outlook 2. Changing landscape of validation 3. Key challenges - Retail 4. Key challenges - Commercial 5. Q&A Webinar: Changing Landscape of Validation – 3/26/19 4
Slide 5: Model Validation: Output, AI, Problems. Oh my!
1 Model Validation: Output, AI, Problems. Oh my!
Slide 6: Model Validation Trends and Issues
Model Validation Trends and Issues March 26th 2019
Slide 7: What is Model Validation
What is Model Validation The historical view of model validation has been focused largely on the calculation or pricing part of model usage Model validation is broader than this and must encompass the whole process of model governance, management, usage and environment This view means that not just
Slide 8: Types of Output Validation
Types of Output Validation Two Core Areas Calculation Validity » Can we confirm the value » Are results consistent across versions or differences as expected » Does the model converge on the right value if applicable Performance and Suitability » Does the model align to benchmark or market observati
Slide 9: What about AI
What about AI AI brings two core challenges (be it ML or “true AI”) Explainability » Current levels of AI (level 1 or 2) frequently suffer from “black box” concerns » Both management and regulators/supervisors need to be able to interpret and explain the results from an AI model or process which is
Slide 10: Bear in Mind....
Bear in Mind…. Market Events Data Bias Characteristics of Success LIBOR replacement • Data selection bias can occur • Comprehensive Approach • Lack of historical data when the source data is not • Unknown behavior properly randomized; it might • Transparency and reflect historical bias, or be Explai
Slide 11: Changing Landscape
2 Changing Landscape
Slide 12: Changing Landscape of Validation
Changing Landscape of Validation We provide validation and advisory services for all aspects of model risk management Loan Lifecycle Regulatory Capital & Financial Management Models Stress Testing Models Reporting Application, Pricing, Origination, Monitoring, Loss Mitigation, Disposition Basel, CCA
Slide 13: Changing Landscape of Validation: why it is important
Changing Landscape of Validation Our take on validation: why it is important Model Risk Management Committee Development Validation (MRMC) Creates models through Control for model risk arising Final arbiters for model risk issues design, estimation and testing from model development Implementation V
Slide 14: Moody's Flexible Approach
Moody’s Flexible Approach We accommodate to a wide range of client needs and portfolio sizes Validation service offering is designed to match the model complexity and state of model risk management program Emerging Advanced State of Model Risk Management Program Model Complexity Expert Judgment Econ
Slide 15: Our Vision for Model Risk Management
Our Vision for Model Risk Management Combine Technology with Expertise to Provide the Best Solutions for our Clients Models Technology Data Monitoring Advisory Governance Webinar: Changing Landscape of Validation – 3/26/19 15
Slide 16: Key Challenges - Retail
3 Key Challenges - Retail
Slide 17: Key Challenges
Key Challenges Scorecard Models Machine Learning Stress Testing Target Variable Definition & Classification Interpretability Data Availability Variable / Feature Selection Non-linearities Peer Comparisons External Validity Data Demands Nature of Next Severe Shock Webinar: Changing Landscape of Valid
Slide 18: CECL Model Validation
CECL Model Validation Frequent questions » How am I incorporating forward-looking information? » How do I determine and defend reasonable and supportable horizon? » How do I determine the lifetime of the loan? Webinar: Changing Landscape of Validation – 3/26/19 18
Slide 19: Key Challenges: IFRS 9
Key Challenges: IFRS 9 Macroeconomic Scenario Forecast Minimum 3 scenarios (baseline, upside & downside) Scenario Probability Weights Macro-dependent, Point-in-Time, Loss Given Exposure at Expected Credit Forward-Looking Probability of X Default X Default X Discount Factor = Loss Default Credit Stag
Slide 20: Key Challenges - Commercial
4 Key Challenges - Commercial
Slide 21: Commercial Banks Risk Modeling has Matured
Commercial Banks Risk Modeling has Matured Model Risk Vendor Models Bespoke Models Specialty Management Portfolios » Critical validation needs now reflect a more mature commercial banking sector, focused on efficiency and leveraging results » Requirements have become more complex with CECL, Basil II
Slide 22: Credit Risk Model Validation and Testing
Credit Risk Model Validation and Testing EDF Back-testing results (Commercial Real Estate example) EDF Performance by DSCR Actual and Predicted Default Rate by Year Results by Property Type 14% 4,000 Ob ser vation s EDF Default ra te 6% 10% 12% 3,500 EDF Default rate EDF Default rate 3,000 5% 8% 10%
Slide 23: Market Risk Model Validation and Testing
Market Risk Model Validation and Testing Impact of market movements and credit degradation on asset values » Trading Book is in-scope, in addition to the Banking Book, under FRTB guidance » Revised Standardized Approach (SA) is conservative, but less complex than the Internal Model Approach (IMA) »
Slide 24: Model Validation: Engagement Process
Model Validation: Engagement Process Preliminary Qualitative Quantitative 01 Model Review 02 Validation 03 Validation 04 Consolidation Model document Review Model Governance Review and verify additional Implementation testing review/understanding » Purpose, scope, materiality analysis submitted by m
Slide 25: Validation Reports Reflect Standard Processes, Content, and Formats
Validation Reports Reflect Standard Processes, Content, and Formats Typical IFRS 9 validation reporting example Broad Report Sections Subsections Detailed findings by Subsection »Scope » Supports regulatory filings Introduction »Executive summary »Key Personnel » Replace/support Internal Audit »Mode
Slide 26: Slide 26
Slide 27: moodysanalytics.com
Michael Denton, Anamaria Pieschacon, Mark Feeley, Director, Senior Economist, Research Director at Chartis, Michael.Denton@moodys.com Anamaria.Pieschacon@moodys.com mark@chartis-research.com moodysanalytics.com
Slide 28: Webinar: Changing Landscape of Validation – 3/26/19
© 2019 Moody’s Corporation, Moody’s Investors Service, Inc., Moody’s Analytics, Inc. and/or their licensors and affiliates (collectively, “MOODY’S”). All To the extent permitted by law, MOODY’S and its directors, officers, employees, agents, representatives, licensors and suppliers disclaim liabilit
Moody's Analytics — Webinar: The Changing Landscape of Model Risk Management (March 2019) on 6ix