Moody's Corporation · NYSE:MCO
Moody's — Technical Presentation (May 2020)
37 slides from Moody's's technical presentation, May 20, 2020
Slide 1: Assessing Model Risk with Effective Validation
Assessing Model Risk with Effective Validation Olga Loiseau-Aslanidi , PhD Anamaria Pieschacon, PhD Petr Zemcik, PhD Director Business Analytics Director Model Validation Senior Director Economics & Business Analytics May 20, 2020
Slide 2: Moody's Analytics independence notice
Moody's Analytics operates independently of the credit ratings activities of Moody's Investors Service. We do not comment on credit ratings or potential rating changes, and no opinion or analysis you hear during this presentation can be assumed to reflect those of the ratings agency. Effective Valid
Slide 3: Presenters
Presenters Olga Loiseau-Aslanidi PhD, Director Anamaria Pieschacon PhD, Director Petr Zemcik PhD, Senior Director Head of APAC Risk Modelling Global Head of Model Validation Head of EMEA Risk and Economics Economics & Business Analytics | Singapore Economics & Business Analytics | USA Economics & Bu
Slide 4: Agenda
Agenda 1. Model Risk in Spotlight 2. Effective Model Validation 3. Application to IFRS 9 Model 4. Key Takeaways Effective Validation Webinar – May 2020 4
Slide 5: Model Risk in Spotlight
1 Model Risk in Spotlight
Slide 6: Institutions Rely on Models to Guide Decisions
Institutions Rely on Models to Guide Decisions Manage risk, identify opportunities and comply with regulation Business & Application Scorecards Scenario Generation Stress Testing Strategic Planning Regulatory Credit Policies Risk Based Limit Management and Pricing Reverse Stress Testing Reporting Or
Slide 7: COVID-19 Calls for Model Revision
COVID-19 Calls for Model Revision Mitigating model risk is a basis for effective crisis management Understand Identify Enhance Act Changes in Market Conditions Affected Models in Scope Validation and Benchmarking Portfolio Management Beware of potential model Which models I should be Assess mo
Slide 8: Credit Risk Models Are Among Most Vulnerable
Credit Risk Models Are Among Most Vulnerable Need to improve model resilience during pandemic and beyond COVID-19 Macroeconomic Impact Scenarios IRB models Dynamic models Scorecards PD | LGD | EAD IFRS9, stress testing PD | LGD | EAD Application, behavioral, transactional, alternative data Pricing E
Slide 9: Effective Model Validation
2 Effective Model Validation
Slide 10: Robust Model Governance as a Precondition for Effective Model Risk Management
Robust Model Governance as a Precondition for Effective Model Risk Management More Greater Model Significant Increased Data Amplified Models Complexity Financial Impact Availability Supervision Effective Validation Webinar – May 2020 10
Slide 11: Effective Model Validation
Effective Model Validation Managing Model Risk Involves Effective Challenge of Models Identify model Effectiveness depends on a limitations combination of incentives, competence, and influence Critical analysis by objective Effective Validation Webinar – May 2020 11
Slide 12: Depth of Validation
Depth of Validation Intensity should be proportional to the materiality of the portfolio In all cases, External Validation » MRM team should establish model performance thresholds for periodic monitoring. Materiality » MRM team should run periodic performance tests and perform formal annual validati
Slide 13: 3 Pillars for Effective Model Validation
3 Pillars for Effective Model Validation Purposeful Expertise Rigor Independence Effective Validation Webinar – May 2020 13
Slide 14: Expertise & Purposeful Rigor
Expertise & Purposeful Rigor Loan Lifecycle Management Regulatory Capital & Models Stress Testing Application, Pricing, Models Origination, Monitoring, Loss Basel, CCAR, PRA, EBA etc. Mitigation, Disposition Financial Other Advisory Reporting Services IFRS 9 and CECL Gap Analysis, Best Practices and
Slide 15: Independence
Independence » Model developers and owners should coordinate all stages of Board model lifecycle, including implementation. Board Risk Committee » Validators should provide effective challenge to existing models, based on purpose and Model Risk Committee 3rd Line materiality. » To avoid conflicts of
Slide 16: Our Validation Process
Our Validation Process Qualitative Validation Report Evaluation of conceptual soundness Assessment based on the qualitative, quantitative and benchmarking analysis Quantitative Benchmarking Replication and outcomes Comparison of inputs and analysis outputs of estimates from alternatives allows to as
Slide 17: Model Evaluation – Action Ratings
Model Evaluation – Action Ratings Satisfactory Satisfactory with Needs Improvement Unsatisfactory The model has no critical Recommendations The validators have There are important flaws in the findings and is suitable for The model’s performance is identified multiple critical model’s underlying dat
Slide 18: Final Assessment: Model Ratings by Category
Final Assessment: Model Ratings by Category Issues Identified and Recommended Actions – Generic Example Overall Rating Risk Category Rating Comments Documentation The documentation needs to include XYZ. Data Cleaning and Treatments … Variable Selection Process … Model Selection … Model Performance …
Slide 19: Our Validation Process
Our Validation Process Preliminary Qualitative Quantitative 01 Model Review 02 Validation 03 Validation 04 Consolidation Model document Evaluate Review and verify additional Replication review/understanding » Purpose, scope, materiality analysis submitted by model In-sample and out-of-sample owners
Slide 20: We Measure Model Risk by Benchmarking
We Measure Model Risk by Benchmarking Effective Validation Webinar – May 2020 20
Slide 21: Application to IFRS 9 Models
3 Application to IFRS 9 Models
Slide 22: Impairment Model
Impairment Model Macroeconomic Scenario Forecast Scenario Probability Weights Probability of Loss Given Exposure at Expected X X X Discount Factor = Default Default Default Credit Loss Survival Probability Probability of Cure Behavioral Effective/contractual Stage Component Interest Rate 1, 2 or 3 U
Slide 23: Credit Risk Models
Credit Risk Models An Integrated Process Macro- economic Scenarios Portfolio Data Macroeconomic Scenarios Results and Data Reports Credit Risk Models IRB Models / Basel Stress Testing Models Models Expected Implementation Credit Loss Model IFRS 9 Models Sensitivity Analysis IFRS 9: Macroeconomic Sce
Slide 24: Scenario Severity Shift
Scenario Severity Shift S3 Apr S3 Feb BL Apr S1 Apr BL Feb S1 Feb Source: Moody’s Analytics April 2020 24
Slide 25: PD Modelling Approaches
PD Modelling Approaches Segment level Account level Transition Matrix Approach Modelling approach with three key factors influencing vintage PD is forecasted using customer and loan characteristics, segment performance: and macroeconomic indicators using panel data 1. Segmentation econometric techni
Slide 26: Looking at Forecast Properties
Looking at Forecast Properties PD & Driver Driver PD Issue Correlation Forecasts Forecasts » Policy variables, e.g. CPI Upside Upside » Changes in past correlations Macro Driver Inconsistent PD » Non-cyclical sectors PD Downside Downside Driver Time Time » Growth rates: Downside – Low range level va
Slide 27: LGD Design Approaches
LGD Design Approaches Balance and Recoveries By Assumption For a facility i, time t and workout period w: LGD of 50-60% for PF, 30-40% for RE and 65-75% for CC; fully insured products usually get LGD of 5-10%. 𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑖𝑖,𝑡𝑡 − 𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑖𝑖,𝑡𝑡+𝑤𝑤…
Slide 28: EAD Design Approaches
EAD Design Approaches Fixed Term Products - Amortization Revolving Products - CCF Credit Limit Balance EAD Undrawn amount x CCF Drawn Amount Time 𝐸𝐸𝐸𝐸𝐷𝐷𝑖𝑖,𝑡𝑡+ℎ = 𝐵𝐵𝐵𝐵𝐿𝐿𝑖𝑖,𝑡𝑡+ℎ + 𝐶𝐶𝐶𝐶𝐶𝐶 ∗ 𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑈𝑁𝑁𝑖𝑖,𝑡𝑡 Effective Validation Webinar – May 2020 28
Slide 29: Evaluation of SICR
Evaluation of SICR Quantitative Approach .75 Characteristics of the metric: .7 » Forward-looking (scenarios) Accuracy Ratio .65 » Capture risk of default .6 » Lifetime information .55 » Available at origination and at reporting date .5 0 .61 1 2 3 Buffer Size (logit) What is the optimal d to identif
Slide 30: ECL Calculation
ECL Calculation ECL by scenario (s) & instrument (i): 𝐸𝐸𝐸𝐸𝐸𝐸 𝑖𝑖 𝑠𝑠 = � 𝑃𝑃𝑃𝑃 𝑖𝑖, 𝑡𝑡 𝑠𝑠 ∗ 𝐿𝐿𝐿𝐿𝐿𝐿 𝑖𝑖, 𝑡𝑡 𝑠𝑠 ∗ 𝐸𝐸𝐸𝐸𝐸𝐸 𝑖𝑖, 𝑡𝑡 𝑠𝑠 ∗ 𝐷𝐷𝐷𝐷 𝑖𝑖, 𝑡𝑡 𝑠𝑠 𝒕𝒕 Probability-Weighted ECL by instrument: 𝐸𝐸𝐸𝐸𝐸𝐸 𝑖𝑖 = 𝑝𝑝1 𝐸𝐸𝐸𝐸𝐸𝐸 𝑖𝑖, 𝑠𝑠1 + 𝑝𝑝2…
Slide 31: IFRS 9 Validation Process
IFRS 9 Validation Process Robustness & Sensitivity Analysis Report MA Baseline 8 Portfolio Behavior to Changing Final Qualitative Quantitative 6 Macroeconomic Conditions Assessment 4 0.1 0.09 2 0.08 Methodology Data analysis Written report 0.07 0.06 Data use, Model Observation, descripti
Slide 32: IFRS 9 Case Study – Impact of COVID-19
IFRS 9 Case Study – Impact of COVID-19 Baseline Feb 2020 Baseline Apr 2020 IFRS 9 Stage # % Exposure ECL=0.03 IFRS 9 Stage # % Exposure ECL=0.05 1 92,090 99.00 8,275,327,246 0.00 1 92,047 98.90 8,266,730,875 0.01 2 717 0.83 69,352,356 0.89 2 760 0.93 77,948,726 1.68 3 146 0.17 13,986,747 12.21 3 146
Slide 33: Key Takeaways
Al Key Takeaways
Slide 34: Proactive Overhaul of Model Risk Management
Proactive Overhaul of Model Risk Management Understand Identify Enhance Act Changes in Market Conditions Affected Models in Scope Validation and Benchmarking Portfolio Management Effective Validation Webinar – May 2020 34
Slide 35: Thank You
Thank You Questions? Contact us at help@economy.com Effective Validation Webinar – May 2020 35
Slide 36: moodysanalytics.com
Contact Us: Content Solutions - Economics & Business Analytics Offices West Chester, EBA-HQ London Prague Singapore +1.610.235.5299 +44.20.7772.5454 +420.23.474.7500 +65.6511.4400 121 North Walnut Street, Suite 500 One Canada Square Pernerova 691/42 6 Shenton Way West Chester PA 19380 Canary Wharf 1
Slide 37: Effective Validation Webinar – May 2020
© 2020 Moody’s Corporation, Moody’s Investors Service, Inc., Moody’s Analytics, Inc. and/or their licensors and affiliates (collectively, “MOODY’S”). All To the extent permitted by law, MOODY’S and its directors, officers, employees, agents, representatives, licensors and suppliers disclaim liabilit