Credit risk assessment is the heartbeat of retail banking. This project involved building, validating, and deploying a robust scoring engine that could handle diverse financial datasets while remaining compliant with local regulations.
Problem: High Default Rates
One of our partner institutions was seeing higher-than-average default rates (15%+) due to an over-reliance on traditional, rule-based scoring models. These models were:
- Inflexible: They couldn't weigh non-traditional data (e.g., utility payments).
- Outdated: They didn't adapt to changing macroeconomic conditions quickly enough.
- Biased: Certain demographic segments were unfairly penalized by static rules.
Solution: Gradient Boosted Governance
I developed an ensemble-based scoring model using LightGBM, optimized with Hyperopt for maximum precision.
Engineering Insight: The Explainability Trap
[!IMPORTANT] In financial services, a "Black Box" model is a liability. While deep learning might offer higher accuracy, I chose Gradient Boosting (LightGBM) because it provides a perfect balance of performance and auditability when paired with SHAP values.
Technical Highlights
- Large-Scale Processing: Used Apache Spark to engineer 200+ features from millions of raw transaction records. We implemented a Feature Store pattern to ensure training/serving consistency.
- Model Governance: Implemented SHAP (SHapley Additive exPlanations) values for model explainability. This ensured that every credit rejection could be legally audited and explained to both regulators and customers.
- Robust MLOps: Deployed a monitoring layer that tracks Feature Drift and PSI (Population Stability Index) to trigger automated alerting when the incoming applicant distribution shifts significantly from the training set.
Regulatory & Ethical AI
To meet strict banking regulations, I implemented a custom bias-detection layer that audited the model for "Disparate Impact." We discovered that certain address-based features were acting as proxies for protected classes; by removing these and using neutral behavioral data, we improved fairness without sacrificing the Gini coefficient.
Impact: Financial Stability
Across the 12+ institutions where the system was deployed, we observed:
- +20% increase in Gini coefficient (predictive power) compared to baseline logistic regression.
- 12% reduction in 90-day default rates (NPL), directly improving the bank's capital reserves.
- Automation Excellence: Reduced the time-to-decision for credit applications by 45%, moving from days to minutes.