Monitor a credit model¶

A score shift does not, by itself, tell you whether outcomes worsened. Imagine a lender whose model was trained on low-risk applicants. A new partner channel or a changing market brings in riskier applicants. The model keeps scoring applications. A rise in predicted risk could mean the pool is riskier, the model performs worse on it, or both.

The dataset¶

HELOC means home equity line of credit: a revolving credit line secured by the borrower's home. This FICO Explainable AI Challenge dataset contains anonymized credit-bureau features. It also records whether an account went 90 days past due, or worse, in its first 24 months.

The model estimates each applicant's probability of that outcome (default probability). A group's mean predicted probability is its model-implied bad rate. The example loads 9,871 applications from OpenML (data_id=45554) with fetch_openml.

The split¶

We split applications at 63 on ExternalRiskEstimate, following Gardner et al. (2023). Higher values indicate safer applicants. Here, source is the lower-risk reference group used to train the model; target is a simulated deployment of higher-risk applicants:

Source: 7,683 applications above 63, the low-risk book, with a 43.5% observed bad rate.
Target: 2,188 applications at or below 63, the riskier book, with an 81.9% observed bad rate.

The code below creates these source and target samples:

import re

import pandas as pd
from sklearn.datasets import fetch_openml

fico = fetch_openml(data_id=45554, as_frame=True)
X, y = fico.data, fico.target

re_obj = re.compile(r"external.*risk.*estimate", flags=re.IGNORECASE)
col_split = next((c for c in X.columns if re_obj.search(c)), None)
mask_high = X[col_split].astype(float) > 63

X_train = X[mask_high].reset_index(drop=True)
y_train = y[mask_high].reset_index(drop=True)
X_deployment = X[~mask_high].reset_index(drop=True)
y_deployment = y[~mask_high].reset_index(drop=True)

Scores¶

With source and target defined, compare three scores. They are not interchangeable: each answers a different question and becomes available at a different time.

Score	Requires labels?	Harmful direction	`worse`	When it helps
Predicted risk	No	Higher risk	`higher`	The output itself represents harm
Confidence (`LogitGap`)	No	Lower certainty	`lower`	Early warning before labels arrive
Prediction error (Brier)	Yes	Larger error	`higher`	Post-outcome accuracy check

Use predicted risk when the model output already represents harm, confidence for an early warning while labels are delayed, and prediction error for the clearest post-outcome check.

The score examples build on the same model and out-of-sample predictions:

import numpy as np
import samesame as ss

from sklearn.ensemble import RandomForestClassifier

y_train_binary = y_train.map({"Good": 0, "Bad": 1}).astype(int).values

rf_bad = RandomForestClassifier(
    n_estimators=500,
    oob_score=True,
    random_state=12345,
    min_samples_leaf=10,
)
rf_bad.fit(X_train, y_train_binary)

train_risk = rf_bad.oob_decision_function_[:, 1].ravel()
deployment_risk = rf_bad.predict_proba(X_deployment)[:, 1].ravel()

Risk, no labels neededConfidence score, no labels neededPrediction error, labels needed

Start with predicted risk when the model output already represents harm. P(default) is directly tied to the outcome: larger means more harm, so use worse="higher".

harm = ss.test_harm(
    source=train_risk, target=deployment_risk,
    worse="higher", rng=np.random.default_rng(12345),
)
print(f"Harm  p-value: {harm.pvalue:.4f}")   # → 0.0001

The test finds evidence of movement toward higher risk. Mean predicted risk—the model-implied bad rate—rises from about 44% in source to 73% in target.

LogitGap (Liang et al., 2025) measures how clearly the model favors one class over the other. A large gap means high confidence; a small gap means the model is undecided. Lower confidence is harmful here, so we use worse="lower".

import numpy as np
from scipy.special import logit
def logit_gap(logits: NDArray[np.float64]) -> NDArray[np.float64]:
    logits = np.asarray(logits, dtype=float)
    max_logits = np.max(logits, axis=1)
    mean_rest = (np.sum(logits, axis=1) - max_logits) / (logits.shape[1] - 1)
    return max_logits - mean_rest


def confidence_scores_from_probabilities(
    probabilities: NDArray[np.float64],
    *,
    clip: float = 1e-6,
) -> NDArray[np.float64]:
    clipped = np.clip(np.asarray(probabilities, dtype=float), clip, 1.0 - clip)
    return logit_gap(logit(clipped))


train_conf = confidence_scores_from_probabilities(
    rf_bad.oob_decision_function_
)
deploy_conf = confidence_scores_from_probabilities(
    rf_bad.predict_proba(X_deployment)
)

harm = ss.test_harm(
    source=train_conf, target=deploy_conf,
    worse="lower",  # lower confidence = harm
    rng=np.random.default_rng(12345),
)
print(f"Harm  p-value: {harm.pvalue:.4f}")  # → 1.0000

Default probability and confidence answer different questions. The first asks, "How risky does the model think this applicant is?" LogitGap asks, "How sure is the model?" Target predictions are both riskier and more confident. The test finds no evidence of a harmful confidence drop. Higher confidence alone does not mean the model is safer; confidence adds a separate dimension to risk.

Once labels arrive, prediction error (Brier) gives a post-outcome check. It uses the same split and source-trained model as the other scores. Now that labels are available, we can measure each prediction's error.

y_deployment_binary = y_deployment.map({"Good": 0, "Bad": 1}).astype(int).values
brier_source = (y_train_binary - train_risk) ** 2
brier_target = (y_deployment_binary - deployment_risk) ** 2

harm = ss.test_harm(
    source=brier_source, target=brier_target,
    worse="higher", rng=np.random.default_rng(12345),
)
print(f"Brier p-value: {harm.pvalue:.4f}")  # → 1.0000

Brier score asks whether those probabilities were accurate once outcomes arrived. With p=1.0000, we do not reject the null of no harmful increase in prediction error. Risk rises on this split, but the test finds no evidence of a harmful error increase.

Wrap up¶

Predicted risk is higher in target, while these tests find no evidence of lower confidence or higher prediction error. The split deliberately selects a riskier target group; it cannot show why risk differs or prove that model quality is unchanged.

If the comparison is driven by applications in regions where the groups barely overlap, see Weight for common support.