Safetycage

Predictive models, especially neural networks, are increasingly deployed in real-world and safety-critical settings. A well-documented failure mode is that they fail silently: a model can be highly confident about a prediction and still be wrong, and that confidence does not reliably drop just because the input differs from what the model was trained on. Measuring accuracy once on a held-out set says nothing about whether any single prediction made after deployment can be trusted.

Misclassification detection addresses this at the level of individual predictions. Given a model’s output on a new input, a misclassification detector estimates whether that specific prediction is likely correct or likely wrong, without access to the true label, since none is available at deployment time. This is a different problem from out-of-distribution (OOD) detection: an input can be entirely in-distribution and still be misclassified, and in safety-critical applications catching that case is often what matters most. Predictions flagged as untrustworthy can then be routed to a human reviewer, which is exactly the kind of error-resilience and human-oversight support called for by regulation such as the EU AI Act.

safetycage collects several misclassification detection methods behind a common interface: softmax-based baselines (MSP, DOCTOR), statistical tests over hidden-layer activations (SPARDACUS, Mahalanobis), and an uncertainty-aware residual model (RED). Swapping between them only requires implementing the two adapter classes described below.

Installation

pip install safetycage
# or: uv add safetycage

Some methods need extra dependencies, installed via pip install safetycage[extra] (or uv add "safetycage[extra]"):

Extra	Uses	Adds
red	RED	torch, gpytorch
spardacus	SPARDACUS	statsmodels, scikit-learn, scipy, tqdm
mahalanobis	Mahalanobis	statsmodels, scipy
torch	TorchModelModule	torch

How it works

Adopting safetycage means writing two adapter classes for your own project: a DataModule and a ModelModule. See How it works for the minimal shape of each and a worked-through safety cage method run.

Choosing a method

Method	Needs	Works with
MSP	class probabilities	any classifier
DOCTOR	class probabilities	any classifier
SPARDACUS	layer activations	neural networks only
Mahalanobis	layer pre-activations	neural networks only
RED	layer activations	neural networks only, needs [red]

Start with MSP. It needs no fitting and works with anything that produces class probabilities.

Worked examples

The examples/ directory contains complete integrations, each with a runnable notebook rendered below. examples/01-mnist wraps a PyTorch MLP and guards it with MSP; examples/03-cifar10-cnn wraps a CNN and guards it with RED; examples/04-activity-monitor wraps a GRU trained on body-worn sensor data and compares MSP against SPARDACUS.

Examples

• Detecting misclassifications on MNIST with safetycage
• Detecting misclassifications on CIFAR-10 with a CNN
• Random baseline safetycage flags on test set
• Detecting misclassifications on sensor-based activity recognition

Contents

• How it works
  • The DataModule
  • The ModelModule
  • Common models: TorchModelModule, SklearnModelModule, AutoModelModule
  • Running a safety cage method
• API reference
  • Core
  • Data modules
  • Methods
  • Model modules
  • Utilities

Project

• Changelog