Trusting AI isn't one score.
It's a journey of questions.

A model can be 99.9% accurate and catch zero fraud. Real trust is earned one question at a time — from framing the problem to keeping it honest long after it ships.

Question assumptions. Build intuition. Create impact.

Read the manifesto → Subscribe to the newsletter

The Journey Toward Trustworthy AI — a hand-drawn map of the ten questions every model must survive. — The Journey Toward Trustworthy AI — every stop is a question a model must survive before humans should rely on it.

Stop asking what a metric means. Start asking where it belongs.

Every metric — accuracy, F1, AUROC, calibration — exists because someone asked a question the last one couldn't answer. Learn the questions, and the metrics finally have a home.

Read the full essay →

The Journey

Ten questions every model must survive

Not one of them is a metric. Metrics are just the tools we use to answer them. A landmark earns its place only if skipping it can sink a model that looks impressive.

Define the Problem

What problem are we solving?

Data Foundation

Can we trust our data?

Features & Signal

What should the model learn?

Model & Training

Is it actually learning?

Evaluation · we are here

How well does it perform?

Validation & Generalization

Will it work elsewhere?

Trust & Human Judgment

Should humans rely on it?

Deployment

Can we safely deploy it?

Monitoring & Drift

Is it still working?

Continual Learning

How do we keep improving?

First region

We start with Evaluation

The Evaluation region map — performance, ranking, calibration, decision thresholds, and AI evaluation.

How well does it perform?

Evaluation is where a model looks most convincing and can mislead you most completely — and it's where all of my writing lives so far. Live now: performance (how 99.9% accuracy can catch zero fraud), ranking (AUROC's false alarms), and calibration (can you trust a 99%?). Next: decision thresholds, and evaluating AI with AI.

Browse the essays on Medium →

Start here

The series so far — read in order

The map The Journey Toward Trustworthy AI Why I stopped collecting AI metrics and started drawing maps. Evaluation · Accuracy Why 90% accuracy doesn't mean you should trust an AI model The trap that starts the whole series. Evaluation · Performance A fraud model can be 99.9% accurate and catch zero fraud Why accuracy lies about rare events. Evaluation · Ranking A model with 0.96 AUROC can still bury you in false alarms What a "great" score quietly hides. Evaluation · Calibration Does your AI know when it might be wrong? When it says "99% sure," can you believe it? Evaluation · The trio Three metrics, three questions: F1, AUROC, and calibration Why one number is never enough. Evaluation · Decisions Your AI model predicts risk. But who decides when to act? Once you trust the number, what threshold do you act on?

Traveling with a cat. Every good expedition needs one. He turns up across the atlas — pointing at charts, squinting at confidence scores, occasionally unconvinced. If a figure looks too clean, he's probably already suspicious.

Connect

Follow the journey

A new deep dive most weeks — one question at a time. The newsletter is the best way to follow along.

Substack LinkedIn Newsletter Medium LinkedIn Facebook