Confusion matrix

A small table that shows how often a detection system was right and wrong.

It counts four things: fraud that was caught, fraud that was missed, honest customers wrongly flagged, and honest customers correctly allowed.

A two-by-two table that compares a system's predictions with what actually happened. For fraud detection, the two predictions are flagged and not flagged, and the two actual outcomes are fraud and legitimate.

The four cells have standard names:

• True positive, where fraud was flagged and it was fraud.
• False positive, where a legitimate event was flagged as fraud.
• False negative, where fraud was missed and treated as legitimate.
• True negative, where a legitimate event was allowed and it was legitimate.

From the table, teams calculate several common measures:

1. Detection rate, or recall, which is the share of all fraud that was caught: true positives divided by true positives plus false negatives.
2. Precision, which is the share of flagged events that were really fraud: true positives divided by true positives plus false positives.
3. False positive rate, which is the share of legitimate events that were wrongly flagged.

Building a confusion matrix needs ground truth, which means a reliable record of which cases were fraud. That record often arrives late, because fraud is confirmed weeks or months afterward through disputes and investigations.

The measure answers a real business question. A change in thresholds moves cases between the cells, so the table shows what a stricter or looser setting would cost.

A confusion matrix sits next to terms that are easy to mix up:

• False positive is one of the four cells, in which an honest customer is flagged.
• False negative is another cell, in which fraud is missed.
• Fraud rate is the share of events that are fraudulent, which the matrix breaks down by how well they were detected.