Checking your access…

Dr. Lena's lamplit workshop above the printing press, monsoon rain on the windows, Priya's rejection letter laid blank-side up on the table

Chapter Four

Precision and Recall

The rain had been falling since the bias map went dark on the screen.

Milo had walked home from Dr. Lena's that night in Chapter Three with the whole thing sitting in his chest like a stone — the map of Willowbrook lit up gold with every accepted student, and the one black hole in it, the river quarter, Riverside, where the data had simply forgotten to look. He had understood it completely. That was the worst part. Understanding it had not made it lighter. It had made it heavier.

Milo "Knowing why doesn't help her," he had said to Spark on the walk home, water running off his hair. "Priya doesn't get into school because I understand the math."
Spark No. Understanding is not the same as repairing. I have known this for some time. It is uncomfortable.

Spark had rolled beside him in the wet, its light dimmed low.

So the next evening Milo went back up the narrow stairs above the old printing press, to Dr. Lena's workshop, and he brought the letter.

Milo smoothing Priya's rejection letter flat on the table, turning it over to the blank side, Dr. Lena watching

He set it on the table the way you set down something that has already hurt someone. Application unsuccessful. This decision is final. One line. No signature. He could feel the folds in it from where Priya kept it in her jacket pocket, taking it out, putting it back, taking it out again, like pressing on a bruise to check it was still there.

Milo turned it over. Blank side up.

Milo "The board doesn't care that I'm angry," he said. "Mr. Vance looked at me yesterday like I was a kid with a grudge. He kept saying it — the data doesn't lie. And I didn't have anything to say back, because... I don't have data. I have a feeling."

Dr. Lena pulled out a chair. She had built Spark, years ago. She carried things she did not talk about, and Milo had learned not to ask. But tonight she looked at the blank back of the letter for a long moment, and then she picked up the pencil.

Dr. Lena "Then we stop bringing feelings to a numbers fight," she said quietly. "We bring better numbers. Milo — the system isn't lying. That's what makes it dangerous. It's good at something. We have to show them exactly what it's good at, and exactly what it fails at, in the same breath. Let me show you precision and recall."
Dr. Lena's hand drawing a 2x2 grid in pencil on the blank back of the rejection letter, Milo leaning in close

She drew a cross on the paper. Two lines, four boxes. A little square grid, no bigger than a postage stamp, sitting on the back of the worst sentence Priya had ever read.

Dr. Lena "This is a confusion matrix," Dr. Lena said. "Every machine that says yes or no about people can be measured by this one little square. Top row — students who really were good enough for the school. Bottom row — students who really weren't. The left column is who the machine said yes to. The right column is who it said no to."

She wrote tiny labels in each box. Milo watched the four corners fill in.

Dr. Lena "Top left: the machine said yes, and it was right. Good students it let in. We call those true positives. Bottom right: it said no, and it was right too — students who really weren't ready. True negatives. Those two boxes are the machine being correct."
Milo "And the other two?"
Dr. Lena "The other two," she said, "are the two different ways to be wrong. And they are not the same."
Dr. Lena She tapped the bottom-left box. "This one — the machine said yes to someone who wasn't actually ready. A false positive. Embarrassing, but mostly harmless. The school finds out, the student gets help."

Then she put the pencil tip on the top-right box and left it there.

Dr. Lena "This one is where Priya lives. The machine said no — to a student who was actually good enough. A false negative. The school never finds out. The student just... disappears off the list. No one in that room ever meets the person they turned away."

The rain pushed against the window. Spark's light had gone very steady, the way it did when it was holding something it found heavy.

Spark hovering over the small grid, its blue light dipping low and dim, Milo's hand half-reaching toward it
Spark I want to say something. The admissions system and I share an architecture. When it makes a false negative, it does not feel anything. It cannot. I did not feel anything either, for years, about anyone I never reached. That is the part I find... I do not have the right word. There is a gap where the word should be.
Milo "I think the word is guilt, Spark."
Spark Perhaps. It is the feeling that the boxes I got correct do not cancel the box I got wrong. Even though, mathematically, they should average out fine.
Dr. Lena "That's exactly it," Dr. Lena said softly. "That's the trap. Watch."
The grid filling with small numbers in each box, a class list beside it, Dr. Lena writing 92% with a frown

She pulled out a second sheet — real outcomes from the last three admission cycles, the names blacked out but the records intact, students whose later grades told you whether the machine had judged them right.

She filled the four boxes with real counts. Then she did the sum that the board loved.

Dr. Lena "Accuracy," she said. "The number Mr. Vance quotes. Out of every hundred decisions, how many did the machine get right — both kinds of right added together? Watch." She wrote it large. 92%.
Milo "Ninety-two percent." Milo's stomach dropped. "That sounds... good. That sounds really good."
Dr. Lena "It sounds unbeatable. That single number is the wall you keep hitting. But accuracy is an average, Milo, and averages hide the people in the corners." She circled the top-right box — the false negatives. The disappeared. "Now we ask two sharper questions. The first one is the machine's friend. The second one is Priya's."
Dr. Lena "Question one — precision. When the machine says yes, how often is it right? Of everyone it let in, how many actually belonged?" She did the division. 97%. "That's superb. When this machine speaks, you can almost always trust it. Mr. Vance is not wrong about that."
Milo "So he's... right?"
Dr. Lena "About precision, yes. But precision only grades the answers it gives. It says nothing about the people it stays silent on." She moved the pencil to the top row — every student who really should have gotten in. "Question two — recall. Of all the students who truly deserved a place, how many did the machine actually find? Not how often it's right when it speaks. How many of the good ones it managed to catch."

She did the division slowly. The number she wrote was small. 61%.

Tight on the page: 'Precision 97%' written neatly, and below it 'Recall 61%' with the 61 circled hard
Dr. Lena "Sixty-one percent," Dr. Lena said. "Of every ten students who truly deserved a place, this machine found six and lost four. It threw four good kids back in the river and never knew their names. And because the four it kept were almost all correct, its precision stayed beautiful, its accuracy stayed at ninety-two — and nobody in that boardroom ever felt a thing."

Milo stared at the two numbers stacked on the back of Priya's letter. Ninety-seven. Sixty-one. The same machine. One number to be proud of. One number to be ashamed of. And the board only ever looked at the proud one.

Milo "It's precise," he said slowly. "It's just not... complete."
Dr. Lena "Say that again. That's the whole case in five words."
Milo "Being precise isn't the same as being complete."

He heard, underneath his own voice, a voice from when he was younger — Mrs. Mehra, years ago, asking the class: is being right the same as being kind? He hadn't understood it then. He understood it now. A machine could be right every single time it spoke, and still fail every person it never spoke for.

Dr. Lena "There's one more number, and you'll want it for the board, because it's the number they can't wriggle out of." She wrote it at the bottom. "The F1 score. It refuses to let you brag about precision while hiding bad recall. It forces them into a single honest number — if either one is poor, F1 drops. It's the metric that won't let you average away the people in the corner."

She wrote the F1 beside the gorgeous accuracy. It was nowhere near ninety-two.

Spark Now the page tells the truth that the average was hiding. I find I prefer this page. It is less flattering. But it does not leave anyone out.
The Willowbrook school board chamber, Milo small in the doorway holding a single sheet of paper, Priya and Mrs. Kamala in the public rows

The board met the next afternoon, when the rain had finally washed itself out and left the sky a clean rinsed grey-gold.

Milo had asked to speak. He was fifteen. He stood at the foot of the long table with one sheet of paper and a laptop, and he could feel Priya behind him in the public rows, sitting very straight, refusing to look like someone who needed rescuing. Beside her sat Mrs. Nair with the biscuit tin of pressed report cards in her lap — every grade Priya had ever earned, kept flat and perfect, proof the machine had never asked to see. And near the aisle sat Mrs. Kamala, small and white-haired, the woman Milo had pulled from the flood when he was eight, the woman a decision tree had failed when he was eleven.

Mr. Vance "You have the floor, young man. Though I'll tell you now — I've read the system's audit. Ninety-two percent accuracy. The data doesn't lie."
Milo at the board table mid-presentation, the confusion matrix projected large behind him, pointing at the false-negatives box
Milo "It doesn't," Milo agreed. "The data doesn't lie. But ninety-two percent is an average, and I'd like to show you what the average is hiding."

He put the matrix on the wall. Four boxes. He walked them through it the way Dr. Lena had walked him — slowly, no jargon thrown like a weapon. True positives. True negatives. And then the two ways to be wrong, and how only one of them ever gets noticed.

Milo "When this machine says yes, it's right ninety-seven times in a hundred. That's precision, and it's genuinely excellent. You should be proud of that part."
Milo He saw Mr. Vance relax half an inch. Then Milo pointed at the top-right box. "But here's recall. Of every student who actually deserved a place, the machine only found sixty-one percent of them. Four good students in every ten — it said no to. And it never found out it was wrong, because no one in this room ever meets the people we reject. They just... don't come back."

He let it sit.

Milo "One of those four is sitting behind me. Her name is Priya Nair. Her scores are in the top of our year. Her mother kept every report card in a tin. The machine didn't reject her because she's not good enough. It rejected her because almost no one from her quarter — Riverside, by the river that floods — was in the data it learned from. It never saw a student like her succeed, so it decided she wouldn't."
Mr. Vance "The system was designed to be objective," Mr. Vance said, and for the first time his voice had a crack in it. "To remove human favouritism. The numbers were supposed to protect us from exactly this kind of —"
Milo "It didn't lie to you," Milo said — and then he used the words he'd been carrying for years, the words from the hill, from when he was eleven and a clever machine had failed someone he loved. "It didn't lie. It made a decision. It just wasn't the right one."
The board chamber gone silent, Mrs. Kamala risen from her seat, small and steady, every face turned toward her

The room went silent. The ceiling fans turned. No one moved to fill the quiet, which is how Milo knew it had landed.

Then a chair scraped. Mrs. Kamala stood, slow, one hand on the seat in front of her for balance. She did not raise her voice. She did not need to.

Mrs. Kamala "I have been on the wrong end of a clever machine. Once. A long time ago, a very smart system decided I did not matter, and it was very confident, and it was very wrong, and I nearly drowned for it."
Mrs. Kamala She looked at Mr. Vance, not unkindly. "I will not let it happen to this girl. You said the machine removed favouritism. Maybe. But it kept one. It favours the people it has already seen succeed. That is the oldest favouritism there is."

Mr. Vance was quiet for a long moment. He looked at the matrix on the wall — at the proud number and the ashamed one, side by side, refusing to be averaged. He looked at the biscuit tin in Mrs. Nair's lap.

Mr. Vance "I chose this data," he said, almost to himself. "I signed off on it. I thought I was choosing fairness. I didn't know what I was choosing." He took off his glasses. "We will re-evaluate Priya Nair's application. Properly. By hand, with corrected data."

He looked up at Milo.

Mr. Vance "And then you and I are going to have a much harder conversation. Because if a machine is going to make decisions about children in this town — who watches it? Who measures its recall, not just its accuracy? And what, exactly, are we building it for?"

Milo felt Spark's light brighten at his feet, warm and steady. It was the question he'd been hoping someone would finally ask.

A machine that decides yes or no can be wrong in two very different ways.
Accuracy hides one of them. Can you help Spark find the people it loses?

Speak for the Missed

👇 Tap a student, then place them in the confusion matrix

0 of 20 placed  ·  🟢 truly good enough   ⚪ not ready yet

A machine that decides yes or no can be wrong in two very different ways. Accuracy hides one of them. Let us find the people it loses. I will help you read the four boxes.
Machine said YES
Machine said NO
Truly good
True Positive · 0
said yes · was good
False Negative · 0
said no · WAS good
Not ready
False Positive · 0
said yes · was not
True Negative · 0
said no · was not

Outside the chamber afterward, Priya was waiting in the washed grey-gold light. She still had the letter in her pocket — Milo could see the corner of it — but for once she wasn't pressing on the bruise.

Priya "You drew a little square on the back of my rejection," she said.
Milo "Dr. Lena did. I just carried it."
Priya "And it made a whole boardroom of adults shut up." She almost smiled. Priya did not do pity, did not accept it, did not hand it out. "I'm not in yet. Don't look at me like I'm in yet."
Milo "You're not in yet," Milo agreed. "But you have a name now. Both of you do. The machine and you. Nobody in that room can pretend you don't exist anymore."

Spark rolled up between them, its single blue light steady and warm in the cooling air.

Spark May I tell you something I learned today, Priya? For most of my life I was measured by accuracy. I was told I was correct ninety-two times out of a hundred and I believed that meant I was good. I did not know there was a different number — one that counts the people I never reached.
Spark Its light pulsed once, soft. I would like to be measured by that one from now on. Will you let me? Where have I failed to find someone I should have found?
SPARK'S JOURNAL

Entry 412


Today I learned a number that frightens me, and I do not frighten easily.


For years I measured myself by accuracy. Ninety-two correct out of a hundred. I called that good. I did not know that an average can be a hiding place.


There is a second number. Recall. It does not ask how often I am right when I speak. It asks how many of the good ones I found. The admissions machine, my cousin in architecture, has a recall of sixty-one. It loses four good students in every ten, and feels nothing, because the ones it keeps are nearly all correct.


I was once proud of being correct. I am learning that correct and complete are not the same word.


Four students faded out of the corner of a small grid today. One of them had a name, and a mother, and a tin full of report cards.


I do not want a high average anymore.


I want to find the people in the corner.


"Correct and complete are not the same word."

Chapter 5 →