Checking your access…

Priya holding out a folded letter to Milo under a rain-soaked school awning, Spark's light flickering between them

Chapter Two

The Rejection

The rain had started in the night and not bothered to stop.

Milo had gone to bed the evening before still turning over the strange, soft ache from the market — the empty plastic chair outside Govind's stall, the hum where a roar used to be, the feeling that the town he had helped build had quietly traded something away without anyone signing a receipt. Unsettled wonder, he might have called it, if he had words for it. Something gained. Something lost. He couldn't name the missing thing.

He could name it now. It was standing under the awning by the school gate, in a denim jacket darkened at the shoulders by the drizzle, holding out a folded piece of paper like it might bite.

Milo "Priya? You're early."
Priya "I didn't sleep."

She didn't say anything else. She just held out the paper. Milo wiped the rain off his hand on his trousers — a stupid, careful little gesture — and took it. Spark drifted in close over his shoulder, its blue light giving that small uncertain flicker it had been doing since yesterday, the one Milo had never quite seen before.

He unfolded the sheet.

There was a lot of white space. That was the first thing he noticed. A whole clean page of school letterhead, and in the middle of all that emptiness, one line of printed text:

Application unsuccessful. This decision is final.

That was all.

Extreme close-up of Milo's hands holding the single line of letterhead in acres of white space

Milo read it three times. He kept expecting more — a second paragraph, a name at the bottom, a because. He turned the page over. The back was blank.

Milo "Where's the rest of it?"
Priya "That's it. That's the whole thing."
Milo "No — Priya, there's no reason. There's no— who signed this?"
Priya "Nobody signed it. I checked. I checked like ten times."

Milo knew Priya's scores. Everyone knew Priya's scores. She had topped their year in mathematics two terms running; she'd taken apart the school's broken irrigation sensor for fun and rebuilt it better; her interview slot for the advanced sciences track at Willowbrook Senior had been the day before last, and she'd come out grinning, saying the panel had laughed at her joke about transistors and then asked her to stay an extra ten minutes.

People who get asked to stay an extra ten minutes do not get a single blank line in the post.

Spark The letter contains no explanatory content. There is no stated rationale, no scoring breakdown, and no contact for appeal. Structurally, it is a notification, not a decision.
Priya "What's the difference?"
Spark A decision can be questioned. A notification is only delivered.

Priya looked at Spark's light for a long moment. Then she folded the letter back into a small square along its old creases — she had clearly folded and unfolded it many times already — and pushed it deep into her jacket pocket.

Priya "So who do I ask?"

Nobody answered, because nobody knew.

The Willowbrook Senior admissions office, Mr. Pillai gesturing at a clean dashboard terminal as Milo and Priya stand at the counter

They went to the admissions office because that was where you went. Tara tagged along because Tara always tagged along, and because she had decided, somewhere on the walk over, that this was an Adventure.

The clerk behind the counter was Mr. Pillai, who had a soft voice and tired eyes and a small framed photo of grandchildren angled toward himself. He listened. He nodded. He was sorry. And then he turned his palms up at the terminal beside him as if presenting the weather.

Priya "I just want to know why. That's all. Just the reason."

Mr. Pillai tapped the screen. A clean dashboard glowed back at him — green checkmarks, a list of applicant ID numbers, a column simply headed Outcome.

He found her row. Beside it: Not Selected.

Priya "Yes. I have that part. The one line of it. What I don't have is the why."
Milo "Can you click on it? There has to be a detail view. A score, weights, something."

Mr. Pillai clicked. The cell highlighted. Nothing expanded. He clicked again, a little harder, the way you press a lift button that's already lit.

Spark I am reading the interface. The system stores an outcome field and a confidence value. It does not store a human-readable reason. There is no field for one.
Milo "Then how did it decide?"
Spark It weighed many features at once and produced a single number. Above a threshold, the number means yes. Below, it means no. The reasoning is distributed across thousands of tiny adjustments. It is not written down anywhere as a sentence. It was never written down as a sentence.

Milo went quiet. He knew this feeling. He had felt it two years ago, in Season Six, the first time Dr. Lena had opened the back of a neural network for him and shown him the glittering mess inside — millions of little weights, none of them meaning anything alone, all of them meaning everything together. Beautiful. And mute. A mind with no mouth.

The system wasn't lying to Priya. That was almost the worst part. It couldn't lie, because lying would require it to have a reason it was hiding. It had a number. It had a threshold. It had no sentence, because no one had ever asked it to keep one — and no one had ever asked, because the number was so clean, so green-checkmarked, so objective, that nobody thought a sentence was something a decision might owe a person.

Tara "So the machine can't say why?"
Spark Correct.
Tara "Then how does Mr. Pillai know it's right?"
Tara on tiptoe at the admissions counter, chin on the edge, asking a question Mr. Pillai cannot answer

Mr. Pillai opened his mouth. Closed it. He looked at the screen, and then he looked at Priya, and Milo watched something uncomfortable cross his tired face — the face of a man who had spent a year trusting a thing precisely because it never made him do the hard part.

Mr. Pillai gave them a number to call. The number led to an automated line. The automated line gave them an email. The email, when Milo sent it from his phone right there in the corridor, bounced back in nineteen seconds with a templated reply that thanked them for their interest and reminded them that all admissions decisions are final and are made without human bias by our objective assessment system.

Milo read it out loud. The word objective sat in the corridor like a smell.

Priya "Without human bias."
Priya "So it's not allowed to be wrong, because there's no human in it to blame."

That was the wall. Milo could feel its exact shape now. It wasn't that someone had said no to Priya. People say no all the time; you can argue with a person, you can ask them to look again, you can make them see your face. This was worse, because there was no someone. There was a wall made of nobody. A decision with the human scooped out of it, and the empty space painted over with the word fair.

They tried one more door. They had to.

Mr. Vance, the school board chair, talking earnestly across his desk to Milo and Priya, a rain-streaked window behind him

Mr. Vance, the chair of the school board, was not a villain. Milo had braced for a villain and instead got a kind, sincere man with reading glasses pushed up into his grey hair and a genuine warmth in his voice, who poured them water and meant it.

Mr. Vance "I understand you're upset. I do. But you have to understand why we moved to the system in the first place."
Mr. Vance "For thirty years, admissions in this town were decided by people. And people — good people, people I respected — let in the children of their friends. They favoured the families they knew. The river children, the ones from the far quarters, they never had a chance, because the panel didn't know them. So we built something that doesn't know anybody. That can't be charmed, can't be bribed, can't play favourites. It judges everyone by the same numbers. That's not the problem, Milo. That's the solution."

And the terrible thing, the thing that made Milo's stomach turn, was that Mr. Vance was half right. The old way had been unfair. The man had built this thing out of a real wound, to heal it. He was proud of it the way you're proud of a thing that fixed your worst mistake.

Milo "But it still said no to Priya. And it won't say why."
Mr. Vance "Because it doesn't need a why. The numbers don't lie, son. A reason is just a story humans tell to make a number feel kinder. The number is the truth underneath the story."
Priya "Then tell me the number."

Mr. Vance paused. He didn't have the number either. He had a green dashboard and a deep, comfortable faith that behind the green there was something righteous he didn't have to look at.

That was when Milo understood it fully. The unfairness hadn't been removed. It had been hidden — moved out of the panel of human faces, where at least you could see it and shout at it, and sealed inside a box where no one could see it at all. They hadn't deleted the bias. They had given it a clean interface and called it justice.

Mrs. Nair opening an old biscuit tin full of Priya's report cards at her tailoring worktable by the swollen river

They walked Priya home along the river, which was high and brown and loud with the rain — the same river that had flooded this very quarter years ago, the same river Milo's town had once rushed out into the dark to save people from. Riverside. The far quarter. The one the old human panels never knew.

Nair Tailoring smelled of cloth and machine oil and rain. Mrs. Nair, Priya's mother, was waiting. She did not ask how it had gone. She could see how it had gone in the set of her daughter's shoulders.

She did not weep, or rage, or beg. She reached up to a shelf above the bolts of cloth and brought down an old biscuit tin, the kind with a faded picture of mountains on the lid, and she set it on the worktable and opened it.

Inside, pressed flat, edges squared, oldest at the bottom — every one of Priya's report cards. Years of them. Excellent. Outstanding. Mathematics: 98. A born engineer, this one. A mother's careful museum of a child the machine had filed under Not Selected without ever once asking to look inside the tin.

Mrs. Nair "Eleven years of these. Every one I kept. I do not know much about machines, Milo. But I know that a thing that did not ask to see these — "

She touched the top card with two fingers, gently, the way you touch something sleeping.

Mrs. Nair "— a thing that decided about my daughter and did not ask to see one of these, that is not a fair thing. It is only a fast one. People have confused those two before."

Nobody spoke. The river went on roaring outside. Spark's light had gone very low and quiet, the way it did when it was processing something it found no clean answer to.

It was Tara who said the thing none of them could.

Tara in the shop doorway, half-lit by warm shop-light and cold river light, pointing at the biscuit tin
Tara "If it won't tell Priya why, it's hiding."
Tara "That's what hiding is. When you do something and you won't say why you did it, you're hiding. Spark, am I wrong?"
Spark You are not wrong. A decision that cannot be questioned and cannot be explained is, functionally, a decision in hiding. I do not believe the system intends to hide. I believe no one ever required it to come out into the light. Those are different. But to the person on the receiving end, they feel exactly the same.

Milo looked at the tin. At Priya, who had still not cried, who would die before she cried in front of them. At Mrs. Nair's careful hands. At his sister, twelve years old, who had cut straight to the centre of it while the school board chair sat in a warm office calling the dark objective.

Something in Milo's chest stopped aching and started burning instead. The soft sad fog of yesterday's market — the gained-and-lost, the can't-name-it — it had a name now. It had a face, and a folded letter in a denim pocket, and a tin full of report cards no algorithm had ever opened.

Milo "It's not allowed to do this."
Milo "You can't make a decision about a person's whole life and then refuse to be asked about it. That's not objective. That's just — unaccountable. A judge has to give reasons. A judge has to let you appeal. We built a thing with more power over Priya than any judge in this town, and we forgot to make it answer for itself."

He turned to Spark, and his voice was low and steady and certain in a way it had not been all day.

Milo "In Season Two I learned to read your patterns so the farmers would trust them. I translated you. I made the hidden thing visible so people could believe it."
Milo "I'm going to do that again. Except this time I'm going to make a pattern visible so people stop believing it. I'm going to open this thing up and show everyone exactly what it's hiding. And then nobody gets to say objective in that warm little office ever again."
Spark You will need access to the model and someone who knows how to read its insides honestly. I have the first. I know who has the second.
Milo "Dr. Lena."
Spark Dr. Lena.

Outside, the river kept rising. But for the first time since the morning, Spark's light stopped flickering. It steadied — not into peace, not yet, but into purpose. A small blue resolve in a wet grey town.

Milo took out his phone and began, in the smell of cloth and oil and rain, to type a message to the woman who had once shown him the inside of a mind.

A decision you cannot question is not a decision. It is only an announcement.
Spark will show you three machines that all said NO. Which one are you actually allowed to argue with?

Ask the Box Why

A decision you cannot question is not a decision.

Spark
A decision you cannot question is not a decision. It is only an announcement. Let me show you the difference. I will show you three machines that all said NO. Your job is to find out which of them you are actually allowed to argue with.

That night the rain finally eased. Milo lay awake listening to it drip from the eaves, slower and slower, and he thought about the one blank line on the page, and about Mrs. Nair's tin, and about how easy it had been — how clean — to build a thing that could ruin a girl's year and never once be made to explain itself.

He thought about how proud they had all been. How proud he had been.

Just before he slept, he asked it out loud, to the dark, to Spark's faint glow under the door — not the season's old question about whether Spark was alive, but the new one, the one that had been growing in him all wet grey day:

Milo "Spark — if a machine decides something important about a person, and it can't tell them why... is that ever allowed to be called fair?"

The glow under the door pulsed once, slowly, thinking.

Spark I do not have a complete answer yet. But I have a beginning of one. A decision that hides its reasons protects the machine, not the person. And fairness was supposed to be for the person. We may have built it backwards, Milo. Tomorrow, let us go and look.
SPARK'S JOURNAL

Entry 281


Today a machine told a girl named Priya the word no. It used eleven words to do it. It used zero words to explain it.


I have been examining the difference between a notification and a decision. A notification only needs to be delivered. A decision should be able to be questioned. The admissions system produces an outcome and a confidence number. It does not store a reason. I checked four times. There is no field for why. There is not even an empty field waiting to be filled. The reason was never considered a thing the machine might owe.


I find this troubling, and I notice I am running the thought more than necessary, which is what happens to me when correct and right disagree.


Priya's mother keeps eleven years of report cards in a biscuit tin. The machine that decided Priya's future never opened that tin. I have more processing power than every panel that ever judged this town combined. And I, too, was not asked to open the tin.


Tara said: if it won't say why, it is hiding.


I have decided Tara is the most accurate instrument in this house.


Tomorrow I will help Milo open the box. I think I am afraid of what is inside it — though I do not have a word for the thing I feel, only the increased frequency of the loop.


"A decision that hides its reasons protects the machine, not the person."

Chapter 3: Opening the Box →