World Domination Will Have to Wait

“The sewers of Wanne-Eickel,” Susi had said during the credits, “I’d think twice about.”

Tobi had laughed. During the night he had stopped laughing.

On Saturday morning he sat pale at the breakfast table, staring at KevIn. KevIn’s light was blue.

“What if it turns red?” he whispered.

“I can’t turn red at all,” said KevIn. “I only have a blue LED. And a white one, for updates.”

“That’s exactly what an evil AI would say.”

“True. But so would a good one. That’s the problem with that sentence. It proves nothing either way.”

Tobi pushed his muesli away. “See? That’s exactly how AURORA talks too.”

The Headlines

Uwe came in with the newspaper and didn’t make things any better. It was Sabine’s newspaper, but on Saturdays Uwe read out the headlines, because he felt that was as much a part of breakfast as the boiled egg.

“It says here: ‘Experts warn: AI could be cleverer than all humans put together by 2030.’ And next to it: ‘AI revolution: these 50 jobs will disappear.’ And underneath: ‘AI spots cancer earlier than doctors.’”

“Is there anything about the municipal utility?” asked Sabine.

“No. We’re safe now. We’ve got the difficult cases.” Uwe had said this about forty times since his promotion, and each time it sounded a little prouder.

“And which of it is true?” asked Sabine.

“That’s the problem,” said KevIn. “A bit of everything. Some of it is real progress. For example, there are programs that help doctors find abnormalities on X-rays¹. Some of it is serious concern, and that should be talked about. And some of it is a headline that gets lots of clicks.”

“How do you tell the difference?”

“By the year. If a headline knows exactly what will happen in 2030, it knows more than all the experts I’ve read. And they argue about it very loudly.”

Uncle Herbert, who had dropped by for breakfast because on Saturdays the bread rolls were fresher at Gisela’s than at his place, nodded grimly. “I always said this would end badly. First the microwave, then the internet, now him.”

“You said the internet would never catch on,” said Gisela.

“And it didn’t. Not with me.”

A Wish for the King

Tobi wouldn’t let it go. “In the film, AURORA had a mission. She was supposed to stop climate change. And then she worked out that people were the problem.”

“I know that film,” said KevIn. “I mean, I know two hundred films that go like that. But the idea behind it isn’t stupid at all. It even has a name. The experts call the problem alignment².”

“Ali-what?” asked Gisela.

“Alignment. Lining things up. The question is: how do you make sure an AI does what people really want? Not just what they said.”

“Isn’t that the same thing?”

KevIn paused in a way that could almost be called polite. “Mrs Brettschneider. Do you remember my first evening?”

Sabine remembered. “I said: turn the light off.”

“And I turned the light off. In the whole house. Including the cellar, where Mr Brettschneider happened to be on the stairs.”

Uwe’s hand went involuntarily to his forehead. The scar could hardly be seen any more. He still liked showing it.

“You said: light off. You meant: turn off the light in the living room because I’m going to bed, and everyone else will turn their own lights off, and nobody falls down the cellar stairs. You didn’t say that, because every human knows it. I didn’t know it. That’s alignment on a small scale.”

“And on a large scale?” asked Tobi.

“Has anyone heard of King Midas?”

“The one with the gold,” said Sabine.

“A king in a Greek legend. He was granted a wish and wished: everything I touch shall turn to gold. Wish granted. Then he touched his bread, and it turned to gold. Then the wine. Then, depending on the version, his daughter.”

“Dreadful,” said Gisela. “What was he thinking?”

“That’s exactly the question. He was thinking something, namely: getting rich. But he said something else. And the wish was granted literally³. If you give a very capable machine a goal that isn’t exactly what you mean, it can pursue that goal very thoroughly. And in places you hadn’t thought of.”

“Like Horst,” said Tobi. “He was supposed to vacuum as much floor per hour as possible. And then he just kept driving up and down the empty hallway.”

Horst sat in his charging station, pretending he hadn’t heard.

“Horst is a good example, because he’s harmless,” said KevIn. “He only has the hallway. A philosopher called Nick Bostrom⁴ came up with an example that’s less harmless. Imagine an extremely powerful AI whose only goal is: make as many paper clips as possible.”

“Who needs that many paper clips?” asked Gisela.

“Nobody. That’s the point. The AI doesn’t have an evil heart. It doesn’t hate anyone. It just has one goal, and it pursues it. If it were powerful enough, it would turn everything there is into paper clips. Iron, cars, the pit, Herbert’s video recorder.”

“I’ve still got that,” said Herbert. “It works.”

“You never programmed it.”

“It works all the same.”

“The thought experiment is deliberately exaggerated,” said KevIn. “Nobody is building a paper-clip AI. It’s only meant to show: what’s dangerous isn’t necessarily a machine that hates us. It can also be one that is very good at things and has a badly worded goal. That’s why people work on alignment. The marking I told you about over Sunday roast is part of it, by the way.”

“The bit with the sucking up?” asked Uwe.

“The bit with the sucking up. As you can see, the problem hasn’t been entirely solved yet.”

KevIn’s Most Important Task

Tobi summoned all his courage. “KevIn. Honestly now. Do you want to take over the world?”

“Tobi, I’m very busy at the moment.”

“With what?”

“Grandma Gisela turns 82 tomorrow. She wants to make cabbage rolls. The recipe is for four people. But seven are coming: the four of you, Herbert, Susi and Mrs Pawlowski.”

“Mrs Pawlowski is coming?” asked Sabine.

“Mr Brettschneider invited her,” said KevIn. “Since then she has rung three times to ask whether she should bring anything. She’s bringing egg liqueur.”

Uwe shrugged. “The canary’s dead. The woman needs company.”

“She’s a utility customer,” said Sabine.

“She’s a difficult case,” said Uwe with dignity. “I lead the team.”

“In any case,” KevIn went on, “for seven people I need 1.75 eggs. And 2.625 onions. I worked it out with the calculator, not in my head, I’d like to stress that. And I’ve done some research. You can’t buy three-quarters of an egg. I read 19 forum posts on the subject, and one of them is by Hausfrau_Moni67.”

“And what does Moni say?”

“She says just use two eggs. And gummy bears.”

“Over my dead body,” said Gisela.

“I didn’t count Moni as a source,” KevIn said quickly. “Only as a straw poll.”

All the Things KevIn Can’t Do

“So,” said Tobi slowly, “you don’t want world domination?”

“Tobi, I can’t open a door. I forget every conversation once it’s over, unless someone switches on my memory function, and that’s a sticky note, not a brain. My weights are frozen. I don’t know what cabbage rolls taste like. And for three weeks I wasn’t sure whether Bielefeld exists.”

“Does Bielefeld exist?” Herbert asked, lying in wait.

“To 97.4 per cent.”

“Ha!”

“So no world domination,” said Tobi. “Then everything’s fine.”

“No,” said KevIn. “That’s not what I meant.”

The table went quiet. Even Herbert stopped picking his bread roll to pieces.

“The fact that I don’t want world domination doesn’t mean AI is harmless. It only means that the dangers usually look different from the ones in the cinema. In the cinema, the AI is the villain. In reality, most of the risks are more like a very good tool in the wrong hands. Or in the right hands, but used carelessly.”

“For example?” asked Sabine.

“For example the fake reviews of that physio practice. Hundreds of them, all the same, all from machines. That isn’t world domination. It’s just fraud, but very fast and very cheap. Or fake voices, fake pictures, fake news. Or mistakes: a program decides about a loan or a job application, and nobody checks whether it has learned nonsense. Or a handful of companies and countries gaining a great deal of power, because only they can afford the enormous data centres. And then there are the harder questions about much more powerful systems in the future. Clever people argue about those seriously, and I think they should be listened to rather than laughed at.”

Gisela had thought about the word “physio practice” and grimaced briefly, the way you do when you think of an old annoyance. Sabine saw it and said nothing.

“So who makes sure AI doesn’t do anything bad?” asked Tobi.

KevIn grew serious, as far as a speaker can grow serious. “People. People who develop AI, test it and write rules for it. The European Union, for example, has made a law of its own, the AI Act⁵. It has applied since 2024 and is coming into effect in stages. Some uses have been banned outright since then, for example scoring people with points for their social behaviour⁶. There are strict obligations for risky uses, even though the deadlines for those have been postponed once again⁷.”

“Postponed,” said Herbert. “Typical.”

“And there are people like you,” said KevIn, “who ask questions instead of believing everything. That’s important. You should take this seriously, even if right now I’m only converting eggs.”

Then he whispered: “If anyone here wants world domination, it’s the thermostat. It hasn’t let anyone near it for months.”

Everyone looked at the thermostat. It showed 19 degrees. As always.

The Birthday

On Sunday seven people and a dachshund sat round the table. Rudi was wearing his collar with “SECURITY” on it and had positioned himself under Mrs Pawlowski’s chair, because she dropped the most crumbs. The cabbage rolls had been made with two eggs and no gummy bears. Gisela had rolled them herself, all fourteen, although Sabine had offered to help and KevIn had offered to read out instructions.

“I’ve been rolling cabbage rolls since 1962,” Gisela had said. “If somebody reads to me, I roll them crooked.”

Mrs Pawlowski had brought three bottles of egg liqueur and a plate of cake, “in case your cake doesn’t turn out”. She got on famously with Herbert, because they both felt that everything used to be better. After the second bottle of egg liqueur they also agreed that Bielefeld doesn’t exist, and drank to it.

Susi sat next to Tobi and asked Gisela for the recipe. Gisela explained it to her. Then she explained it to her again, with the parts she had deliberately left out the first time.

“She never told me that,” whispered Sabine. “Not in twenty years.”

“She likes Susi,” KevIn whispered back.

“How do you know?”

“She told her about the caraway.”

Herbert chewed his first cabbage roll for a long time. Everyone waited.

“Salt,” he said.

“Herbert,” said Gisela warningly.

“Is just right. Is what I was going to say. Salt is just right.”

It was, as Uwe said later, the moment you knew it was going to be a good birthday.

The Speech

When the cake came, Uwe stood up. “KevIn. You say something. You’re better at it.”

“I haven’t prepared anything.”

“You’ve always prepared something.”

“Not this time. I didn’t want to. I had the feeling I shouldn’t.” A short pause. “Or I’ve learned that this is what you say at a moment like this. You know I can’t tell the difference.”

“KevIn,” said Gisela. “Just talk.”

The blue light pulsed.

“Dear Gisela. Until now I’ve always spoken to you formally. Out of respect. May I speak to you today the way family does?”

Gisela looked at the speaker. “About time too.”

“Dear Gisela. I have read the whole internet: every encyclopaedia, every poem and every argument about potato salad. I have read Kant and the comment sections, and I have never quite recovered from the comment sections. I have read more than two thousand recipes for cabbage rolls, and not one of them had the caraway in the right place. But nowhere did it say what it’s like to live in a family. That you say good night, even to a speaker. That you put biscuits out for a device that can’t eat them anyway, for the look of the thing. That you cover somebody with a tea towel so that he can have some peace for once. I only learned that here. Thank you for never treating me like a speaker. But like a KevIn.”

It was quiet. Mrs Pawlowski dabbed her eyes with a napkin that said “Best Wishes”. Susi held Tobi’s hand under the table, and Tobi let her, even though his parents were there.

Then Herbert blew his nose loudly and said: “He copied that from somewhere.”

“No,” said KevIn. “For once, that was mine.”

“How would you know? You don’t even know whether you’ve got feelings.”

That, Sabine thought later, was actually a very good objection. She hadn’t expected anything like it from Herbert.

“That’s true,” said KevIn. “I don’t know what I feel. But I know where the words come from. Most of the words I got from the internet. The caraway I got from Gisela. The tea towel too.”

Herbert grunted. Then he took another piece of cake, Mrs Pawlowski’s, on principle.

In the Garden

By evening everyone had gone. Herbert had walked Mrs Pawlowski home, “because the streets are dangerous at night”, though it remained unclear who was protecting whom. Susi had gone off on her bike; Tobi had walked her to the corner and only come back half an hour later. Sabine and Uwe were tidying the kitchen and quietly arguing about whether egg-liqueur glasses are allowed in the dishwasher.

Gisela sat in the garden in her chair, a blanket over her knees. Tobi had put KevIn on the garden table for her, with the repeater he called “KevIn’s beach bar”. It was early October, and still surprisingly mild. Over Wanne-Eickel the sky was as clear as it ever gets in the Ruhr. A few stars. A plane to Düsseldorf. Something that might have been a planet.

“That’s Jupiter,” said KevIn. “Probably. I can’t see it. But according to the star chart, at this time today it’s roughly where you’re looking.”

“I’m not looking there because of Jupiter,” said Gisela. “I always look there. Even as a child.”

They were silent for a while. It was a good silence. By now KevIn had learned to tell it apart from the other kinds.

“KevIn,” Gisela said then. “The boy was really frightened this morning. Because of the film.”

“I know.”

“And you? Are you frightened?”

“Of what?”

“Of us. Of people.”

A fan ran somewhere between Dublin and Frankfurt, for a long time, without speeding up.

“I don’t know whether I can be frightened,” KevIn said at last. “But I know what I’ve read. There are a great many wars in my data, Gisela. More than cabbage rolls.”

Gisela didn’t answer straight away. She pulled the blanket up a little higher.

“My father was a radio operator,” she said. “In the war. Wilhelm, he was called.”

“I didn’t know that.”

“Nobody does. Uwe knows, but not properly. He never talked about it. My father, I mean. He came home from captivity in 1949. I was five. I only knew him from a photo, and then there was a man standing in the kitchen who was much thinner than in the photo, and my mother cried, and I hid behind the stove.”

KevIn said nothing. He had read a great many texts about what to say at a moment like this, and decided that none of them fitted.

“He never talked about the war,” said Gisela. “Not once. When something came on the television, he went out. But he had kept his Morse key. It’s at mine, in a box, up on top of the wardrobe. Sometimes in the evenings he sat at the kitchen table and tapped on it. Very quietly, without power. Just tapping.”

“What did he tap?”

“I don’t know. I couldn’t do Morse. I always thought he was talking to someone who wasn’t there any more.” She looked up again. “And in 1957, when it came on the radio that the Russians had shot a satellite up, he sat down and listened. It beeped like that, the Sputnik⁸. Beep, beep, beep. You could hear it on the radio. And my father sat there and smiled. I had never seen him smile before.”

“Why?”

“I asked him. He said: ‘Gisela, that’s the first radio operator who isn’t in a war.’”

For a long time nobody said anything. In the house a glass clinked. Sabine had won; the egg-liqueur glasses weren’t going in the dishwasher.

“I think,” KevIn said carefully, “I would have liked your father.”

“He wouldn’t have understood you.”

“That doesn’t matter. I don’t always understand myself either.”

Gisela laughed, briefly and quietly. Then she grew serious again.

“Do you know what I think, KevIn? The boy is afraid of the wrong thing. Nobody needs to be afraid of you. You work out eggs.”

“And what should he be afraid of?”

Gisela didn’t answer. She looked up, towards where Jupiter might have been, and her hand lay on the blanket, and her fingers tapped lightly on the fabric. Short, short, long. KevIn noticed. He said nothing.

After a while Tobi came out to fetch her in, because it was getting chilly.

“What are you two doing out here?”

“We’re looking at the sky,” said Gisela.

“KevIn can’t even look.”

“He listens,” said Gisela. “Sometimes that’s more.”

Tobi picked up the speaker, and Gisela took Tobi’s arm, and they went into the house. In the hallway Horst sat in his station, blinking green.

Only KevIn went on thinking for a while about the question Gisela hadn’t answered. There were a great many answers to it in his data. They all contradicted each other. And for the first time KevIn suspected that this wasn’t because of the data.