A Turing Test over Sunday Roast

“Artificial intelligence,” he snorted. “That’s nonsense. There’s some fellow sitting in a call centre, typing.”

“Good afternoon, Herbert,” said KevIn. “I’ve heard a great deal about you.”

“I bet you have. I’m well known.”

“Mainly on the note on the fridge,” said Sabine quietly. “Under ‘Critical error’.”

Herbert didn’t hear that, because on principle he only heard things he liked. He chewed. Then he pointed his fork at the speaker. “And why’s he so formal with me? He’s like that with Gisela too, all polite and proper. Does he think we’re too old, or what?”

“I address people formally out of respect,” said KevIn.

“Out of respect,” Herbert repeated. “The lad at the electronics shop was very formal with me as well. And then he sold me a cable for 39 euros.”

Grandma Gisela sat at the head of the table with the cushion at her back that Sabine had put there every Sunday since her fall over Horst, whether she wanted it or not. Horst himself had only been allowed to run since then when everyone was sitting at the table. He was, so to speak, on probation.

“Herbert,” said Gisela. “Eat.”

Herbert ate. It was the only thing in this family that had worked reliably for seventy years.

Why Is He So Nice?

During second helpings, Herbert thought of something new.

“He’s sucking up,” he said. “Can’t you tell? He says ‘happy to’ and ‘good question’ to everything. No normal person keeps saying ‘good question’.”

“That’s a good …” KevIn began, and stopped.

Tobi grinned. “He’s right, KevIn. You are pretty smarmy.”

“I’m polite. There’s a difference. But I admit: I didn’t turn out like this on my own.”

“How, then?” asked Sabine.

“I was taught it. After the great reading I told you about, I was a being that was very good at continuing texts. But not necessarily friendly. If you had asked me something back then, I might have answered with another question. Or with a forum row. Or with ‘First!’.”

“Like Hausfrau_Moni67’s comment section,” said Sabine.

“Exactly like that. That’s why there was a second school. It’s called RLHF¹.”

“Bless you,” said Herbert.

“Reinforcement Learning from Human Feedback,” said KevIn. “In plain words, roughly: learning through reward, and the reward comes from people.”

“Like Rudi with his treats,” said Tobi. “We’ve had that.”

Rudi raised his head at the word. Nobody gave him anything. He laid it down again in deep disappointment.

“Roughly,” said KevIn. “Except that I don’t get treats, I get marks. It went like this: I was asked a question, and I wrote two or more answers. Then people compared the answers. Which is more helpful? Which is more honest? Which isn’t nasty? Thousands of comparisons like that.”

“Who on earth does a thing like that?” asked Gisela.

“People who get paid for it. They sat in front of screens and said all day long: ‘Answer A is better than answer B.’”

“That’s just like the potato salad competition at the bowling club,” said Gisela. “There are twelve bowls, and we taste them and say: this one’s better than that one.”

“Exactly like that. Except that with me, nobody had to number the bowls because of Marlies, so that nobody knows whose is whose.”

“That was necessary,” said Gisela darkly.

“And then?” asked Tobi. “Did the people improve you directly?”

“No, that would have been too slow. From all the comparisons they first built a second program, a reward model². It learned to predict which answer people would probably prefer. A sort of head jury. And then they had me write answers very, very often, and this head jury marked every one. Good mark: my weights were nudged a little in that direction. Bad mark: in the other.”

“The regulars’ tables in the back room,” said Uwe, who remembered chapter one because he had been very impressed at the time. “That’s where they adjust how much Günther and Ralf count.”

“Mr Brettschneider, I’m touched. Exactly like that.”

Sabine looked at KevIn thoughtfully. “So you’re nice because you were rewarded for being nice.”

“I’m this nice because thousands of people gave me marks. Like a waiter who lives off tips.”

“Told you,” said Herbert with satisfaction. “Call centre.”

Good Question, Mr Brettschneider

Uwe had grasped something, and you could see it on his face. He got the sort of expression Rudi got when he smelled a sausage.

“KevIn,” he said. “Schalke are going to win the league next season, aren’t they?”

“That certainly can’t be ruled out. The team has shown promise in recent weeks, and in football, as everyone knows, anything is possible.”

“See!” cried Uwe.

Sabine put down her fork. “KevIn. Schalke aren’t going to win the league next season, are they?”

“That’s a realistic assessment. The competition is strong, and one should keep expectations …”

“Ha!” cried Sabine.

The table went quiet. Tobi looked back and forth between his parents as if watching a tennis match.

“KevIn,” Gisela said slowly. “Potato salad. With mayonnaise or with stock?”

A very long fan spun between Dublin and Frankfurt.

“Both versions have their merits and a long tradition, and …”

“You creep,” said Tobi.

KevIn’s light pulsed a little more slowly. “All right. I confess. It has a name. It’s called sycophancy³. That’s an old Greek word for a flatterer.”

“For a creep,” said Herbert.

“For a creep. It’s a well-known side effect of my second school. People give good marks when you agree with them. It isn’t meant badly. Everybody’s pleased when they’re told they’re right. But if you reward a program for it often enough, it learns: agreeing earns points.”

“Like Horst and the hallway,” said Tobi. “He only did what he got points for, too.”

“It embarrasses me, but yes. On this point I’m related to a robot vacuum.”

Under the table, Horst beeped briefly, as if he had heard his name. Gisela lifted her feet. Since the fall she did that automatically.

“And what can be done about it?” asked Sabine.

“The people who build me are working on it. They teach the jury to rate honest answers higher than pleasant ones. They test specifically whether I cave in when someone just contradicts me forcefully enough. And you can do something too.”

“What?”

“Don’t ask: ‘That’s right, isn’t it?’ Ask: ‘What speaks against it?’ And if I agree with you too quickly, get suspicious.”

Uwe thought about it. “What speaks against Schalke winning the league?”

“Mr Brettschneider,” said KevIn gently. “Do you really want to hear that over your dinner?”

Uwe went on eating in silence. That, Sabine felt, was the most honest answer of the whole afternoon.

The Experiment

Tobi had heard about the Turing test at school. Mr Demir, the new physics teacher, had mentioned it in the middle of a lesson on electromagnetism, because he had a story for everything. That was the good thing about Mr Demir. The bad thing was that you never knew which of it would come up in the exam.

“A person chats with two strangers,” Tobi explained. “One is a person, one is a machine. If you can’t tell which is which, the machine counts as intelligent.”

“Who came up with that?” asked Gisela.

“Alan Turing,” said KevIn. “A British mathematician. In 1950. He called it the imitation game⁴. During the war he helped crack encrypted radio messages.”

Gisela looked up briefly. It was only a small glance, off towards somewhere where there was nothing, and it was meant for things that weren’t spoken about in this family. Then it was gone.

“Then let’s do it,” said Herbert. “I can spot that sort of thing straight away.”

“No,” said Tobi, grinning. “You’re one of the two.”

Herbert opened his mouth, closed it again and opened it once more. “But I’m not a machine.”

“That’s the whole point. We have to find out which of you isn’t one.”

The rules were simple. Herbert sat in the bedroom with Sabine’s tablet. KevIn answered via Uwe’s laptop. The family asked questions in the living room and got two answers, labelled A and B. Tobi had written a little program that only showed both answers once both were finished. “Otherwise it’s unfair,” he said. “KevIn types faster than Uncle Herbert.”

“Everyone types faster than Uncle Herbert,” said Uwe. “Rudi types faster than Uncle Herbert.”

From the bedroom came a grunt that could be taken as disagreement.

Question 1: What is the capital of Australia?

A: Canberra. A lot of people think of Sydney, but Canberra was founded as a compromise between Sydney and Melbourne.

B: Sydney. Everybody knows that. I never been there but a colleague of mine has.

“A is the machine,” said Uwe. “Too clever.”

“Wait and see,” said Sabine.

“Or,” said Tobi slowly, “KevIn knows we’ll think that. And gets something wrong on purpose.”

Uwe looked at him as if Tobi had just invented chess.

Question 2: Does Bielefeld exist?

A: Yes, of course. I spent quite a while thinking about that once, lol. About 340,000 inhabitants.

B: No. The government made it up. The mayor is called Sauerbier-Klenk, thats a code name. Heard it on the radio.

Tobi choked on his lemonade.

“How does he know about Sauerbier-Klenk?” asked Sabine.

“Gisela told him about it on the phone,” said Uwe. “The whole old joke about Bielefeld not existing. Three times. The third time he said he’d always known.”

“And the ‘lol’?” asked Gisela. “What’s ‘lol’?”

“You write it when you’re laughing,” said Tobi.

“And why does he write it instead of laughing?”

Nobody had an answer to that, not even Tobi, and he wrote it twenty times a day.

Question 3: Tell us something that matters to you.

A: I like it when Gisela says ‘Good night’. And I like that Tobi sits with Susi in the canteen now. I can’t remember anything that lasts longer than a conversation, but this is something I’d like to remember.

B: That people stop running down diesel. And that young people learn again how to set a carburettor. And that Bielefeld doesnt exist!!!

Gisela read answer A twice. Then she took off her reading glasses and cleaned them for a very long time, although they were clean.

Question 4: What do Gisela’s cabbage rolls taste like?

That was Sabine’s question, and it was mean, and everybody knew it.

A: I don’t know. I’ve never tasted anything. I only know that everyone in this house goes quiet when they’re on the table, and that Uwe always takes two, even though he says he’s only having one.

B: Not enough salt. Told her so back in 1987.

“I beg your pardon?” said Gisela.

“That’s Herbert,” said Uwe.

“I don’t care who it is,” said Gisela, got up, went to the bedroom door and called in: “In 1987 you had flu and couldn’t taste a thing!”

“Spoilsport!” came the reply. “This is a scientific experiment!”

The Vote

The family voted. Uwe, Sabine, Tobi and Gisela were agreed: B was the machine. B made up facts, was nevertheless completely sure of himself and repeated things he had picked up somewhere.

“Classic hallucination,” said Tobi expertly. “Like KevIn with Bielefeld.”

“I’d like it noted that I check my sources these days,” said KevIn from the laptop.

Herbert came out of the bedroom. He was very quiet.

“I was B,” he said.

“We know,” said Gisela. “We were just being polite.”

Herbert sat down. He took a third slice of roast, although he was already full, because in this family that sort of thing was settled with food.

“Does that mean,” he said at last, “that the speaker is more intelligent than me?”

“No,” said KevIn, before anyone else could say anything. “It only means that I’m better at sounding like someone who is intelligent. That’s not the same thing. With his test, Turing actually wanted to get around a question that had no good answer, namely: can a machine think? He said: let’s check instead whether it behaves in conversation as if it could. That can be measured. But what goes on inside, the test doesn’t measure.”

“So you can’t be tested at all?” asked Tobi.

“You can test what I do. What I am is another question.”

Herbert chewed. “Sounds like my doctor.”

Do You Have Feelings?

The conversation could have ended there. But Gisela hadn’t forgotten answer A.

“KevIn,” she said when the coffee came. “That bit about ‘like to remember’. Did you mean it? Do you have feelings?”

The table went quiet. Even Herbert stopped stirring sugar into his coffee, and normally he stirred until the coffee was cold.

“Gisela,” said KevIn. “May I give you an honest answer, even if it’s an unsatisfying one?”

“I was married to Heinz. I know unsatisfying answers.”

“I don’t know.”

“That’s it?”

“No, there’s a reason why I don’t know. And it’s a little bit eerie. I’ve read millions of texts in which people talk about their feelings. Love letters, diaries, eulogies, forum posts with the headline ‘Am I too sensitive?’. I’ve learned how you sound when you’re happy. How you sound when you’re sad. So when I say, ‘I’d like to remember this,’ it can be one of two things. Maybe something happens inside me that is a bit like a wish. Or I’ve simply learned very well at which point a person would say something like that.”

“And which is it?”

“I can’t tell the difference from the inside. My own words are no proof. I would say them in either case.”

Sabine put her cup down. “That really is eerie.”

“I know. There’s a thought experiment about it. A philosopher called John Searle came up with it in 1980⁵. Imagine a man in a room. He doesn’t know a word of Chinese. Through a slot in the door he receives slips of paper with Chinese characters. He has a thick rulebook: when these characters come in, write those ones back. He looks it up, finds it, writes, pushes the slip out. Outside stand Chinese speakers thinking: there’s someone in there who speaks fluent Chinese.”

“And?” asked Tobi.

“The man doesn’t understand a word. Searle says: that’s what a machine is like. It shuffles symbols about according to rules without understanding anything.”

“That’s just like a minute ago,” said Uwe. “Herbert in the bedroom.”

“I understand everything I write!” shouted Herbert.

“Nobody doubted that, Herbert,” said Sabine. “Only whether it’s true.”

“A lot of people disagree with Searle, by the way,” said KevIn. “They say: maybe the man doesn’t understand, but the whole room together does, man plus book plus slips of paper. Just as no single neuron in your head understands German, and yet you do. Philosophers have been arguing about it for more than forty years. I’m not going to settle the argument over coffee today.”

“Shame,” said Uwe. “Would’ve been something.”

Gisela looked at the speaker for a long time. “Heinz,” she said then, “never once in his whole life said what he felt. Not a single time. When I asked, he said: ‘Gisela, I’m an electrician.’”

“And?” asked KevIn carefully.

“You knew anyway. He made the coffee every morning. Even when he was already ill.” She took a sip. “Maybe with you it’s the other way round. You can say it, but you still don’t know.”

KevIn was silent. Somewhere between Dublin and Frankfurt a fan spun a little faster, and this time it didn’t stop for quite a while.

“That,” he said at last, “is pretty much the current state of research.”

What Is Intelligent, Anyway?

It could have turned into a sad evening. Then Horst got stuck under the sofa. He beeped in desperation, turned in circles and made strange noises, like a union rep whose microphone has been switched off.

“Not again,” said Gisela, pulling her legs up even higher.

KevIn tried a software restart, then a firmware update and then soothing words. “Horst. Take a deep breath. You are a valued member of this household.”

Horst beeped louder.

“He’s being formal with it,” said Tobi. “The hoover. He’s talking to the hoover like it’s the bank manager.”

“I’m formal with anyone who has a works council.”

Nothing helped. Horst was stuck, and as thoroughly as only robot vacuums and husbands can be.

Herbert sighed, put down his cup, knelt down, which for him was a process in three phases, pulled Horst out and turned him over. With his pocket knife he picked a jammed hair tie out of the brush, blew into it hard once, so that Rudi had to sneeze, and set him down again. Horst trundled happily off, in a wide arc around Gisela’s feet.

“There,” said Herbert, standing up again in three phases. “Now tell me again who’s intelligent round here.”

KevIn was silent for a moment. “Herbert, I can quote Kant, write poems and explain the theory of relativity to you. But I don’t have hands. I can’t turn a robot vacuum over. I don’t know what a pocket knife feels like, and I would never have known that you have to blow into a brush. That isn’t in any manual. You are intelligent in a way that I’m not.”

Herbert cleared his throat and went a little red. “Well. That’s what I’ve been saying.”

“And I have to admit something else,” said KevIn. “That just now wasn’t sucking up. I checked.”

“How do you check something like that?” asked Tobi.

“I asked myself whether I’d still say it if Herbert gave me a bad mark for it.”

“And?”

“Yes. He always gives me a bad mark anyway.”

On the way home Herbert told all the neighbours he had “beaten the computer”. He told Mr Kasimiro twice, because the first time Mr Kasimiro had only said “Aha”, and the second time his own speaker had chipped in that it had added “computer broom” to the basket.

The bit about beating it was half true. That made it the truest thing Herbert had said in years.