Human-AI echoborgs make chatbots more real, but still fail Turing test - Ars Technica

Artificial intelligence research is still quite far from developing chatbots that seem convincingly human. But how much does the delivery of the chatbot’s chat affect the result? What would happen if chatbot responses were delivered by a real human?

Researchers at the London School of Economics and Political Science have designed a series of experiments that test whether human delivery makes a difference in how people perceive an artificial intelligence system. The experiments, published last week in the journal Frontiers in Psychology, investigated this problem by using an “echoborg.” This is a person who acts as a human front for a chatbot: they say out loud the chatbot’s response to each input.

The echoborg is an artificial intelligence-themed adaptation of the “cyranoid” technique used in various psychology experiments, which sees one person shadow the speech of another. It turns out that shadowing speech (repeating exactly what someone else has said, while someone else is saying it) isn’t very hard for people. They’re able to do it with a delay of only a few hundred milliseconds, and they even automatically mimic features of the original speech like stress and pitch. This means that shadowed speech doesn’t sound noticeably different from the real thing.

This technique allows experimenters to tease apart the content of a conversation from the person who delivers that content. For example, if the shadower remains constant (say, a white male adult) while the conversational content is supplied by a variety of people (say, black vs. white, male vs. female, adult vs. child), it become possible to work out which parts of someone’s reaction are determined by who they’re speaking to, compared to what is said.

In the case of the echoborg, the technique helps to separate what effects are caused by content generated by a computer, compared to content delivered by a computer in text format.

The echoborg Turing test

The first echoborg experiment conducted by the researchers was a Turing test, designed to work out whether human delivery made it harder to spot a chatbot, or made the chatbot seem more human. Two experimenters were kitted out with in-ear monitors, and trained to shadow speech convincingly. These experimenters shadowed the responses read out by a researcher in a control room.