‘Bias’, ‘Companion’, ‘Hallucinations’: three examples to understand the significance of the terminology used in and around AI

More generally, language influences the way we frame problems. But before we can seek a solution or an answer, we first need to agree on the question itself. Let’s be honest: are we really capable of understanding the reality of technology clearly enough to achieve this? Especially when the words we use sometimes tend to obscure that reality rather than help us understand it?

Arnaud Billion: Certain terms commonly used in the field of artificial intelligence influence strategic decisions, whilst also highlighting the errors and weaknesses of the conceptual frameworks within which they are embedded.

For example, it is often claimed that AI is biased (in terms of its data or its algorithms (2)) in order to consider how to correct this or at least control the harmful consequences of these limitations. For instance, due to the training data, we know that AI-driven CV selection heavily excludes statistically under-represented groups, such as women (3) or older people (4). An AI system used by an organisation’s human resources department would therefore be ‘discriminatory by design’.

Björn Fasterling: I agree with Arnaud that, in a technical context, ‘bias’ simply refers to a formal property assessed against a given benchmark. Similarly, ‘discrimination’ can simply mean distinguishing between different elements. This is something that any digital decision-making system does, as it classifies, selects or processes situations differently according to certain criteria.

Arnaud Billion: Yet if every choice is discriminatory, then we would have to resort to drawing CVs at random – a task which, incidentally, AI would perform very well. The point is that the concept of bias is rather pointless, as it encourages us to seek a ‘normal’ answer. However, in many fields, this ‘normality’ is in fact conventional and arbitrary, not scientifically grounded.

Björn Fasterling: Returning to the terms ‘bias’ and ‘discrimination’, these two terms are also commonly used in a normative sense. The moral principle of non-discrimination requires that every person be treated with the same dignity and respect (5). It prohibits any unjustified differential treatment based on protected characteristics such as skin colour, gender, religion or age… .

In this normative sense, ‘discriminatory bias’ constitutes a deviation from this moral standard. Confusion arises when these different meanings of ‘bias’ become conflated. A technical bias may be mistaken for an injustice. Unjust discrimination may be reduced to a technical flaw. Well, everything then ends up becoming a ‘bias’, which means we can continue the discussion without really knowing what we’re talking about any more…

Arnaud Billion: Ultimately, we are entitled to ask ourselves: hasn’t this notion of ‘AI bias’ primarily served as an argument for developing and selling new (AI) software designed to combat bias? In short, numerous solutions to a problem that is poorly understood.

Arnaud Billion: What a sense of security it is to have an AI companion by your side – a highly powerful and knowledgeable entity, like Dr Faust’s Mephisto – who will help me with my administrative tasks, organising my life and my business, or simply support me in my social and emotional life!

Unfortunately, the AI companion, the AI assistant or the AI agent are nothing more than mirages; worse still: mere facades. One need only remember that ‘AI’ is merely software – or even less than that: a piece of code corresponding to an electronic procedure (6).

‘AI’ is the name we give to a fiction that conceals value chains orchestrated by digital giants who, ultimately, seek to generate profits from individuals or businesses. The problem with positing and believing in an AI companion is that it assumes an entity within the computer will obey me and defend my interests. Yet we have no credible reason to assume this, and we should question the series of misunderstandings that have led us to this belief.

Björn Fasterling: So this “companion” doesn’t really resemble someone with whom we’d share a meal. Perhaps it’s more like a catheter inserted into our brain. Behind this companion - this seemingly singular access point (7) - lies an entire distributed infrastructure, made up of machines, programs, permissions, and institutional arrangements. Through this catheter, the infrastructure extracts information from our environment and injects content into it to trigger our actions. Yet, as if by magic, all this complexity disappears behind the illusion of a single assistant, who stands by your side and shares with you the burden of existence.

Arnaud Billion: Beyond that, all these anthropomorphic concepts (learning, hallucination, memory, attention, etc.) are particularly misleading and reflect an irrational and persistent belief, a belief fostered by powerful messaging, or even propaganda, toward which many are very gullible, and which we would do well to question.

Arnaud Billion: When we receive a response from the AI that seems clearly wrong to us, or that we find unsatisfactory, we often refer to it as an “AI hallucination” (8). It’s a bit as if the AI had made a mistake, had been fooled by a deceptive appearance, and was providing us with incorrect information. Technically, however, there is no error or bug present at the moment of the so-called “hallucination”: on the contrary, the calculation proceeds perfectly.

Björn Fasterling: To hallucinate is to experience something that isn’t there. But there’s no reason to believe that an AI experiences anything at all. It’s, in a way, a hallucination squared. We’re hallucinating an AI that’s hallucinating.

Arnaud Billion: So what happens then? It’s precisely that we stop fantasizing about a “truth-telling machine”; we have a chance to remember that the output of the software suite called “AI” will simply be a response calculated based on numerous criteria (9) (potentially reusing my profiling data, my purchase predictions, maximizing my engagement, executing real-time ad targeting, seeking to influence my next vote, etc.)

Above all, a prompt is nothing more than a computer command designed to trigger extremely sophisticated calculations - but there is little reason to believe these calculations will be appropriate (10). In any case, this is a far cry from the everyday computing experience of searching a database configured to provide the expected answer.

Björn Fasterling: You’re talking about the prompt: now that’s an honest term, one that describes almost exactly what’s going on. In theater, the person we call the “souffleur” in French is known as “the prompter” in English. The prompter whispers the lines to the actor, but doesn’t perform in the play. With generative AI, it’s now the human who provides the “prompt.” In other words, the human whispers the line to the massive computing infrastructure, and it’s the infrastructure that takes the stage!