Why AI sometimes sells solemn-sounding nonsense

Soon I'll give a talk for our church council about AI. Great that they want to engage with this (and yes, also pretty brave of me to just do this 😂).

I was looking for an example to show that AI is not a parrot that repeats sentences literally from a database, but is creative. So I asked a question I assumed had no ready-made answer in the databases/online: "Why does a church elder wear a black sling?" (One of the men wears one after an accident.)

The answer I got was very solemn. According to ChatGPT, the black sling comes from the Protestant tradition, especially within the reformed community. It would fit with the academic gown and with the custom of dressing church officials soberly and recognisably. Such a garment would help guard the boundary between person and calling, especially in a time when everything has to be personal and expressive.

This is a typical case of what we call "hallucinating": when AI sells nonsense that sounds good.

This morning that very term was in the news as word of the year, because it has gained a new meaning since the arrival of AI.

AI's hallucinations have little to do with human hallucinations. AI doesn't really understand anything, but is actually just a big calculator that works with words and probabilities.

In this case the system calculated that words like church official, black, clothing, reformed, soberness and calling often occur together. So that sling would fit right in too. You could say AI made a small calculation error here.

Usually the answers are correct, but it's good to realise that hallucinations are part of AI; they keep you sharp about the truth. (We humans need that sharpness too, by the way.)

And should you actually see that sling widely on Sunday after all, do let me know. Then I was mistaken.