More thoughts on chatbot design; TruthGPT, really?
"It would be more ethical to design chatbots to be noticeably different from humans."
One would need a good AI-powered assistant just to keep up with the daily flood of news about AI-powered assistants. My purpose is not to help you keep up (I’m human, after all), but to select and cherrypick ideas within the flood that perhaps aren’t as prominent in the prevailing discourse about AI as I think they should be. So, let me make the assumption that my readers have already read Ian Hogarth’s FT essay titled “We must slow down the race to God-like AI”, which has lots to recommend about it, except that I take issue with the “God-like AI” label, where the contextual shift from science to religion probably does no one any good.1
More thoughts on chatbot design and model purpose
If you agreed with me that LLMs should be configured and trained not to use the first person in answering prompts by humans, you will also enjoy this Nature piece by philosopher Carissa Véliz: “Chatbots shouldn’t use emojis”. Underestimated due to their apparent triviality, emojis are in fact another tool of language that machines, in aping humans, can employ to seem more human-like:
Limits need to be set on AI’s ability to simulate human feelings. Ensuring that chatbots don’t use emotive language, including emojis, would be a good start […] Humans lie and manipulate each other’s emotions all the time, but at least we can reasonably guess at someone’s motivations, agenda and methods. We can hold each other accountable for such lies, calling them out and seeking redress. With AI, we can’t […] My worry is that, without appropriate safeguards, such technology could undermine people’s autonomy.
As I noted earlier this month, we should also refrain from saying that a chatbot “hallucinates”. Giada Pistilli at Hugging Face is quoted in Bloomberg’s Tech Daily on this same topic, saying: “Language models do not dream, they do not hallucinate, they do not do psychedelics.” Rachel Metz comments:
In this case, the term “hallucinate” obscures what’s really going on. It also serves to absolve the systems’ creators from taking responsibility for their products. (Oh, it’s not our fault, it’s just hallucinating!)
More generally, as Google engineer Dana Fried wrote here and here, it might be worth again reflecting on what we’re asking LLMs - or even less advanced models - to do for humans:
My wife says that any time someone proposes doing anything with an ML model, you should replace “AI” in the proposal with “trained weasels” and if it still sounds like a good idea you can go ahead with it.
“We trained weasels to count how many customers came into the store” - reasonable; unproblematic; probably a thing you could do.
“We trained weasels to screen resumes of people applying to jobs” - probably a bad idea, especially if the weasels are racist or sexist (the weasels are both racist and sexist).
“We trained weasels to drive a two-ton vehicle on busy city streets” - terrible idea; someone is almost certainly going to die.
Now, let me take that one step further. Suppose somebody told you “We trained weasels to pretend they’re your kid’s best friend and keep your kid glued to the smartphone screen several hours a day chatting to, and exchanging pictures with, a bunch of weasels who were tasked with maximizing the time your kid spends on the app and the ad revenue your kid generates for the owners of the weasels, regardless of the fact that the trained weasels are not your kid’s best friend, and actually have no concept of friendship whatsoever.” That, indeed, is what Snapchat announced this week, when it opened its MyAI chatbot for free to all users (see TechCrunch, along with the Washington Post for background). Now, Snapchat last reported 375 million daily active users (DAUs), making its user base larger than Twitter’s (notwithstanding the disparity in press coverage). If Snapchat’s posing-as-best-friend-trained-weasels prove a hit with teenagers, TikTok and Instagram will no doubt be under pressure to introduce their own similarly trained weasels. What could go wrong? (I’m looking if I can find an emoji here to underline that I’m being sarcastic, but you probably get the drift anyway).
Now, where do you stand on safeguards to protect vulnerable users such as kids, suicidal people, and the elderly? Where do you even draw the line? Who isn’t vulnerable? A blanket self-regulation agreement for guardrails against AIs pretending that they’re human (saying “I”, using emojis, etc.) would be a Pareto gain here - we’d all be better off, with no one worse off.
TruthGPT, really? and a short piece I wrote in 2010
Elon Musk has gone on the record with his claim to be working on TruthGPT: go figure. 🤷♀️🤷♀️🤷♀️
The news reminded me of a short essay I wrote for myself and my friends in the publishing industry back in January of 2010, as I became aware that the Internet was flooded with all sorts of doubtful material and that news sources could and should strive for fact-checking accuracy, but readers had better remain vigilant, and finding “truth” on the internet was perhaps too lofty an aspiration even for the best editors and curators. At the time I illustrated my rather lyrical attempt to convey this idea with a scene from Lorenzetti’s Siena frescoes from 1338-39 depicting the effects of Good Government. Here, instead, find below another scene in the same cycle, that of Bad Government, depicting Elon… ooops, I mean the Tyrant, surrounded by Greed, Pride, Vainglory, Deceit, Fraud, Fury, and Division. Anyway, here’s the piece (2010): today, the megalopolis I evoked is probably a decent metaphor for the training data fed to most LLMs.
The Internet is a megalopolis
The Internet is like a city. A big city.
As long as we only had the telegraph, telephone, newspapers and television, it was as if we all lived in the country.
Now that we have the Internet, we live in the city.
A huge, chaotic, sprawling city: an Internet that is as different from that of the pioneers as the Shanghai of today is different from the Athens of the fifth century before Christ. But still a city. A place with streets, squares, boulevards, stores, cinemas, parks, sewers, cemeteries. If we have been there for a while, we can read the signs and move around nimbly. If we are new, like the illiterate farmer who just sold cows and pigs to live in a slum with tin roofs, we equally learn to read the city: this is the palace of power, this is the market square, this is the red light district, this is the garbage dump. We will have a harder time finding our way around at first, but we rarely go back.
People come to the Internet for the same reasons they have been coming to the city for centuries and in numbers that have taken off exponentially since the Industrial Revolution: economic opportunity, trade and exchange, knowledge, creativity, sociability, entertainment. And for the same reasons they will continue to arrive.
We do not ask cities for truth. We demand justice from the courts, we demand beauty from museums, we demand knowledge from universities, we demand truth from religions. Every great city is made up of honest people and snake charmers, of the powerful and the lepers, of cathedrals and rubble, of stages of the sublime and banlieues with burning cars, of gestures of everyday goodness and corpses dissolved in acid, of cowards and heroes.
An Internet of Utopia would live in peace and prosperity, like the city Ambrogio Lorenzetti painted in the Allegory of Good Government in the Sala dei Nove at the Palazzo Pubblico in Siena. An Internet of Dystopia would live in fear under tyranny, like the city painted on the opposite wall.
Neither is a real city. For millennia civilization has wondered about good city government: the solution that leads to perfect peace and prosperity has yet to be found. Are we surprised, then, that the great city we call the Internet has not found its solution?
Further reading
The Economist on Digital Poisons: LLMs multiply the poisoning of the Internet.
A delightful essay by Jill Lepore in the New Yorker: The Data Delusion. “Imagine that all the world’s knowledge is stored, and organized, in a single vertical Steelcase filing cabinet […] It’s got four drawers. […] The drawers are labelled, from top to bottom, “Mysteries,” “Facts,” “Numbers,” and “Data.”” Must read.
Aside from the rhetorical confusion from shifting planes, I seem to recall that even many religious people, to my knowledge, don’t believe in an interventionist God.