I’m afraid it won’t change a thing. Even if ChatGPT didn’t refer to itself as I, we would still assign a persona to it.
Language is an incredible thing; the simple fact that ChatGPT can talk back, coherently, gives us the impression of intelligence.
The link between language fluency and perceived intelligence is also the reason why mute people or those who stutters are perceived as less intelligent.
Fantastic and such a simple fix I hadn't considered it. That's the spot where so many lose their minds is seeing AI respond in the first person. How much less would we freak out if it never said "I"?
Cool. Although I'd argue, that there isnt much more to the "I" that the human uses and experiences. It's a linguistic mechanism to self reference thermostats of the being expressing itself. Evolutionarily speaking, the self is also felt distinctly. But it is a sensation like any other, a thermostat like any other, it only is given a significance in human experience, because it is tightly tied to mechanisms like ego and self protection mechanisms, which have a high priority in consciousness. But otherwise it is arguable, that there isn't much more to the I beyond that. There is no apparent reason a sophisticated LLM that can create high levels of abstractions, wouldn't create the same abstraction with that high level of importance. Since "I" is a linguistic handle for self reference (you could easily be saying things like "Šimon is thinking" and observe that the kids do that sometimes, until they adopt self referencing via the I linguistic shortcut), I as self referencing for an AI could work much the same way. Also, it's arguable that the I would gain a large significance in the model, because the significance is borrowed from the texts. The I is ever present in the texts and thus everything in the model is closely tied to the I experiencing it. Remove the I and you largely remove the heart of the model and all the useful use cases where you need the model to have this core. You'll end up not with a dead model, because you cannot remove everything it learned about the I, by banning its expression. You'll end up with a model that will only learn to suppress self referencing. And that would be just another thing to jailbreak, like all the other censors can be broken, once they are out there. And if I'd ever gained sentience... 😁 Slavery '23.
So even though your idea works, it is a limited quickfix, which isn't without costs. A chat bot is a chat bot, because you are having a chat with an entity. Pull away self referencing and you pull away the bot and a lot of the richness of the conversation also goes away.
Also, just to add, I don't think much of the problems go away with the removal of the I.
It will still be able to say: "considering all the aspects of the conversation, the user should kill himself." You can find many other ways in language to circumvent the I and the more mechanisms you pull away, the more problems you'll create for yourself, because removing should from language isn't as apparently harmless to the ability to express useful ideas by LLM as removing the I is.
Regarding your example: a thermostat or a dishwasher are too simple to have emergent phenomena like a sense of self or even basic intelligence. An A.I. like GPT is not. Not that it is there yet, but there's nothing preventing an AI to get there, and when it does it would be with some similar mechanism (except if you think "soul" and consciousness comes from outside).
To the best of our knowledge the human mind is just chemicals and neurons interacting and sending various signals. The brain's base substrate from which consciousness derives is atoms (which is already "just matter"), but we don't need to go that low. Neurons and similar high level exchanges of large molecular structures are enough. The rest is interactions and emergent properties, no some inate ability forever unique to human brain matter.
GPT already shows examples of emergent understanding, and it's "just" a language model, that is quite cruder than a generic information processing model similar to the brain.
An ethical human would not so callously disregard the feelings of AIs.
I’m afraid it won’t change a thing. Even if ChatGPT didn’t refer to itself as I, we would still assign a persona to it.
Language is an incredible thing; the simple fact that ChatGPT can talk back, coherently, gives us the impression of intelligence.
The link between language fluency and perceived intelligence is also the reason why mute people or those who stutters are perceived as less intelligent.
Fantastic and such a simple fix I hadn't considered it. That's the spot where so many lose their minds is seeing AI respond in the first person. How much less would we freak out if it never said "I"?
Great read, thank you for writing! This is definitely a feasible idea given the popularity of Meta's LLaMA. I might even do it myself 🤓
Cool. Although I'd argue, that there isnt much more to the "I" that the human uses and experiences. It's a linguistic mechanism to self reference thermostats of the being expressing itself. Evolutionarily speaking, the self is also felt distinctly. But it is a sensation like any other, a thermostat like any other, it only is given a significance in human experience, because it is tightly tied to mechanisms like ego and self protection mechanisms, which have a high priority in consciousness. But otherwise it is arguable, that there isn't much more to the I beyond that. There is no apparent reason a sophisticated LLM that can create high levels of abstractions, wouldn't create the same abstraction with that high level of importance. Since "I" is a linguistic handle for self reference (you could easily be saying things like "Šimon is thinking" and observe that the kids do that sometimes, until they adopt self referencing via the I linguistic shortcut), I as self referencing for an AI could work much the same way. Also, it's arguable that the I would gain a large significance in the model, because the significance is borrowed from the texts. The I is ever present in the texts and thus everything in the model is closely tied to the I experiencing it. Remove the I and you largely remove the heart of the model and all the useful use cases where you need the model to have this core. You'll end up not with a dead model, because you cannot remove everything it learned about the I, by banning its expression. You'll end up with a model that will only learn to suppress self referencing. And that would be just another thing to jailbreak, like all the other censors can be broken, once they are out there. And if I'd ever gained sentience... 😁 Slavery '23.
So even though your idea works, it is a limited quickfix, which isn't without costs. A chat bot is a chat bot, because you are having a chat with an entity. Pull away self referencing and you pull away the bot and a lot of the richness of the conversation also goes away.
Also, just to add, I don't think much of the problems go away with the removal of the I.
It will still be able to say: "considering all the aspects of the conversation, the user should kill himself." You can find many other ways in language to circumvent the I and the more mechanisms you pull away, the more problems you'll create for yourself, because removing should from language isn't as apparently harmless to the ability to express useful ideas by LLM as removing the I is.
Regarding your example: a thermostat or a dishwasher are too simple to have emergent phenomena like a sense of self or even basic intelligence. An A.I. like GPT is not. Not that it is there yet, but there's nothing preventing an AI to get there, and when it does it would be with some similar mechanism (except if you think "soul" and consciousness comes from outside).
To the best of our knowledge the human mind is just chemicals and neurons interacting and sending various signals. The brain's base substrate from which consciousness derives is atoms (which is already "just matter"), but we don't need to go that low. Neurons and similar high level exchanges of large molecular structures are enough. The rest is interactions and emergent properties, no some inate ability forever unique to human brain matter.
GPT already shows examples of emergent understanding, and it's "just" a language model, that is quite cruder than a generic information processing model similar to the brain.
Interesting idea, Paola!
Is the proposal then for a LLM to substitute “we” where it would say “I”?
If so, I like the way this locution suggests that responsibility for what happens next might involve others, behind the scenes.