@datarama Yes! I actually wrote about exactly this in my very first long-form criticism of "AI" back in 2023.

«What should be questioned is why OpenAI chooses to let the bot present the answers in this way. It is entirely according to the company's design. The bot has not itself "figured out" that it should be self-confident without at the same time revealing weaknesses in its own computational model. That has been OpenAI's decision. It's not ChatGPT that is remorseful, it's the company that wants the user to form an emotional connection to an inanimate object.

The illusion has largely succeeded. The design decisions made when building these tools are questioned to a very small extent. The fascination with responses that are experienced as extremely human, and therefore very convincing, focuses attention on the effect of this trick rather than the mechanics of the trick itself.

"Look over here!", as the magician would say.»

https://axbom.com/ai-responsibility/