The Delusion Machine – What Happened When I Fed My Soul into an LLM

The author's point could be made far more succinctly and clearly than this. I found the writing indulgent, overcomplicated and unfocused.

Overwrought prose (so many lists of three or more things! Why?) that takes fully half of its word count to get to the point, which is that LLMs are appeasing and solicitous by default. Not sure what else I'm supposed to take away.

LLMs are decent at providing feedback and validation. I often have few sources of that from humans, and long periods without it are bad for my motivation and sense of well-being.

LLMs filling that hole is great if it's done in discrete and intermittent bumps. TFA shows the psychological risks of binging on artificial validation.

All things in moderation, especially LLMs.

I’m always in awe of characters like the author. When they recount their lives, it feels like watching someone narrate a mythological curse. Just enough introspection to map the pattern of their own compulsion and inevitable failure, yet not quite enough to grasp what would actually sever the loop and set them free.

On one hand, she sees with piercing clarity her inability to take hints, read the room, or navigate social subtext. On the other, she holds to this undeniable conviction that rejection itself will eventually sculpt her into something new, rather than seeing it as a painful signal to do the reshaping herself.

That conviction fascinates me. Where does it come from? An unshakeable sense of justice? A craving for recognition and boundaries that she cannot receive from other people? Or some deeper? Has the author always behaved like this or are there mile markers in the development of her beliefs?

Humans have been feeding their souls into a lot of things throughout history, LLMs are just another vessel to carry it.

I don't think it succeeds very well as fiction, but the concept is worth further exploration. I've been very disturbed recently to see AI avatars of dead people participating in TV interviews, funerals, and even court cases. To me this is the technological equivalent of inviting in a necromancer or ouija board operator and treating their pronouncements as facts. This can only result in a social environment hostile to rationality.

I think it'd be well worth performing the experiment of creating some people's digital avatars and then testing them against the original to see how faithful of a model they are.