The Case That A.i. Is Thinking

Obviously, it is different or else we would just use JIRA and a database to replace GPT. Models very obviously do NOT store training data in the weights in the way you are imagining.

>So basically if you have that stance you'd have to agree that when we FIRST invented computers, we created intelligence that is "thinking".

Thinking is by all appearances substrate independent. The moment we created computers, we created another substrate that could, in the future think.

“So if the severed head isn’t alive, the disembodied heart isn’t alive, the jar of blood we drained out isn’t alive - then what are you saying is alive and thinking?”

- Some silicon alien life forms somewhere debating whether the human life form they just disassembled could ever be alive and thinking

In practical terms it does not matter - it is reasonable for us to act as if others do experience the same we do. But if we are to talk about the nature of conscience and sentience it does matter that the only basis we have for knowing about other sentient beings is their self-reported experience.

How do you know that model processing text or image input doesn't go through feeling of confusion or excitement or corrupted image doesn't "smell" right for it?

Just the fact that you can pause and restart it doesn't mean it doesn't emerge.

But asked for reasons I can only point to the social nature of their societies, where love and anger make sense, or of their hurt-behavior.

I also find it very hard to believe that everything else is slow evolution of components, and here all of a sudden something super complex comes into being out of nowhere.

But I still have no idea how it could work. What are the components and their interplay?

Either it is (and continues to be) the same consciousness, or it is not. If it were the same consciousness, then you would have a person who exists in two places at once.

does not mean that the essential thing gives rise to consciousness is only approximate. To give an example from software, you can write software is robust against bad input, attempts to crash it, even bit flips. But, if I came in and just changed a single character in the source code, that may cause it to fail compilation, fail to run, or become quite buggy.

> If you don't want that to be true, you need some kind of magic,

This is just what I'm saying is a false dichotomy. The only reason some are unable to see beyond it is that we think the basic logic we understand are all there could be.

In this respect physics has been very helpful, because without peering into reality, we would have kept deluding ourselves that pure reason was enough to understand the world.

It's like trying to explain quantum mechanics to a well educated person or scientist from the 16th century without the benefit of experimental evidence. No way they'd believe you. In fact, they'd accuse you of violating basic logic.

The bound informational dynamic that supervenes on the activity of the individual units in the cluster. What people typically miss is that the algorithm when engaged in a computing substrate is not just inert symbols, but an active, potent causal/dynamical structure. Information flows as modulated signals to and from each component and these signals are integrated such that the characteristic property of the aggregate signal is maintained. This binding of signals by the active interplay of component signals from the distributed components realizes the singular identity. If there is consciousness here, it is in this construct.

Of course, "the consciousness" is a nebulous concept. It would be like asking "which part of my processor is Windows" to some extent. But it's still fair to say that Windows is contained within my computer, and that the metal framing of the computer is not part of Windows.

But yes, we built a machine that generates text similar to what we built it from, and now we're looking at it generating text and are all impressed.

A scientific paper has to be verifiable, you should be able to recreate the experiment and come to the same conclusion. It's very very difficult to do with brains with trillions of parameters and that can't be controlled to the neuron level. Nothwithstanding the ethical issues.

We don't have a world weather simulator that is 100% accurate either given the complex interplay and inability to control the variables i.e. it's not verifiable. It'd be a bit silly to say we don't know why it's going to rain at my house tomorrow.

Until then it is a hypothesis, and we can't say we know even if the overwhelming evidence indicates that in fact that we do know.

Actually, the way LLMs are extended with tools is a pretty much the same (an LLM itself has no access to the internet, but if it returns some specific symbols, the external "glue" will do a search and then the LLM is free to use the results)

Yes, with an emphasis on can. That does not mean they necessarily will. Though I would consider it unlikely that they won't, we the only way of proving that they will would be to do it.

> But if that is the right chain of assumptions, there's lots of issues with it. First, whether LLMs are Turing complete is a topic of debate. There are points for[0] and against[1].

It's trivial to prove that a system comprised of an LLM with a loop is Turing computable. A single inference step can not be Turing computable, but one with a loop only requires the LLM to be capable of executing 6 distinct steps with temperatur set to 0. You can wire up a toy neural network by hand that can do this.

This is in fact a far more limited claim than what the paper you linked to makes.

The article you linked to, on the other hand, is discussing if an LLM can act like a Turing machine without that loop. That is why "state management" matters. State management is irrelevant when you wrap a loop around because you can externalise the state, and you only need 2 states (and 3 symbols, or 3 states and 2 symbols) for the smallest known universal Turing machine.

The entire article is thus entirely irrelevant to this question. Sure, you will struggle to make an LLM act as a Turing machine without going "off the rails". But that is irrelevant - you only need it to be able to execute one state transition by deterministically producing the right next tape operation and next state when given the next symbol and current state.

From that you can build up to arbitrarily complex computation, because for every given Turing machine, you can construct a larger Turing machine that uses additional symbols or states to encode an operation that takes multiple steps for the smaller machine.

> The processor I'm using to write this might be able to perform all the computations needed for human intellect, but even if it could, that doesn't mean it can do it quickly enough to compute even a single nanosecond of actual human thought before the heat-death of the universe, or even the end of this century.

Irrelevant, because absent physics that exceeds the Turing computable, the human brain is an existence-proof for the possibility of computing everything the human brain does in a package the size of an average human brain.

It is very likely that we will need architectural changes to compute any given model efficiently enough, but to suggest it is not possible is an extraordinary claim not supported by anything.

If you take LLM to mean a very specific architecture, or specific computational methods to execute a model, then you have a point. If so we are talking about very different things.

Many consider "thinking" something only animals can do, and they are uncomfortable with the idea that animals are biological machines or that life, consciousness, and thinking are fundamentally machine processes.

When an LLM generates chain-of-thought tokens, what we might casually call “thinking,” it fills its context window with a sequence of tokens that improves its ability to answer correctly.

This “thinking” process is not rigid deduction like in a symbolic rule system; it is more like an associative walk through a high-dimensional manifold shaped by training. The walk is partly stochastic (depending on temperature, sampling strategy, and similar factors) yet remarkably robust.

Even when you manually introduce logical errors into a chain-of-thought trace, the model’s overall accuracy usually remains better than if it had produced no reasoning tokens at all. Unlike a strict forward- or backward-chaining proof system, the LLM’s reasoning relies on statistical association rather than brittle rule-following. In a way, that fuzziness is its strength because it generalizes instead of collapsing under contradiction.

I am not sure if chimps can do the same.

If one is to quantify consciousness it would probably make sense to think of it as an area of awareness and cognizance across time.

Awareness scales with sensory scale and resolution (sensory receptors vs input token limits and token resolution). E.g. 128k tokens and tokens too coarse to count rs in strawberry.

Cognizance scales with internal representations of awareness (probably some relation to vector space resolution and granularity, though I suspect there is more to it than just vector space)

And the third component is time, how long the agent is conscious for.

So something like...

Time * awareness (receptors) * internal representations (cell diversity * # cells * connection diversity * # connections)

There is no way this equation is right but I suspect it's sort of directionally correct.

I'm deep in the subject but just riffing here, so take this with a lot of salt.

It's why anyone who's ever taken care of a needy pet will inevitably reach the comparison that taking care of a pet is similar to taking care of a very young child; it's needy, it experiences emotions but it can't quite figure out on its own how to adapt to an environment besides what it grew up around/it's own instincts. They experience some sort of qualia (a lot of animals are pretty family-minded), but good luck teaching a monkey to read. The closest we've gotten is teaching them that if they press the right button, they get food, but they take basically their entire lifespan to understand a couple hundred words, while humans easily surpass that.

IIRC some of the smartest animals in the world are actually rats. They experience a qualia very close to humans to the point that psychology experiments are often easily observable in rats.