Searles's Chinese Room: Case Study in Philosophy of Mind and Cognitive Science

In Searle's Chinese Room, we're asked to imagine a system that appears intelligent but lacks intentionality, the capacity of mental states to be about or directed toward something. In Searle's setup he didn't conceive of a system with either learning or an internal state and instead we have a static rulebook that manipulates symbols purely according to syntactic rules.

What you're suggesting is that if the rulebook or maybe the agent could learn and remember then it could adapt and is closer to an intelligent system and in turn would have understanding. Which is something Searle anticipated.

Searle covered this idea in the original paper and in a series of replies: Minds, Brains, and Programs (1980, anticipation p. 419 + peer replies), Minds, Brains, and Science (1984), Is the Brain’s Mind a Computer Program? (1990), The Rediscovery of the Mind (1992), and many more clarifications from lectures and interviews. He was responded to in papers from Dennett, the Churchlands, Hofstadter, Boden, Clark, and Chalmers (which you may be interested in if you're looking to go deeper).

To try and summarize Searle: adding learning or state will only complicate the syntax, it's still a purely rule-governed symbol manipulation system; there is no semantic content in the symbols; and the learning or internal changes remain formal operations (not experiences or intentions).

So zooming out, even adding learning and states, we're still dealing with syntax and no amount of syntactic complexity will get us to understanding. Of course, this leads to debate from Functionalists like Putnam, Fodor, and Lewis. This is similar to what you're pointing at and they would say that if a system with an internal state and learning can interpret new information, reason about it, and act coherently, then it functionally understands. And I think this is sort of the place where people are landing with modern AI.

Searle’s deeper claim, however, is that the mind is non-computational. Computation manipulates symbols; the mind means. And the best evidence for that, I think, lies not in metaphysics but in philosophy of language, where we can observe how meaning continually outruns syntax.

Phenomena such as deixis, speech acts, irony and metaphor, reference and anaphora, presupposition and implicature, and reflexivity all reveal a cognitive and contextual dimension to language that no formal grammar explains.

Searle’s view parallels Frege’s insight that meaning involves both sense (how something is presented) and reference (what it designates), and it also echoes Kaplan’s account of indexicals in Demonstratives (1977), where expressions such as I, here, now, today, and that take their content entirely from the context of utterance: who is speaking, when, and where. Both Frege and Kaplan, in different ways, reveal the same limit that Searle emphasizes: understanding depends on an intentional, contextual relation to the world, not on syntactic form alone.

Before this becomes a rambling essay, we're left with Frege's tension of coextensivity (A = A and A = B), where logic treats them as equivalent but understanding does not. If the functionalists are right, then perhaps that difference, between meaning and mechanism, is only apparent, and we’re making distinctions without real differences.

Like understanding how to bake a cake. I can have a simplistic model, for example taking a box cake and making it. Or a more complex model, using the raw ingredients in the right proportions. Both of these have some level of understanding on what's necessary to bake a cake.

And I think AI models have this too. When they have some base knowledge on a topic, and you ask a question that can require a tool without asking for a tool directly, they can suggest a tool to use. Which at least to me make it appear the system as a whole has understanding.