Why Can't Transformers Learn Multiplication?

Take your damned upvote, and go away.

But it's very hard to define "real-world CoT" -- think about human, we learn multiplications by vertical calculation and we learn division in a similar way -- all these learning process requires an "information dense" tools (calculation process) with intrinsic math rules in it. Isn't that an adapted way of CoT?

Language _is_ the symbolic manipulation system par excellence though.

A lot of savants that are able to do really cool calculations, or even people that have synesthesia seeing numbers as colors, don't actually do "real" calculations.

I think most humans that do math aren't actually literally computing things as some kind of logic machine.

We can produce logic, and follow the steps of using that logic, but it doesn't seem to me that our cognition is some kind of logic machine itself.

Hmm, I wonder what happens if you let them manipulate their own context symbolically, maybe something like a stack machine. Perhaps all you need is a "delete" token, or a "replace" flag. That way you don't have context full of irrelevant information.

I guess the challenge is, where would the training data come from? Data on the internet is in its final form so "next token" is never a delete.

Edit: I guess in essence, that's what reasoning LLMs already do. IIUC the thought blocks are ephemeral, and only the response is maintained for the chat. Maybe there'd be some benefit of doing this recursively? But that's also kind of what subagents are for. So, perhaps nothing new here.

I agree with you, seems like we are trying to make the shoe fit. Not only are we missing the understanding of what is happening inside transformers, but now we are trying to teach them and see how they respond and then interpret it. That seems fine with viruses and animals, but we are talking about a piece of software here. Shouldn't we know what's happening inside? Maybe these kinds of papers can shine more light and give us better understanding though, still it feels backwards to me...Regarding the multiplication itself, shouldn't pure understanding of the meaning of multiplication(it's a summation basically) be enough for 'AI' to call it a day? If AI or human understands that, then the rest is computation part. We already got that covered, so instead of having 'AI' learn it on its own on crazy amount of data and get it right 99% of time, shouldn't we just give it a calculator? Somebody PLEEAASE give this AI a calculator :-)

I think you might be missing some appropriate context. I agree that it is ridiculous to expect a language model to be good at symbolic manipulation; that is best done with tool use. However, there is a significant line of work dedicated to algorithm discovery for mathematical problems using neural networks. Transformers are used here due to their popularity, but also some theoretical analysis to suggest that they are the among the most efficient architecture for learning automata. It's still unclear whether this is truly sound though, which is where this kind of research matters.

The paper uses a number representation that is designed to make attention easy to learn: each digit is a separate token and the least significant digit is put first, so that the first digit of the output is simply the sum of the first digits of the inputs and the second digit is the sum of the second digits plus an optional carry from the first digits and so on.

If the numbers are represented with the most significant digit first as usual, you need a bunch of intermediate steps before outputting even the first digit just to determine whether it is affected by a carry or not.

The paper looks at multiplication of numbers represented with the least significant digit first as a toy task requiring several additions as intermediate steps to study why a model large enough to perform those additions in principle fails to learn to do so in practice.

They compare with a model that is first trained to produce the intermediate additions explicitly (as a "chain of thought" with a specific format) and then has this CoT progressively shortened during training until there's nothing left of it. But that second model successfully multiplies.

The difference appears to be that the presence of the intermediate results induces a better number representation in latent space, whereas the model without CoT gets stuck in a less efficient local minimum.

So the answer to the question "Why can't transformers learn multiplication?" is that the training process is insufficient for the model to discover the best intermediate steps on its own.

You could do a similar experiment where the CoT involves first taking the logarithm, adding, and then exponentiating to get the final result, but I think logarithms are probably another computation that's too difficult to learn without additional hints for intermediate steps.

Yes, I also wonder about this! Progress from children books to scientific papers etc. Could it learn e.g. language structure faster in a pre-training stage? Also somehow one needs to define a proxy to generalization to compute a loss and do backpropagation.

"an architecture that learned more like humans"

i.e. enduring countless generations of evolutionary selection and cross breeding, then fine-tuning a bit?

although it could be interesting, i don't think training on progressively complex strings entirely recapitulates this.

Isn’t that what all the hundreds of billions are banking on? “General” intelligence.

"Would love to see an architecture that learned"

Would be a far more accurate statement. Training != Learning.

There's an interesting question here.

Would a single human/entity learn more in ..say.. three million years or would short lived ones evolving over three million years and then ~20 years of education learn more?

The current AI tech cycle is focusing on the first, but we don't really know if there are benefits of both.

There's no obvious way to combine these yet.

Would like to see a car that moved like a horse.

Not true though. Internally they can “shell out” to sub-tasks that know how to do specific things. The specific things don’t have to be models.

(I’m specifically talking about commercial hosted ones that have the capability i describe - obviously your run of the mill one downloaded off of the internet cannot do this).

If being probabilistic prevented learning deterministic functions, transformers couldn’t learn addition either. But they can, so that can't be the reason.

Bad take. It's not that it's hard for everyone - there's critical pushback because we don't know for certain if LLM technology can or cannot do the task in question. Which is the reason there's a paper being discussed.

If we were to take the stance of "ok, that happened so it must be the case" we wouldn't be better off in many cases, we would still be accusing people of being witches most likely.

Science is about coming up with a theory and trying to poke holes into it until you can't and in which case, after careful peer-review to ensure you're not just tricking yourself into seeing something which isn't there a consensus is approached in which we can continue to build more truth and knowledge.

Transformers do just fine on many deterministic tasks, and are not necessarily probabilistic. This is not the issue at all. So, it's hard for everyone else because they're not confidently wrong like you are.

> It DOES fail more when the numbers are longer (because it results with more text in the context),

I tried to raise this question yesterday. https://news.ycombinator.com/item?id=45683113#45687769

Declaring victory on "reasoning" based on cherry-picking a correct result about arithmetic is, of course, very narrow and absurdly optimistic. Even if it correctly works for all NxM calculations. Moving on from arithmetic to any kind of problem that fundamentally reduces to model-checking behind the scenes.. we would be talking about exploring a state-space with potentially many thousands of state-transitions for simple stuff. If each one even has a small chance of crapping out due to hallucination, the chance of encountering errors at the macro-scale is going to be practically guaranteed.

Everyone will say, "but you want tool-use or code-gen for this anyway". Sure! But carry-digits or similar is just one version of "correct matters" and putting some non-local kinds of demands on attention, plus it's easier to check than code. So tool-use or code-gen is just pushing the same problem somewhere else to hide it.. there's still a lot of steps involved, and each one really has to be correct if the macro-layer is going to be correct and the whole thing is going to be hands-off / actually automated. Maybe that's why local-models can still barely handle nontrivial tool-calling.

Then your transformer would need to know Python.

The amount of paths in the wrong direction are infinitely more than then number in the right direction. You'll quickly realize this doesn't actually scale.

The feedback from compilation tools / linters fed into the training loops is an example of this.

What we end up with however is a model good at coding for example but bad at something else. And without enough general coding, good at one language over another.

And we're back to square one. The problem of being able to achieve true intelligence by distilling the essence of it not just knowing the answers to specific problems.

Given enough time, we'll plug the gaps and maybe get good enough but it's not true intelligence until it can learn in a way that excels at all fields in a cross-disciplinary way - much better than the side-effect way it's doing now where some other knowledge does actually contribute to achieving goals in other domains.

That's very cool, but it's not an apples to apples comparison. The reasoning model learned how to do long multiplication. (Either from the internet, or from generated examples of long multiplication that were used to sharpen its reasoning skills. In principle, it might have invented it on its own during RL, but no, I don't think so.)

In this paper, the task is to learn how to multiply, strictly from AxB=C examples, with 4-digit numbers. Their vanilla transformer can't learn it, but the one with (their variant of) chain-of-thought can. These are transformers that have never encountered written text, and are too small to understand any of it anyway.

Math is just symbol manipulation with a set of rules, no?

Well. I don't like your limits... I'm looking forward to my zebra farm utopia. :D

> For example, cryptography falls into precision computing. There is no room for being incorrect even by a single bit. Where as machine learning is about getting a range of answers, with tolerance for error.

Doesn't both of them rely on randomness in real use cases/usage? And it's only once you have fixed seeds that cryptography becomes deterministic, and then you can make the same claim for most of ML, when the seeds are fixed you get fixed replies.

It happens to be that most people seem to use LLM clients that aren't deterministic, as they're using temperature + random seeds for each inference, but that doesn't mean someone couldn't do it in a different way.

> Banks wouldn't use AI to compute the account balance after a transaction or for authenticating a customer. Network software wouldn't use AI for encryption and decryption of the TLS traffic.

Not directly, no. But they might use AI to write the code that computes account balance, or authenticates a user, or encrypts/decrypts TLS.