Amd's Chiplet Apu: an Overview of Strix Halo

I read somewhere, but can't remember where, that a major reason those APUs aren't as efficient as the Apple ones is a conscious decision to share the architecture with Epyc and therefore accept worse efficiency at lower wattage as a tradeoff.

Can someone confirm/refute that?

I love the concept of it and have been thinking about getting one the only problem I see right now is no ability as far as I can see to get an external dock to run an additional external gpu in the future.

So potentially competitive with a 5070M for graphics? Sounds very nice, as long as price and power draw are reasonable.

Comparing this against mobile dGPUs and the (finally real) DGX Spark, this feels like a latent market segment that has not arrived at its final form. I don't know what delayed the DGX Spark so long, but it granted AMD a huge boon by allowing them capture some market mindshare first.

Compared to discrete GPUs (mobile or not), the advantage of a dGPU is memory bandwidth. The disadvantage of a dGPU is power draw and memory capacity—if we set aside CUDA, which I grant is a HUGE thing to just "set aside".

If we mix in the small DGX Spark desktops, then those have an additional advantage in the dual 200Gb network ports that allow for RDMA across multiple boxes. One could get more from of a small stack (2, 3 or 4) of those than from the same number of Strix Halo 395 boxes. However, as sexy as my homelab-brain finds a small stack of DGX Spark boxes with RDMA, I would think that for professional use, I would rather have a GPU server (or Threadripper GPU workstation) than four DGX Spark boxes?

Because the DGX Spark isn't being sold in a laptop (AFAIK, CMIIW), that is another differentiator in favor of the Strix Halo. Once again, it points to this being a weird, emerging market segment, and I expect the next generation or two will iterate towards how these capabilities really ought to be packaged.

The saddest part of this is the lack of availability: at this point there's 2 standard laptops using this chip, the Z13 being the only high perf one. There's the Framework lines as well, but they aren't available in many countries, and it's a very specific public.

And that's after half a year after the first machines to come to the market.

I love the Z13, but it's clearly a niche machine, so I'm assuming they are having a really hard time manufacturing the chips ? All the capacity is getting eaten by Apple ?

I wonder if higher TDP is possible with framework desktop. That one probably has much better cooling than these laptops with the same chip and if numbers are different.

I would love to try out one of the mini-PCs that ship this, but they seem to be made of either platinum (hugely overpriced in EU) or unobtainium (no retailers carry them here, and getting something direct from China is dicey warranty-wise). ROCm 7 looks to be working already under most Linux distros and having this as a workstation with a local LLM or a “home inference server” with Ollama and a few services seems like a great solution.

Related question: Can I buy a desktop Zen 5 CPU and something like an RX 7600 XT and some RAM and have a high shared memory bandwidth situation between the system memory and the GPU ala Strix Halo and Apple Silicon without spending a ton of money?

And get pretty reasonable local LLM performance on some of the larger models for hobbyist use?

Edit: I don’t have a good grasp on this but I’m thinking I can only do shared memory when I’m using an APU and not a discrete GPU. Is this correct?

I was just thinking the other day that AMD can match Nvidia pound for pound on the raw hardware specs, and if they don’t just yet, they get pretty close. If AI is a bubble, then AMD should not catch up. If there isn’t a bubble, then there is no choice but to learn to use whatever is out there and AMD is truly set to be another trillion dollar company. The 10% stake OpenAI took is going to look like a Google buying YouTube moment in the long run.

And it’s worth noting, AMD has always matched up with Nvidia hardware wise for decades, plus or minus. They are an interesting company in that they took on both Nvidia and Intel, and is still continuing to do so.

High Yield has a video that deep dive into the 395 chip on the silicon level: https://youtu.be/maH6KZ0YkXU

how does the gpu compare though to the ones in m-series macs ?

I picked up a framework desktop and am running it through it's paces right now. So far, it's a impressive little box. I'm really hopeful that this continues to drive more and more enthusiast support and engagement. Getting strong vulcan or rocm supported infrastructure would be great for everyone.