What Works: The Hardware Compatibility Filter in Neural Architecture

OR: How GPU primitives shape adoption, why MoE spread, and why architecture innovation is slowing down.

Ever wondered what happened to capsule networks and neural ODEs?

I was curious about what makes some architectures work, while others disapear after being hyped.

This is my first take, on what limits LLM architecture innovations from adoption.

(there are more then one reasons, as I understood afterall)