Jit: So You Want to Be Faster Than an Interpreter on Modern Cpus

My take is that you can get pretty far these days with a simple bytecode interpreter. Food for thought if your side project could benefit from a DSL!

Why not do the same thing the CPU does and fetch N jump addresses at once?

Now the overhead is gone and you just need to figure out how to let the CPU fetch the chain of instructions that implement the opcodes.

You simply copy the interpreter N times, store N opcode jump addresses in N registers and each interpreter copy is hardcoded to access its own register during the computed goto.

No, that's speculative execution you just described. Branch prediction was implemented long before out-of-order CPUs were a thing, as you need branch prediction to make the most of pipelining (eg. fetching and decoding a new instruction while you're still executing the previous one--if you predict branches, you're more likely to keep the pipeline full).

Also I recommend reading the previous blog post first, then this one, for additional context: https://www.pinaraf.info/2024/03/look-ma-i-wrote-a-new-jit-c...

Sql server hekaton punted this problem in a seemingly effective way by requiring the client to use stored procedures to get full native compilation. Not sure though if they recompile if the table statistics indicate a different query plan is needed.