Deep Dive Into G-Eval: How Llms Evaluate Themselves

I haven’t come across any research showing that a specific LLM consistently outperforms others for this. It generally works best with strong reasoning models that produce consistent outputs.

LLM Benchmark leaderboard for common evals sounds like a fun idea to me.

That's true. Even small API or model version updates can shift evaluation behavior. G-Eval helps reduce that variance, but it doesn’t eliminate it completely. I think long-term stability will probably require some combination of fixed reference models and calibration datasets.