Bits-Per-Byte (bpb): a Tokenizer-Agnostic Way to Measure Llms

The article introduces 'Bits-per-Byte' (BPB) as a tokenizer-agnostic metric for measuring Large Language Models (LLMs), but received little attention or discussion on HN.

Snapshot generated from the HN discussion