Netflix Optimized Its Petabyte-Scale Logging System with Clickhouse

This is an eye-popping 25 megabytes of logs per streamed hour of video.

Considering that 99% of what NetFlix is supposed to “do” is stream static content to users, this is an indirect signal that they’re too busy pleasing their middle managers with internal reports instead of doing “the thing” their business is primarily about.

Every time this comes up, someone makes the comment that they need those 40,000 microservices to provide recommendations and the like, but the end result is objectively and subjectively terrible.

I’m forced to Google for “what’s good on NetFlix” because the in-app recommendations are hopeless.

Also: they can collect up to 12 petabytes of writes to disk daily, but still struggle with the apparently much more difficult task of distributing more than five subtitle languages at a time! Those 100 KB text files must have been a step too far for their infrastructure to handle.

Maybe next year I’ll read a self-congratulatory blog article where they’ve scaled logging to fifty exabytes per second and subtitles to six languages per region.