Vortex: an Extensible, State of the Art Columnar File Format
Postedabout 2 months agoActiveabout 1 month ago
github.comTechstory
supportivepositive
Debate
10/100
Data StorageFile FormatsColumnar Databases
Key topics
Data Storage
File Formats
Columnar Databases
Vortex is a new, extensible columnar file format that has been open-sourced on GitHub, offering a state-of-the-art solution for data storage.
Snapshot generated from the HN discussion
Discussion Activity
Active discussionFirst comment
5d
Peak period
17
120-132h
Avg / period
8.7
Comment distribution26 data points
Loading chart...
Based on 26 loaded comments
Key moments
- 01Story posted
Nov 14, 2025 at 9:55 PM EST
about 2 months ago
Step 01 - 02First comment
Nov 19, 2025 at 6:56 PM EST
5d after posting
Step 02 - 03Peak activity
17 comments in 120-132h
Hottest window of the conversation
Step 03 - 04Latest activity
Nov 20, 2025 at 12:03 PM EST
about 1 month ago
Step 04
Generating AI Summary...
Analyzing up to 500 comments to identify key contributors and discussion patterns
ID: 45934665Type: storyLast synced: 11/21/2025, 6:22:07 PM
Want the full context?
Jump to the original sources
Read the primary article or dive into the live Hacker News thread when you're ready.
https://www.youtube.com/watch?v=zyn_T5uragA
Vortex is, roughly, how you save data to files and Iceberg is the database-like manager of those files. You’ll soon be able to run Iceberg using Vortex because they are complementary, not competing, technologies.
Parquet is ..fine, I guess. It is good enough. Why invoke churn? Sell me on the vision.
Mutability would be one such pitch I would like to see ...
There are other formats though that it can be compared to.
The Lance columnar format is one: https://github.com/lancedb/lancedb
And Nimble from Meta is another: https://github.com/facebookincubator/nimble
Parquet is so core to data infra and widespread, that removing it from its throne is a really really hard task.
The people behind these projects that are willing to try and do this, have my total respect.
[1] https://github.com/vortex-data/vortex/issues/2116
I'd generally encourage new type systems to include sum types as a first-class concept.
3 more comments available on Hacker News