Behind the Scenes of Bun Install

I tried using Bun a few times, and I really liked working with it.

Much better than Node.

However...!

I always managed to hit a road block with Bun and had to go back to Node.

First it was the crypto module that wasn't compatible with Nodejs signatures (now fixed), next Playwright refused working with Bun (via Crawlee).

Node also has a built-in server and SQLite these days though? Or if you want a lot more functionality with just one dependency, Hono is great.

Youre looking at it wrong. Python has nix, JS has nix, ruby and php have nix : D

Thats closer to how pnpm achieves speed up though. I know there is 'rv' recently, but havent tried it.

PHP has Composer, and it's extremely good!

bundler is generally pretty fast on the ruby side. it also reuses dependencies for a given ruby version so you don't have the stupid node_folder in every project you use with every dependency re-downloaded and stored. if you have 90% of the dependencies for a project you only have to download and install/compile 10% of them. night and day difference.

PHP is getting Mago (written in Rust).

Repo: https://github.com/carthage-software/mago

Announcement 9 months ago:

https://www.reddit.com/r/PHP/comments/1h9zh83/announcing_mag...

For now its main features are 3: formatting, linting and fixing lint issues.

I hope they add package management to do what composer does.

it's pretty new, but in Ruby there's `rv` which is clearly inspired by `uv`: https://github.com/spinel-coop/rv.

>Brought to you by Spinel

>Spinel.coop is a collective of Ruby open source maintainers building next-generation developer tooling, like rv, and offering flat-rate, unlimited access to maintainers who come from the core teams of Rails, Hotwire, Bundler, RubyGems, rbenv, and more.

> I wonder why that is?

LLMs default to npm

I think part of the issue is that a lot of the changes have been fairly incremental, and therefore fairly easy to include back into NodeJS. Or they've been things that make getting started with Bun easier, but don't really add much long-term value. For example, someone else in the comments talked about the sqlite module and the http server, but now NodeJS also natively supports sqlite, and if I'm working in web dev and writing servers, I'd rather use an existing, battle-tested framework like Express or Fastify with a larger ecosystem.

It's a cool project, and I like that they're not using V8 and trying something different, but I think it's very difficult to sell a change on such incremental improvements.

It’s a newer, vc funded competitor to the open source battle tested dominant player. It has incentives to lock you in and ultimately is just not that different from node. There’s basically no strategic advantage to using bun, it doesn’t really enable anything you can’t do with node. I have not seen anyone serious choose it yet, but I’ve seen plenty of unserious people use it

Take a look at their issue tracker, it's full of crashes because apparently this Zig language is highly unsafe. I'm staying on Node.

Neither Bun nor Deno have any killer features.

Sure, they have some nice stuff that should also be added in Node, but nothing compelling enough to deal with ecosystem change and breakage.

To beat an incumbent you need to be 2x better. Right now it seems to be a 1.1x better (for any reasonably sized projects) work in progress with kinks you’d expect from a work in progress and questionable ecosystem buy-in. That may be okay for hobby projects or tiny green field projects, but I’m absolutely not gonna risk serious company projects with it.

I really want to like Bun and Deno. I've tried using both several times and so far I've never made it more than a few thousand lines of code before hitting a deal breaker.

Last big issue I had with Bun was streams closing early:

https://github.com/oven-sh/bun/issues/16037

Last big issue I had with Deno was a memory leak:

https://github.com/denoland/deno/issues/24674

At this point I feel like the Node ecosystem will probably adopt the good parts of Bun/Deno before Bun/Deno really take off.

I am also very curious what people think about this. To me, as a project, Node gives off a vibe of being mature, democratic and community driven, especially after successfully navigating then io.js fork drama etc a few years ago. Clearly neither bun nor deno are community driven democratic projects, since they are both VC funded.

There are some rough edges to Bun (see sibling comments), so there's a apparent cost to switching, namely wasted developer time in dealing with Node incompatibility. Being able to install packages 7x faster doesn't matter much to me so I don't see an upside to making the switch.

Tried it last year - I spent a few hours fighting the built in sqlite driver and found it buggy (silent errors) and the docs were very lacking.

Bun is much newer than pnpm, looking at 1.0 releases pnpm has about a 6 year head start.

I write a lot of one off scripts for stuff in node/ts and I tried to use Bun pretty early on when it was gaining some hype. There were too many incompatibilities with the ecosystem though, and I haven't tried since.

I am Bun's biggest fan. I use it in every project I can, and I write all my one-off scripts with Bun/TS. That being said, I've run into a handful of issues that make me a little anxious to introduce it into production environments. For instance, I had an issue a bit ago where something simple like an Express webserver inside Docker would just hang, but switching bun for node worked fine. A year ago I had another issue where a Bun + Prisma webserver would slowly leak memory until it crashed. (It's been a year, I'm sure they fixed that one).

I actually think Bun is so good that it will still net save you time, even with these annoyances. The headaches it resolves around transpilation, modules, workspaces etc, are just amazing. But I can understand why it hasn't gotten closer to npm yet.

I tried to run my project with bun - it didn't work so I gave up. Also, there needs to be a compelling reason to switch to a different ecosystem.

There's still a few compatibility sticking points... I'm far more familiar with Deno and have been using it a lot the past few years, it's pretty much my default shell scripting tool now.

That said, for many work projects, I need to access MS-SQL, which the way it does socket connections isn't supported by the Deno runtime, or some such. Which limits what I can do at work. I suspect there's a few similar sticking points with Bun for other modules/tools people use.

It's also very hard to break away from entropy. Node+npm had over a decade and a lot of effort to build that ecosystem that people aren't willing to just abandon wholesale.

I really like Deno for shell scripting because I can use a shebang, reference dependencies and the runtime just handles them. I don't have the "npm install" step I need to run separately, it doesn't pollute my ~/bin/ directory with a bunch of potentially conflicting node_modules/ either, they're used from a shared (configurable) location. I suspect bun works in a similar fashion.

That said, with work I have systems I need to work with that are already in place or otherwise chosen for me. You can't always just replace technology on a whim.

Honestly, it doesn't really solve a big problem I have, and introduces all the problem with being "new" and less used.

To me, the reasoning seems to be:

Npm, pnpm, and yarn are written in JS, so they have to use Node.js facilities, which are based on libuv, which isn't optimal in this case.

Bun is written in Zig, so it doesn't need libuv, and can so it's own thing.

Obviously, someone could write a Node.js package manager in C/C++ as a native module to do the same, but that's not what npm, pnpm, and yarn did.

Isn't the issue not that libuv is C, but that the thing calling it (Node.js) is Javascript, so you have to switch modes each time you have libuv make a system call?

Why you prefer WSL1 over WSL2?

It's straight from the GZIP spec if you assume there's a single GZIP "member": https://www.ietf.org/rfc/rfc1952.txt

> ISIZE (Input SIZE)

> This contains the size of the original (uncompressed) input data modulo 2^32.

So there's two big caveats:

1. Your data is a single GIZP member (I guess this means everything in a folder)

2. Your data is < 2^32 bytes.

gzip.py [1]

---

def _read_eof(self):

# We've read to the end of the file, so we have to rewind in order

# to reread the 8 bytes containing the CRC and the file size.

# We check the that the computed CRC and size of the

# uncompressed data matches the stored values. Note that the size

# stored is the true file size mod 2*32.

---

[1]: https://stackoverflow.com/a/1704576

I believe it's because you get to stream-compress efficiently, at the cost of stream-decompress efficiency.

They don’t seem to clear the cache in between fresh runs. This is evident from the lower bound range time being the same time as the cached mean.

This leads them to the incorrect conclusion that bun fresh runs are faster than npm cached, which doesn’t seem to be the case.

I think Deno isn't included in the benchmark because it's a harder comparison to make than it might seem.

Deno's dependency architecture isn't built around npm; that compatibility layer is a retrofit on top of the core (which is evident in the source code, if you ever want to see). Deno's core architecture around dependency management uses a different, URL-based paradigm. It's not as fast, but... It's different. It also allows for improved security and cool features like the ability to easily host your own secure registry. You don't have to use npm or jsr. It's very cool, but different from what is being benchmarked here.

For my own curiousity on a React app on my work desktop.

    - Clean `bun install`, 48s - converted package-lock.json
    - With bun.lock, no node_modules, 19s
    - Clean with `deno install --allow-scripts`, 1m20s
    - with deno.lock, no node_modules, 20s
    - Clean `npm i`, 26s
    - `npm ci` (package-lock.json), no node_modules, 1m,2s (wild)

So, looks like if Deno added a package-lock.json conversion similar to bun the installs would be very similar all around. I have no control over the security software used on this machine, was just convenience as I was in front of it.

Hopefully someone can put eyes on this issue: https://github.com/denoland/deno/issues/25815

It's more than enough to lose your focus. If you can make a process take a couple seconds or less vs over 15, you should do that.

I am thrilled that anyone in the web dev community is giving a shit about performance, and clearly knows something about how a computer actually works.

I am so, so tired of the “who cares if this library eats 100 MiB of RAM; it’s easier” attitude.

Truth!

Lydia is very good at presenting complex ideas simply and well. I've read and watched most of her work or videos. She really goes to great lengths in her work to make it come to life. Highly recommend her articles and YouTube videos.

Though she's been writing less I think due to her current job

I didn’t know it was written in Zig. That’s a fascinating choice to me given how young the language is.

Amazing to see it being used in a practical way in production.

why leave pnpm?

> Now multiply that by thousands of concurrent connections each doing multiple I/O operations. Servers spent ~95% of their time waiting for I/O operations.

Well, no. The particular thread of execution might have been spending 95% of time waiting for I/O, but a server (the machine serving the thousands of connections) would easily run at 70%-80% of CPU utilization (because above that, tail latency starts to suffer badly). If your server had 5% CPU utilization under full load, you were not running enough parallel processes, or did not install enough RAM to do so.

Well, it's a technicality, but the post is devoted to technicalities, and such small blunders erode the trust to the rest of the post. (I'm saying this as a fan of Bun.)

I'm guessing that's an LLM hallucination. The conclusion section especially has some hints it was pulled out of an LLM:

> The package managers we benchmarked weren't built wrong, they were solutions designed for the constraints of their time.

> Buns approach wasn't revolutionary, it was just willing to look at what actually slows things down today.

> Installing packages 25x faster isn't "magic": it's what happens when tools are built for the hardware we actually have.

> even low-end smartphones have more RAM than high-end servers had in 2009

That's even less accurate. By two orders of magnitude. High-end servers in 2009 had way more than 4GB. The (not even high-end) HP Proliant I installed for a small business in 2008, that was already bought used at the time, had 128GB of RAM.

I understand why one would want to make an article entertaining but that seriously makes me doubt the rest of the articles when diving into a topic I don't know as much.

This is about software engineering not computer science.

CS is the study of computations and their theory (programming languages, algorithms, cryptography, machine learning, etc).

SE is the application of engineering principles to building scalable and reliable software.

Streaming prevents many optimizations because the code can’t assume it’s done when run once, so it has to suspend / resume, clone extra data for longer, and handle boundary cases more carefully.

It’s usually only worth it after ~tens of megabytes, but vast majority of npm packages are much smaller than that. So if you can skip it, it’s better.

Read further. In one of the later benchmarks, yarn makes 4 million syscalls.

Still only about 2 secs, but still.

AFAIK `bun install` is similar to `npm install` in the sense that it installs everything in a flat structure inside node modules, why didn't you choose something like pnpm that I believe is better, because you cannot by mistake import a transitive dependency. Maybe that's a non issue for most, but I care about those things.

Kinda offtopic but how to you manage to stay so productive for so long?

Vitamins/supplements? Sleep? Exercise? Vacations?

I have sprints of great productivity but it's hard to keep it for long.

I just want to say thanks to you and the team and community. Bun is a treat to use.

> Really wish the norm was that companies hosted their own registries for their own usage

Is this not the norm? I've never worked anywhere that didn't use/host their own registry - both for hosting private packages, but also as a caching proxy to the public registry (and therefore more control over availability, security policy)

https://verdaccio.org/ is my go to self hosted solution, but the cloud providers have managed solutions and there's also jFrog Artifactory.

One corollary of this is that many commercial usages of packages don't contribute much to download stats, as often they download each version at most once.

Here is the code:

https://github.com/oven-sh/bun/blob/7d5f5ad7728b4ede521906a4...

We trust the self-reported size by gzip up to 64 MB, try to allocate enough space for all the output, then run it through libdeflate.

This is instead of a loop that decompresses it chunk-by-chunk and then extracts it chunk-by-chunk and resizing a big tarball many times over.

CICD is a major usage. But dependencies version bumps are also a big part of it. In the python ecosystem I’ve had poetry take minutes to resolve the ansible dependencies after bumping the version. And then you see uv take milliseconds to do a full install from scratch.