Light Sleep: Waking Vms in 200ms with Ebpf and Snapshots

Always nice to see folks talking about VM snapshots - they're an extremely powerful tool for building systems of all kinds. At AWS, we use snapshots in Lambda Snapstart (along with cloning, and snapshots are distributed across multiple workers), and in Aurora DSQL (where we clone and restore a snapshot of Postgres on every database connection), in AgentCore Runtime, and a number of other places.

> But Firecracker comes with a few limitations, specifically around PCI passthrough and GPU virtualization, which prevented Firecracker from working with GPU Instances

Worth mentioning that Firecracker supports PCI passthrough as of 1.13.0. But that doesn't diminish the value of Cloud Hypervisor - it's really good to have multiple options in this space with different design goals (including QEMU, which has the most features).

> We use the sk_buff.mark field — a kernel-level metadata flag on packets - to tag health check traffic.

Clever!

> Light Sleep, which reduces cold starts to around 200ms for CPU workloads.

If you're restoring on the same box, I suspect 200ms is significantly above the best you can do (unless your images are huge). Do you know what you're spending those 200ms doing? Is it just creating the VMM process and setting up kvm? Device and networking setup? I assume you're mmapping the snapshot of memory and loading it on demand, but wouldn't expect anywhere near 200ms of page faults to handle a simple request.

> Alongside the eBPF program, we run a lightweight daemon — scaletozero-agent — that monitors those counters. If no new packets show up for a set period, it initiates the sleep process.

> No polling. No heuristics. Just fast, kernel-level idle detection.

Isn't the `scaletozero-agent` daemon effectively polling eBPF map counters...?

Slightly OT but would be cool if there was a way to run computations in some on-demand VM that cold started in 200ms, did it thing, died and you only paid for the time you used it. In essence s lambda that exposed you a full blown VM rather than a limited environment.

I feel like I'm missing something here when this is being used with nomad. Caveat being that the only comparable technologies I've worked with are k8s and ECS. In the article they mention that they are using a containerd shim to launch micro VMs, so from the perspective of the scheduler, whether the VM is actually "sleeping" or not, it looks like it's running since they continue to respond to health checks. So what exactly is the point of suspending the VMs on idle if the scheduler still thinks they're running? Whatever memory is reserved for that job is still going to be reserved, so you're not able to oversubscribe the host regardless.

> Saves the full VM state to disk

Does this include the RAM for the VM? For auto-idle systems like this where to park the RAM tends to be a significant concern. If you don't "retire" the RAM too the idling savings are limited to CPU cycles but if you do, the overheads of moving RAM around can easily wreck any latency budget you may have.

Curious how you are dealing with it.

Great post. Not sure if 200ms is fast though, you can definitely boot from zero to pid1 in <10ms.

I guess it depends on the workload, if you are snapshotting an already-loaded Python program, the time savings are huge, but if it's a program with fast startup, it's probably slower to snapshot.

> Waking up instantly on real traffic without breaking clients

is this for new TCP connections? Or also for connections opened prior to sleep?

How is this comparing to Rund? https://www.usenix.org/conference/atc22/presentation/li-ziju...

How does this stack up against unikernel-based VM snapshots?