Python Workers Redux: Fast Cold Starts, Packages, and a Uv-First Workflow

Do you happen to know what build issues they're referring to?

"SnapStart for Python" costs extra though. If we are paying then you can even have prewarmed Python lambdas with no cold start on AWS (Provisioned Concurrency).

In the linked detailed benchmark results they include Lambda SnapStart which seems to be faster than Cloudflare:

> AWS Lambda (No SnapStart) Mean Cold Start: 2.513s Data Points: 1008 > AWS Lambda (SnapStart) Mean Cold Start: 0.855s Data Points: 17 > Google Cloud Run Mean Cold Start: 3.030s Data Points: 394 > Cloudflare Workers Mean Cold Start: 1.004s Data Points: 981

https://cold.edgeworker.net

I'm betting against wasm and going with containers instead.

I have warm pool of lightweight containers that can be reused between runs. And that's the crucial detail that makes or breaks it. The good news is that you can lock it down with seccomp while still allowing normal execution. This will give you 10-30ms starts with pre-compiled python packages inside container. Cold start is as fast as spinning new container 200-ish ms. If you run this setup close to your data, you can get fast access to your files which is huge for data related tasks.

But this is not suitable for type of deployment Cloudflare is doing. The question is whether you even want that global availability because you will trade it for performance. At the end of the day, they are trying to reuse their isolates infra which is very smart and opens doors to other wasm-based deployments.

(I work at Cloudflare, but not on D1)

I believe this is possible, you can create D1 databases[1] using Cloudflare's APIs and then deploy a worker using the API as well[2].

1 - https://developers.cloudflare.com/api/resources/d1/subresour...

2 - https://developers.cloudflare.com/api/resources/workers/subr...

I'm always a little hesitant to use D1 due to some of these constraints. I know I may not ever hit 10GB for some of my side projects so I just neglect sharding, but also it unsettles me that it's a hard cap.

Why not durable objects? I think it's the recommended pattern for having a db per user

Try Durable Objects. D1 is actually just a thin layer over Durable Objects. In the past D1 provided a lot of DX benefits like better observability, but those are increasingly being merged back into DO directly.

What is a Durable Object? It's just a Worker that has a name, so you can route messages specifically to it from other Workers. Each one also has its own SQLite database attached. In fact, the SQLite database is local, so you can query it synchronously (no awaits), which makes a lot of stuff faster and easier. You can easily create millions of Durable Objects.

(I am the lead engineer for Workers.)

It relies entirely on the WebAssembly runtime, see the discussion of how ASLR problems don’t occur with WASM. Doing this with WASM is pretty easy, doing it with system memory is quite tricky.

it's only absurd if you don't want to pay cloudflare money

That's basically not how Cloudflare works.

Your app works distributed/globally on the go.

Additionally, every Enterprise feature will become available in time ( discussed during their previous quarter earnings)

> The startup time of a simple .py script can easily be in the 100 to 300 ms range

I can't say I've ever experienced this. Are you sure it's not related to other things in the script?

I wrote a single file Python script, it's a few thousand lines long. It can process a 10,000 line CSV file and do a lot of calculations to the point where I wrote an entire CLI income / expense tracker with it[0].

The end to end time of the command takes 100ms to process those 10k lines, that's using `time` to measure it. That's on hardware from 2014 using Python 3.13 too.

[0]: https://github.com/nickjj/plutus

    import requests

    package main
    import (
       _ "net/http"
    ) 
    func main() {
    }

    import requests
    print(requests.get("http://localhost:3000").text)

    package main

    import (
      "fmt"
      "io"
      "net/http"
     )

    func main() {
        resp, _ := http.Get("http://localhost:3000")
        defer resp.Body.Close()
        body, _ := io.ReadAll(resp.Body)
        fmt.Println(string(body))
    }

    python3:  0.08s user 0.02s system 91% cpu 0.113 total
    go 0.00s user 0.01s system 72% cpu 0.015 total

    python 0.03s user 0.01s system 83% cpu 0.059 total
    go 0.00s user 0.00s system 80% cpu 0.010 total

it depends somewhat on what you import, too. some people would sell their grandmothers to get below 1s when you start importing numpys and scikits.

Are you comparing the startup time of an interpreted language with the startup time of a compiled language? or you mean that `time python hello.py` > `( time gcc -O2 -o hello hello.c ) && ( time ./hello )` ?

Yes, that is also my feeling. But comparing an interpreted language with a compiled one is not really fair.

Here is my quick benchmark. I refrain from using Python for most scripting/prototyping task but really like Janet [0] - here is a comparison for printing the current time in Unix epoch:

    $ hyperfine "python3 -c 'import time;print(time.time())'" "janet -e '(print (os/time))'"
    Benchmark 1: python3 -c 'import time;print(time.time())'
  Time (mean ± σ):      22.1 ms ±   1.0 ms    [User: 12.3 ms, System: 4.1 ms]
  Range (min … max):    20.7 ms …  25.5 ms    118 runs

    Benchmark 2: janet -e '(print (os/time))'
  Time (mean ± σ):       3.8 ms ±   0.3 ms    [User: 1.2 ms, System: 0.7 ms]
  Range (min … max):     3.3 ms …   5.2 ms    479 runs

  Warning: Command took less than 5 ms to complete. Note that the results might be inaccurate because hyperfine can not calibrate the shell startup time much more precise than this limit. You can try to use the `-N`/`--shell=none` option to disable the shell completely.

    Summary
  'janet -e '(print (os/time))'' ran
    5.80 ± 0.48 times faster than 'python3 -c 'import time;print(time.time())''

[0]: https://janet-lang.org/

Reminds me of mercurial cvs!!

It might not be the fastest but I suspect something weird is happening with python resolution.

For instance `uv run` has its own fair share of overhead.

    $ hyperfine --warmup 10 -L py "uv run python,~/.local/bin/python3.14,/usr/local/bin/python3.12,~/.local/share/uv/python/pypy-3.11.13-macos-aarch64-none/bin/pypy3.11" "{py} -c 'exit(0)'"
    Benchmark 1: uv run python -c 'exit(0)'
      Time (mean ± σ):      58.4 ms ±  19.3 ms    [User: 26.4 ms, System: 21.7 ms]
      Range (min … max):    48.2 ms … 138.0 ms    50 runs
    
    Benchmark 2: ~/.local/bin/python3.14 -c 'exit(0)'
      Time (mean ± σ):      13.3 ms ±   6.9 ms    [User: 8.0 ms, System: 2.5 ms]
      Range (min … max):     9.9 ms …  53.7 ms    174 runs
    
    Benchmark 3: /usr/local/bin/python3.12 -c 'exit(0)'
      Time (mean ± σ):      16.4 ms ±   7.6 ms    [User: 8.9 ms, System: 3.7 ms]
      Range (min … max):    12.2 ms …  65.2 ms    152 runs
    
    Benchmark 4: ~/.local/share/uv/python/pypy-3.11.13-macos-aarch64-none/bin/pypy3.11 -c 'exit(0)'
      Time (mean ± σ):      18.6 ms ±   7.4 ms    [User: 10.0 ms, System: 5.0 ms]
      Range (min … max):    14.4 ms …  63.5 ms    138 runs
    
    Summary
      ~/.local/bin/python3.14 -c 'exit(0)' ran
        1.23 ± 0.86 times faster than /usr/local/bin/python3.12 -c 'exit(0)'
        1.40 ± 0.92 times faster than ~/.local/share/uv/python/pypy-3.11.13-macos-aarch64-none/bin/pypy3.11 -c 'exit(0)'
        4.40 ± 2.72 times faster than uv run python -c 'exit(0)'

You can run .pyc stuff “directly” with some creativity, and there are some tools to pack “executables” that are just chunked blobs of bytecode.

Completely agree on this. First of all, congrats to the Cloudflare team on their work on Python on Workers, it's inspiring to all providers on the space.

Regarding cold-starts, I strongly believe V8 snapshots are perhaps not the best way to achieve fast cold starts with Python (they may be if you are tied to using V8, though!), and will have wide side effects if you go out of the standards packages included on the Pyodide bundle.

To put some perspective: V8 snapshots are storing the whole state of an application (including it's compiled modules). This means that for a Python package that is using Python (one wasm module) + Pydantic-core (one wasm module) + FastAPI... all of those will be included in one snapshot (as well as the application state). This makes sense for browsers, where you want to be able to inspect/recover everything at once.

The issue about this design is that the compiled artifacts and the application state are bundled into one piece artifact (this is not great for AOT design, but might be the optimal design for JITs though).

Ideally, you would separate each of the compiled modules from the state of the application. When you do this, you have some advantages: you can deserialize the compiled modules in parallel, and untie the "deserialization" from recovering the state of the application. This design doesn't adapt that well into the V8 architecture (and how it compiles stuff) when JavaScript is the main driver of the execution, however it's ideal when you just use WebAssembly.

This is what we have done at Wasmer, which allows for much faster cold starts than 1 second. Because we cache each of the compiled modules separately, and recover the state of the application later, we can achieve cold-starts that are a magnitude faster than Cloudflare's state of the art (when using pydantic, fastapi and httpx).

If anyone is curious, here is a blogpost where we presented fast-cold starts for the application state (note that the deserialization technique for Wasm modules is applied automatically in Wasmer, and we don't showcase it on the blogpost): https://wasmer.io/posts/announcing-instaboot-instant-cold-st...

It’s not Python runtime startup, but loading all your modules.

Use lazy/dynamic imports and you will see it drop .

I don't know why people care so much about a few hundreds of milliseconds for python scripts versus compiled languages that take just ten times less.

Real question : what would you do more with the spared time ? You are that in a hurry in your life ?

Run strace on Python starting up- you will see it statting hundreds if not thousands of files. That gets much worse the slower your filesystem is.

On my linux system where all the file attributes are cached, it takes about 12ms to completely start, run a pass statement, and exit.

Big packages shouldn’t be imported until the cli has been parsed, and handed off to main. There’s has been work to do this automatically, but it’s good hygiene to avoid them anyway.

A modern machine shouldn’t take this long, so likely something big is being imported unnecessarily at startup.

Scale down to actual 0, really easy edge distribution, stupidly simple to deploy to. That's really it.

Think about how often that box needs patching for code outside of your app and you’re talking load-balancing, autoscaling, etc. to avoid downtime or overloads, but also paying for idle capacity. Then, of course, you have to think about security partitions if anything you run on that box shouldn’t have access to everything else. None of that is unknown, we have decades of experience and tooling dealing with it, etc. but it’s a job that you can just choose not to have and people often do, especially for things which are bursty. There’s something really nice about not needing to patch anything for years because all you’re using is the Python stdlib and scaling from zero to many, many thousands with no added effort.