The Future of Python Web Services Looks Gil-Free

But OOP does not necessitate mutable state; you can do OOP with immutable objects and pure methods (except the constructor). Objects are collections of partially-applied functions (which is explicit in Python) that also conceal internal details within its own namespace. It is convenient in certain cases.

If you want immutability as a lynchpin, you need to look at a different language. This one is very much not designed for it.

  def f():
      l = [0]
      for i in range(100000000):
          l += [2]
      return l

  x = f()
  print(x[0])

  3.9:                    8.6s

  3.14:                   8.7s

  3.14 (free-threading): 11.6s

Because literally every import, class definition, or function definition that you make at top-level is a global.

Now some people do in fact do all those things inside a function, too, and then call that function as the only thing that actually happens globally. And I've done such hacks myself to squeeze the last few % of perf out of CPython on the very rare occasions where you need to do that but dropping into C is not an option. But that's certainly not idiomatic Python.

I have not been seeing good Python code, for sure. Hopefully it's not a majority. But it's very far from non-existent.

The question is whether 1+ thread per core with GIL free Python perform as well as 1+ process per core with GIL.

My understanding is that this global is just a way to demonstrate that the finely grained locking in the GIL free version may make it so that preforking servers may still be more performant.

Also, sharing memory between processes is very very very slow compared to sharing memory between threads.

https://docs.python.org/3/howto/free-threading-extensions.ht...

And a dedicated web site:

https://py-free-threading.github.io/

But as an example neither include PySequence_Fast which is in the json.c changes I pointed to. The folks doing the auditing of stdlib do have an idea of what they are looking for, and so would be best suited to keep a list (and tool) up to date with what is needed.

The free threaded implementation adds what amounts to individual object locks at the C level (critical sections). This still means developers writing Python code can do whatever they want, and they will not experience corruption or crashes. The base objects have all been updated.

Python is popular because of many extensions written in C, including many in the standard library. Every single piece of that code must be updated to operate correctly in free threaded mode. That is a lot of work and is still in progress in the standard library. But in order to make the free threaded interpreter useful at this point, some have been marked as free thread safe, when that is not the case.

Nit, that's true iff x is a primitive without the volatile modifier. That's not true for a volatile primitive.

it's quite possible to make a python app that requires libraries A and B to be able to be loaded into a free-threaded application, but which doesn't actually do any unsafe operations with them. we need to be able to let people load these libraries, but say: this thing may not be safe, add your own mutexes or whatever

You have to explicitly compile the extension against a free threaded interpreter in order to get that ABI tag in your extension and even be able to load the extension. The extension then has to opt-in to free threading in its initialization.

If it does not opt-in then a message appears saying the GIL has been enabled, and the interpreter continues to run with the GIL.

This may seem a little strange but is helpful. It means the person running Python doesn't have to keep regular and free threaded Python around, and duplicate sets of extensions etc. They can just have the free threaded one, anything loaded that requires the GIL gives you the normal Python behaviour.

What is a little more problematic is that some of the standard library is marked as supporting free threading, even though they still have the audit and update work outstanding.

Also the last time I checked, the compiler thread sanitizers can't work with free threaded Python.

* Point out using APIs that return borrowed references

* Suggest assertions that critical sections are held when operating on objects

* Suggest alternate APIs

* Recognise code patterns that are similar to those done during the stdlib auditing work

The compiler thread sanitizers didn't work the last time I checked - so get them working.

Edit: A good example of what can be done is Coccinelle used in the Linux kernel which can detect problematic code (locking is way more complex!) as well as apply source transformations. https://www.kernel.org/doc/html/v6.17/dev-tools/coccinelle.h...