Pep 810 – Explicit Lazy Imports
Posted3 months agoActive3 months ago
peps.python.orgTechstoryHigh profile
controversialmixed
Debate
80/100
PythonLazy ImportsProgramming Languages
Key topics
Python
Lazy Imports
Programming Languages
The Python community is discussing PEP 810, which proposes introducing explicit lazy imports to improve performance, with some users supporting the idea and others raising concerns about its implications.
Snapshot generated from the HN discussion
Discussion Activity
Very active discussionFirst comment
40m
Peak period
99
0-6h
Avg / period
14.5
Comment distribution160 data points
Loading chart...
Based on 160 loaded comments
Key moments
- 01Story posted
Oct 3, 2025 at 2:24 PM EDT
3 months ago
Step 01 - 02First comment
Oct 3, 2025 at 3:05 PM EDT
40m after posting
Step 02 - 03Peak activity
99 comments in 0-6h
Hottest window of the conversation
Step 03 - 04Latest activity
Oct 6, 2025 at 9:25 AM EDT
3 months ago
Step 04
Generating AI Summary...
Analyzing up to 500 comments to identify key contributors and discussion patterns
ID: 45466086Type: storyLast synced: 11/20/2025, 8:23:06 PM
Want the full context?
Jump to the original sources
Read the primary article or dive into the live Hacker News thread when you're ready.
https://pep-previews--4622.org.readthedocs.build/pep-0810/#f...
Q: How does this differ from the rejected PEP 690?
A: PEP 810 takes an explicit, opt-in approach instead of PEP 690’s implicit global approach. The key differences are:
Explicit syntax: lazy import foo clearly marks which imports are lazy.
Local scope: Laziness only affects the specific import statement, not cascading to dependencies.
Simpler implementation: Uses proxy objects instead of modifying core dictionary behavior.
It's really beautiful work, especially since touching the back bone (the import system) of a language as popular as Python with such a diverse community is super dangerous surgery.
I'm impressed.
edit: ok well "xxx in sys.modules" would indeed be a problem
In fact, all the code you see in the module is "side effects", in a sense. A `class` body, for example, has to actually run at import time, creating the class object and attaching it as an attribute of the module object. Similarly for functions. Even a simple assignment of a constant actually has to run at module import. And all of these things add up.
Further, if there isn't already cached bytecode available for the module, by default it will be written to disk as part of the import process. That's inarguably a side effect.
Sure thing you can declare globals variable and run anything on a module file global scope (outside funcs and class body), but even that 'global' scope is just an illusion, and everything declared there, as yourself said, is scoped to the module's namespace
(and you can't leak the 'globals' when importing the module unless you explicity do so 'from foo import *'. Think of python's import as eval but safer because it doesn't leaks the results from the module execution)
So for a module to have side-effect (for me) it would either:
- Change/Create attributes from other modules
- Call some other function that does side-effect (reflection builtins? IO stuff)
If’s not:
* Importing other modules.
* Taking a long time to import.
* Writing .pyc files.
If any program can “import foo” and still execute exactly the same bytecode afterward as before, you can say that foo doesn’t have side effects.
In general, "import this" and "import antigravity" - anything with import side-effects - would stop working.
Oh, and as the proto-PEP points out, changes to sys.path and others can cause problems because of the delay between time of lazy import and time of resolution.
Then there's code where you do a long computation then make use of a package which might not be present.
All of these would be replaced with "import module; module.__name__" or something to force the import, or by an explicit use to __import__.This can already happen with non top level imports so it is not a necessarily a new issue, but could become more prevalent if there is an overall uptake in this feature for optional dependencies.
I have zero concerns about this PEP and look forward to its implementation.
I have bad memories of using a network filesystem where my Python app's startup time was 5 or more seconds because of all the small file lookups for the import were really slow.
I fixed it by importing modules in functions, only when needed, so the time went down to less than a second. (It was even better using a zipimport, but for other reasons we didn't use that option.)
If I understand things correctly, your code would have the same several-second delay as it tries to resolve everything?
(Trying to do "fallback" logic with lazily-loaded modules is also susceptible to race conditions, of course. What if someone defines the module before you try to use it?)
Also if that doesn't strike your fancy all of the importlib machinery is at your disposal and it's really not very much work to write an import_path() function. It's one of the patterns plug-in systems use and so is stable and expected to be used by end users. No arcane magic required.
haha no
> all of the importlib machinery is at your disposal
This breaks all tooling. Awful option.
`__init__.py` has nothing to do with making this work. It is neither necessary (as of 3.3, yes, you got it right: see https://peps.python.org/pep-0420/) nor sufficient (careless use of sys.path and absolute imports could make it so that the current folder hasn't been imported yet, so you can't even go "up into" it). The folder will already be represented by a module object.
What `__init__.py` does is:
1. Prevents relative imports from also potentially checking in other paths.
2. Provides a space for code that runs before sub-modules, for example to set useful package attributes such as `__all__` (which controls star-imports).
The flexibility of this system also entails that you can in effect define a completely new programming language, describe the process of creating Python bytecode from your custom source, and have clients transparently `import` source in the other language as a result. Or you can define an import process that grabs code from the Internet (not that it would be a good idea...).
If you mean "by explicitly specifying a relative path, and having it be interpreted according to the path of the current module's source code", well first you have to consider that the current module isn't required to have source code. But if it does, then generally it will have been loaded with the default mechanism, which means the module object will have a `__file__` attribute with an absolute path, and you just set your path relative to that.
Using `importlib` is a horrible hack that breaks basically all tooling. You very very obviously are not supposed to do that.
This is an assertion that has absolutely no reasoning behind it. I'm not saying I disagree; I'm just saying there is a time and a place for importlib.
Curious how much reliable Python code was written before those tools existed.
For that matter, curious how much was written before the `types` and `typing` standard library modules appeared.
Exactly. This gives you the flexibility to distribute a complex package across multiple locations.
> Often when you run Python you don't even have a package path.
Any time you successfully import foo.bar, you necessarily have imported foo (because bar is an attribute of that object!), and therefore bar can `from . import` its siblings.
> Using `importlib` is a horrible hack that breaks basically all tooling. You very very obviously are not supposed to do that.
It is exactly as obvious (and true) that you are not "supposed to", in the exact same sense, directly specify where on disk the source code file you want to import is. After all, this constrains the import process to use a source code file. (Similarly if you specify a .pyc directly.) Your relative path doesn't necessarily make any sense after you have packaged and distributed your code and someone else has installed it. It definitely doesn't make any sense if you pack all your modules into a zipapp.
I was skeptical and cautious with it at first but I've since moved large chunks of my codebase to it - it's caused surprisingly few problems (honestly none besides forgetting to handle some import-time registration in some modules) and the speed boost is addictive.
Although I guess that doesn't work in all cases, like defining foreign key relationships when using an orm (like sqlalchemy) for example. But in the orm case, the way to get around that is... lazy resolution :^)
IME circular import errors aren't due to poor organization; they're due to an arbitrary restriction Python has.
I hope this proposal succeeds. I would love to use this feature.
You got three other responses before me all pointing at uv. They are all wrong, because uv did not introduce this functionality to the Python ecosystem. It is a standard defined by https://peps.python.org/pep-0723/, implemented by multiple other tools, notably pipx.
In fact there's already many packages already defining __version__ at a package level.
https://packaging.python.org/en/latest/discussions/versionin...
Edit: What they are solving with UV is at the moment of standing up an environment, but you're more concerned about code-level protection, where are they're more concerned about environment setup protection for versioning.
This only helps for those that do, and it hasn't been any kind of standard the entire time. But more importantly, that helps only the tiniest possible bit with resolving the "import a specific version" syntax. All it solves is letting the file-based import system know whether it found the right folder for the requested (or worse: "a compatible") version of the importable package. It doesn't solve finding the right one if this one is wrong; it doesn't determine how the different versions of the same package are positioned relative to each other in the environment (so that "finding the right one" can work properly); it doesn't solve provisioning the right version. And most importantly, it doesn't solve what happens when there are multiple requests for different versions of the same module at runtime, which incidentally could happen arbitrarily far apart in time, and also the semantics of the code may depend on the same object being used to represent the module in both places.
You're making the common mistake of conflating how things currently work with how things could work if the responsible group agrees to change how things work. Something being the way it is right now is not the same as something else being "not possible".
Yes, you absolutely can create a language that has syntax otherwise identical to Python (or at least damn close) which implements a feature like this. No, you cannot just replace Python with it. If the Python ecosystem just accepted that clearly better things were clearly better, and started using them promptly, we wouldn't have https://pypi.org/project/six/ making it onto https://pypistats.org/top (see also https://sethmlarson.dev/winning-a-bet-about-six-the-python-2...).
Nobody is claiming this is a trivial problem to solve but its also not an impossible problem. Other languages have managed to figure out how to achieve this and still maintain backwards compatibility.
Note that you will be expected to have familiarized yourself generally with previous failed proposals of this sort, and proactively considered all the reasonably obvious corner cases.
If you want examples then just look at one of the other languages that have implemented compiler / runtime dependency version checks.
Even Go has better dependency resolution than Python, and Go is often the HN poster child for how not to do things.
The crux of the matter is this is a solvable problem. The real issue isn’t that it’s technically impossible, is that it’s not annoying enough of a day to day problem for people who are in a position to influence this change. I’m not that person and don’t aspire to be that person (I have plenty of other projects on my plate as it is)
There are many people here who think enabling lazy imports is as simple as flipping a light switch. They have no idea what they're talking about.
This thread was a tangent from lazy imports.
And actually people do appreciate the complexities of changes like this. We were responding to a specific comment that that said “it’s impossible”. Saying something is “possible” isn’t the same as saying “it’s easy”.
Saying something is possible isn’t the same as saying something is easy.
If you were to have said “it’s a different problem to solve because of…” then you wouldn’t have had any pushback. But you didn’t. You said it “this is not possible”. And that’s the part that people were disputing.
Modules being singletons is not a problem in itself I think? This could work like having two versions of the same library in two modules named like library_1_23 and library_1_24. In my program I could hypothetically have imports like `import library_1_23 as library` in one file, and `import library_1_24 as library` in another file. Both versions would be singletons. Then writing `import library==1.23` could be working like syntax sugar for `import library_1_23 as library`.
Of course, having two different versions of a library running in the same program could be a nightmare, so all of that may not be a good idea at all, but maybe not because of module singletons.
Not possible? Come on.
Almost everyone already uses one of a small handfull of conventional ways to specify it, eg `__version__` attribute. It's long overdue that this be standardized so library versions can reliably be introspected at runtime.
Allowing multiple versions to be installed side-by-side and imported explicitly would be a massive improvement.
Some situations could be improved by allowing multiple library versions, but this would introduce new headaches elsewhere. I certainly do not want my program to have N copies of numpy, PyTorch, etc because some intermediate library claims to have just-so dependency tree.
The charitable interpretation of this proposed feature is that it would handle this case exactly as well as the current situation, if the situation isn't improved by the feature.
This feature says nothing about the automatic installation of libraries.
This feature is absolutely not about supporting multiple simultaneous versions of a library at runtime.
In the situation you describe, there would have to be a dependency resolution, just like there is when installing the deps for a program today. It would be good enough for me if "first import wins".
When an installer resolves dependency conflicts, the project code isn't running. The installer is free to discover new constraints on the fly, and to backtrack. It is in effect all being done "statically", in the sense of being ahead of the time that any other system cares about it being complete and correct.
Python `import` statements on the other hand execute during the program's runtime, at arbitrary separation, with other code intervening.
> This feature says nothing about the automatic installation of libraries.
It doesn't have to. The runtime problems still occur.
I guess I'll have to reproduce the basic problem description from memory again. If you have modules A and B in your project that require conflicting versions of C, you need a way to load both at runtime. But the standard import mechanism already hard-codes the assumptions that i) imports are cached in a key-value store; ii) the values are singleton and client code absolutely may rely on this for correctness; iii) "C" is enough information for lookup. And the ecosystem is further built around the assumption that iv) this system is documented and stable and can be interacted with in many clever ways for metaprogramming. Changing any of this would be incredibly disruptive.
> This feature is absolutely not about supporting multiple simultaneous versions of a library at runtime.
You say that, but you aren't the one who proposed it. And https://news.ycombinator.com/item?id=45467350 says explicitly:
> and support having multiple simultaneous versions of any Python library installed.
Which would really be the only reason for the feature. For the cases where a single version of the third-party code satisfies the entire codebase, the existing packaging mechanisms all work fine. (Plus they properly distinguish between import names and distribution names.)
Installed. Not loaded.
The reason is to do away with virtual environments.
I just want to say `import numpy@2.3.x as np` in my code. If 2.3.2 is installed, it gets loaded as the singleton runtime library. If it's not installed, load the closest numpy available and print a warning to stderr. If a transient dependency in the runtime tree wants an incompatible numpy, tough luck, the best you get is a warning message on stderr.
You already have the A, B, C dependency resolution problem you describe today. And if it's not caught at the time of installing your dependencies, you see the failure at runtime.
But virtual environments are quite simply not a big deal. Installed libraries can be hard-linked and maybe even symlinked between environments and this can be set up very quickly. A virtual environment is defined by the pyvenv.cfg marker file; you don't need to use or even have activation scripts, and you especially don't (generally) need a separate copy of pip for each one, even if you do use pip.
On the flip side, allowing multiple versions of a library in a virtual environment has very little effect on package resolution; it just allows success in cases of conflict, but normally there aren't conflicts (because you're typically making a separate environment for a single "root" package, and it's supposed to be possible to use that package in Python as it actually exists, without hacks). The installer still has to scrounge up metadata (and discover it recursively) and check constraints.
import numpy==2.1
And let's say numpy didn't expose a version number in a standard (which could be agreed upon in a PEP) field, then it would just throw an import exception. It wouldn't break any old code. And only packages with that explicit field would support the pinned version import.
And it wouldn't involve trying to extract and parse versions from older packages with some super spotty heuristics.
But it would make new code impossible to use with older versions of python, and older packages, but that's already the case.
Maybe the issue is with module name spacing?
Yes, this part actually is as simple as you imagine. But that means in practical terms that you can't use the feature until at best the next release of Numpy. If you want to say for example that you need at least version 2 (breaking changes, after all), well, there are already 18 published packages that meet that requirement but are unable to communicate that in the new syntax. This can to my understanding be fixed with post releases, but it's a serious burden for maintainers and most projects are just not going to do that sort of thing (it bloats PyPI, too).
And more importantly, that's only one of many problems that need to be solved. And by far the simplest of them.
That sounds like it is absolutely fixable to me, but more of a matter of not having the will to fix it based on some kind of traditionalism. I've used python, a lot. But it is stuff like this that is just maddeningly broken for no good reason at all that has turned me away from it. So as long as I have any alternative I will avoid python because I've seen way too many accidents on account of stuff like this and many lost nights of debugging only to find out that an easily avoidable issue became - once again - the source of much headscratching.
Do you know what happens when Python does summon the will to fix obviously broken things? The Python 2->3 migration happens. (Perl 6 didn't manage any better, either.) Now "Python 3 is the brand" and the idea of version 4 can only ever be entertained as a joke.
However it isn't trivial. First problem coming to my mind:
module a importing first somelib>=1.2.0 and then b and b then requiring somelib>1.2.1 and both being available, will it be the same or will I have a mess from combining?
uv venv —seed —python=3.12 && source .venv/bin/activate && pip3 install requests && …
I should be able to do "python foo.py" and everything should just work. foo.py should define what it wants and python should fetch it and provide it to foo. I should be able to do "pyc foo.py; ./foo" and everything should just work, dependencies balled up and statically included like Rust or Go. Even NodeJS can turn an entire project into one file to execute. That's what a modern language should look and work like.
The moment I see "--this --that" just to run the default version of something you've lost me. This is 2025.
#!/usr/bin/env -S uv run --script # # /// script # requires-python = ">=3.12" # dependencies = ["httpx"] # ///
import httpx
print(httpx.get("https://example.com"))
https://docs.astral.sh/uv/guides/scripts/#improving-reproduc...
There are also projects py2exe pyinstaller iirc and others that try to get the whole static binary thing going.
You’re trying imo to make Python into Golang and if you’re wanting to do that just use Golang. That seems like a far better use of your time.
I don't want to lose multiple hours debugging why something did go wrong because I am using three versions of numpy and seven of torch at the same time and there was a mixup
This is not fearmongering. There is a reason why the only flavor of Python with lazy imports comes from Meta, which is one of the most well-resourced companies in the world.
Too many people in this thread hold the view of "importing {pandas, numpy, my weird module that is more tangled than an eight-player game of Twister} takes too long and I will gladly support anything that makes them faster". I would be willing to bet a large sum of money that most people who hold this opinion are unable to describe how Python's import system works, let alone describe how to implement lazy imports.
PEP 690 describes a number of drawbacks. For example, lazy imports break code that uses decorators to add functions to a central registry. This behavior is crucial for Dash, a popular library for building frontends that has been around for more than a decade. At import-time, Dash uses decorators to bind a JavaScript-based interface to callbacks written in Python. If these imports were made lazy, Dash would break. Frontends used by thousands, if not millions of people, would immediately become unresponsive.
You may cry, "But lazy imports are opt-in! Developers can choose to opt-out of lazy imports if it doesn't work for them." What if these imports were transitive? What if our frontend needed to be completely initialized before starting a critical process, else it would cause a production outage? What if you were a maintainer of a library that was used by millions of people? How could you be sure that adding lazy imports wouldn't break any code downstream? Many people made this argument for type hints, which is sensible because type hints have no effect on runtime behavior*. This is not true for lazy imports; import statements exist in essentially every nontrivial Python program, and changing them to be lazy will fundamentally alter runtime behavior.
This is before we even get to the rest of the issues the PEP describes, which are even weirder and crazier than this. This is a far more difficult undertaking than many people realize.
---
* You can make a program change its behavior based on type annotations, but you'd need to explicitly call into typing APIs to do this. Discussion about this is beyond the scope of this post.
What if we have a program where one feature works only when lazy imports are enabled and one feature only when lazy imports are disabled?
This is not a contrived concern. Let’s say I’m a maintainer of an open-source library and I choose to use lazy imports in my library. Because I’m volunteering my time, I don’t test whether my code works with eager imports.
Now, let’s say someone comes and builds an application on top of this library. It doesn’t work with lazy imports for some unknown reason. If they reach for a “force all imports” flag, their application might break in another mysterious way because the code they depend on is not built to work with eager imports. And even if my dependency doesn’t break, what about all the other packages the application may depend on?
The only solution here would be for the maintainer to ensure that their code works with both lazy and eager imports. However, this imposes a high maintenance cost and is part of the reason why PEP 690 was rejected. (And if your proposed solution was “don’t use libraries made by random strangers on the Internet”, boy do I have news for you...)
My point is that many things _will_ break if migrated to lazy imports. Whether they should have been written in Python in the first place is a separate question that isn’t relevant to this discussion.
Right now in python, you can move import statement inside a function. Lazy imports at top level are not needed. All lazy imports do is make you think less about what you are writing. If you like that, then just vibe code all of your stuff, and leave the language spec alone.
I don't see how. It adds a new, entirely optional syntax using a soft keyword. The semantics of existing code do not change. Yes, yes, you anticipated the objection:
> What if these imports were transitive? ... How could you be sure that adding lazy imports wouldn't break any code downstream?
I would need to see concrete examples of how this would be a realistic risk in principle. (My gut reaction is that top-level code in libraries shouldn't be doing the kinds of things that would be problematic here, in the first place. In my experience, the main thing they do at top level is just eagerly importing everything else for convenience, or to establish compatibility aliases.)
But if it were, clearly that's a breaking change, and the library bumps the major version and clients do their usual dependency version management. As you note, type hints work similarly. And "explicitly calling into typing APIs" is more common than you might think; https://pypistats.org/packages/pydantic exists pretty much to do exactly this. It didn't cause major problems.
> Import statements fundamentally have side effects, and when and how these side effects are applied will cause mysterious breakages that will keep people up for many nights.
They do have side effects that can be arbitrarily complex. But someone who opts in to changing import timing and encounters a difficult bug can just roll back the changes. It shouldn't cause extended debugging sessions unless someone really needs the benefits of the deferral. And people in that situation will have been hand-rolling their own workarounds anyway.
> Too many people in this thread hold the view of "importing {pandas, numpy, my weird module that is more tangled than an eight-player game of Twister} takes too long and I will gladly support anything that makes them faster".
I don't think they're under the impression that this necessarily makes things faster. Maybe I haven't seen the same comments you have.
Deferring imports absolutely would allow, for example, pip to do trivial tasks faster — because it could avoid importing unnecessary things at all. As things currently stand, a huge fraction of the vendored codebase will get imported pretty much no matter what. It's analogous to tree shaking, but implicitly, at runtime and without actually removing code.
Yes, this could be deferred to explicitly chosen times to get more or less the same benefit. It would also be more work.
Regarding risks in practice:
> Libraries such as PyTorch, Numba, NumPy, and SciPy, among others, did not seamlessly align with the deferred module loading approach. These libraries often rely on import side effects and other patterns that do not play well with Lazy Imports. The order in which Python imports could change or be postponed, often led to side effects failing to register classes, functions, and operations correctly. This required painstaking troubleshooting to identify and address import cycles and discrepancies.
This isn't precisely the scenario I described above, but it is a concrete example of how deferred imports can cause issues that are difficult to debug.
Regarding performance benefits:
> At Meta, the quest for faster model training has yielded an exciting milestone: the adoption of Lazy Imports and the Python Cinder runtime. ... we’ve been able to significantly improve our model training times, as well as our overall developer experience (DevX) by adopting Lazy Imports and the Python Cinder runtime.
As this comment mentions Dash apps would not support lazy loaded imports until the underlying Dash library changes how it loads in callbacks and component libraries (the two features which would be most impacted here), but that doesn't mean there's no path to success. We've been discussing some ways we could resolve this internally and if this PEP is accepted we'd certainly go further to see if we can fully support lazy loaded imports (of both the Dash library itself/Dash component libraries and for relative imports in Dash apps).
From merely browsing through a few comments, people have mostly positive opinions regarding this proposal. Then why did it fail many times, but not this time? What drives the success behind this PEP?
What I want is for imports to not suck and be slow. I’ve had projects where it was faster to compile and run C++ than launch and start a Python CLI. It’s so bad.
I know/heard there are "some" (which I haven't seen by the way) libraries that depend on import side effects, but the advantage is much bigger.
First of all, the circular import problem will go away, especially on type hints. Although there was a PEP or recent addition to make the annotation not not cause such issue.
Second and most important of all, is the launch time of Python applications. A CLI that uses many different parts of the app has to wait for all the imports to be done.
The second point becomes a lot painful when you have a large application, like a Django project where the auto reload becomes several seconds. Not only auto reload crawls, the testing cycle is slow as well. Every time you want to run test command, it has to wait several seconds. Painful.
So far the solution has been to do the lazy import by importing inside the methods where it's required. That is something, I never got to like to be honest.
Maybe it will be fixed in Python 4, where the JIT uses the type hints as well /s
I’m pretty sure there will be new keywords in Python in the future that only solve one thing.
Introducing new keyword has become a recent thing in Python.
Seems Python has a deep scare since Python2 to Python3 time and is scared to do anything that causes such drama again.
For me, the worst of all is "async". If 2to3 didn't cause much division, the async definitely divided Python libraries in 2. Sync and Async.
Maybe if they want backward compatible solution, this can be done by some compile or runtime flag like they did with free threading no-gil.
My main complaint, though, about Python async is - because it is opt-in I never know if I forgot a sync IO call somewhere that will block my worker. JS makes everything async by default and there is effectively no chance of blocking.
They took a decade to solidify that. At some point, you have to balance evolution and stability. For a language as popular as Python, you can not break the world every week, but you can't stay put for 5 years.
Right now all the imports are getting resolved at runtime example in a code like below
When you write this, the entire file1 module is executed right away, which may trigger side effects.If lazy imports suddenly defer execution, those side effects won’t run until much later (or not at all, if the code path isn’t hit). That shift in timing could easily break existing code that depends on import-time behavior.
To avoid using lazy, this there is also a proposal of adding the modules you want to load lazily to a global `__lazy_modules__` variable.
The problem with new keywords is that you have to stick to the newest Python version every time a new keyword is added. Older Python versions will give a syntax error. It's a big problem for libraries. You need to wait for 3-5 years before adding it to a library. There are a lot of people who still use Python 3.8 from 2019.
But I don’t think I really agree, the extensible annotation syntaxes they mention always feel clunky and awkward to me. For a first-party language feature (especially used as often as this will be), I think dedicated syntax seems right.
Just might as well add `defer` keyword like Golang.
I ended up adding a note to the plugin author docs suggesting lazy loading inside of functions - https://llm.datasette.io/en/stable/plugins/advanced-model-pl... - but having a core Python language feature for this would be really nice.
Note that this is global to the entire process, so for example if you make an import of Numpy lazy this way, then so are the imports of all the sub-modules. Meaning that large parts of Numpy might not be imported at all if they aren't needed, but pauses for importing individual modules might be distributed unpredictably across the runtime.
Edit: from further experimentation, it appears that if the source does something like `import foo.bar.baz` then `foo` and `foo.bar` will still be eagerly loaded, and only `foo.bar.baz` itself is deferred. This might be part of what the PEP meant by "mostly". But it might also be possible to improve my implementation to fix that.
https://pep-previews--4622.org.readthedocs.build/pep-0810/#f...
Q: Why not use importlib.util.LazyLoader instead?
A: LazyLoader has significant limitations:
Requires verbose setup code for each lazy import.
Has ongoing performance overhead on every attribute access.
Doesn’t work well with from ... import statements.
Less clear and standard than dedicated syntax.
> Has ongoing performance overhead on every attribute access.
I would have expected so, but in my testing it seems like the lazy load does some kind of magic to replace the proxy with the real thing. I haven't properly dug into it, though. It appears this point is removed in the live version (https://peps.python.org/pep-0810).
> Doesn’t work well with from ... import statements.
Hmm. The PEP doesn't seem to explain how reification works in this case. Per the above it's a solved problem for modules; I guess for the from-imports it could be made to work essentially the same way. Presumably this involves the proxy holding a reference to the namespace where the import occurred. That probably has a lot to do with restricting the syntax to top level. (Which is the opposite of how we've seen soft keywords used before!)
> Requires verbose setup code for each lazy import.
> Less clear and standard than dedicated syntax.
If you want to use it in a fine-grained way, then sure.
Only do imports when you know you need them -- or as an easy approximation, only if the easy command line options have been handled and there's still something to do.
https://llm.datasette.io/en/stable/plugins/plugin-hooks.html...
I'm happy with the solution I have now, which is to encourage plugin authors not to import PyTorch or other heavy dependencies at the root level of their plugin code.
That might be considered a design mistake -- one that should be easy to migrate away from.
You won't need to do anything, of course, if the lazy import becomes available on common Python installs some day in the future. That might take years, though.
Bad performing third party plugins are user error.
If a tool has different capabilities that use different imports, why load all of them if only a subset is required?
As a simple example, a tool that can generate output in various formats (e.g., json, csv, xml, ...) should only import the appropriate modules to handle the output format after having determined which ones will be used in this invocations.
That's not to say this PEP should not be accepted. One could always apply a no-lazy-imports style rule or disable it via global lazy import control.
https://peps.python.org/pep-0810/#global-lazy-imports-contro...
- It reduces visibility into a module’s dependencies.
- It increases the risk of introducing circular dependencies later on.
However, there is a pattern in python to raise an error if, say, pandas doesn't have an excel library installed, which is fine. In the future, will maintainers opt to include a bunch of unused libraries since they won't negatively impact startup time? (Think pandas including 3-4 excel parsers by default, since it will only be loaded when called). It's a much better UX, but, now if you opt out of lazy loading, your code will take longer to load than without it.
80 more comments available on Hacker News