Skip to content

Commit

Permalink
Minor comments and name changes.
Browse files Browse the repository at this point in the history
- Also, adjust benchmark functions to not capture import hook uninstall time.
  • Loading branch information
Sachaa-Thanasius committed Oct 10, 2024
1 parent e2cac6a commit 2b7349b
Show file tree
Hide file tree
Showing 4 changed files with 41 additions and 28 deletions.
12 changes: 6 additions & 6 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ defer-imports
:alt: PyPI supported Python versions


A library that implements `PEP 690 <pep_690_text_>`_–esque lazy imports in pure Python.
A library that implements `PEP 690`_–esque lazy imports in pure Python.

**Note: This is still in development.**

Expand Down Expand Up @@ -69,7 +69,7 @@ The function call's result can be used as a context manager, which makes sense w
Making this call without arguments allows user code with imports contained within the ``defer_imports.until_use`` context manager to be deferred until referenced. However, its several configuration parameters allow toggling global instrumentation (affecting all import statements) and adjusting the granularity of that global instrumentation.

**WARNING: Avoid using the hook as anything other than a context manager when passing in configuration; otherwise, the explicit (or default) configuration will persist and may cause other packages using ``defer_imports`` to behave differently than expected.**
**WARNING: Avoid using the hook as anything other than a context manager when passing in module-specific configuration; otherwise, the explicit (or default) configuration will persist and may cause other packages using ``defer_imports`` to behave differently than expected.**

.. code-block:: python
Expand Down Expand Up @@ -155,7 +155,7 @@ Caveats
- May clash with other import hooks.

- Examples of popular packages using clashing import hooks: |typeguard|_, |beartype|_, |jaxtyping|_, |torchtyping|_, |pyximport|_
- It's possible to work around this by reaching into ``defer-imports``'s internals and combining its instrumentation machinery with that of another package's, but it's currently not supported well beyond the presence of a ``loader_class`` parameter in ``defer_imports.install_import_hook()``'s signature.
- It's possible to work around this by reaching into ``defer-imports``'s internals and combining its instrumentation machinery with that of another package's, but it's currently not supported well beyond ``defer_imports.install_import_hook()`` accepting a ``loader_class`` argument.

- Can't automatically resolve deferred imports in a namespace when that namespace is being iterated over, leaving a hole in its abstraction.

Expand All @@ -167,7 +167,7 @@ Caveats
Why?
====

Lazy imports alleviate several of Python's current pain points. Because of that, `PEP 690 <pep_690_text_>`_ was put forth to integrate lazy imports into CPython; see that proposal and the surrounding discussions for more information about the history, implementations, benefits, and costs of lazy imports.
Lazy imports alleviate several of Python's current pain points. Because of that, `PEP 690`_ was put forth to integrate lazy imports into CPython; see that proposal and the surrounding discussions for more information about the history, implementations, benefits, and costs of lazy imports.

Though that proposal was rejected, there are well-established third-party libraries that provide lazy import mechanisms, albeit with more constraints. Most do not have APIs as integrated or ergonomic as PEP 690's, but that makes sense; most predate the PEP and were not created with that goal in mind.

Expand Down Expand Up @@ -249,7 +249,7 @@ The design of this library was inspired by the following:
- |metamodule|_
- |modutil|_
- `SPEC 1 <https://scientific-python.org/specs/spec-0001/>`_ / |lazy-loader|_
- `PEP 690 and its authors <pep_690_text_>`_
- `PEP 690`_ and its authors
- `Jelle Zijlstra's pure-Python proof of concept <https://gist.github.com/JelleZijlstra/23c01ceb35d1bc8f335128f59a32db4c>`_
- |slothy|_
- |ideas|_
Expand All @@ -262,7 +262,7 @@ Without them, this would not exist.
Common/formatted hyperlinks

.. _pep_690_text: https://peps.python.org/pep-0690/
.. _PEP 690: https://peps.python.org/pep-0690/

.. |timeit| replace:: ``timeit``
.. _timeit: https://docs.python.org/3/library/timeit.html
Expand Down
14 changes: 8 additions & 6 deletions bench/bench_samples.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,18 @@ def bench_slothy() -> float:


def bench_defer_imports_local() -> float:
with CatchTime() as ct: # noqa: SIM117
with defer_imports.install_import_hook(uninstall_after=True):
import bench.sample_defer_local
with CatchTime() as ct:
hook_ctx = defer_imports.install_import_hook()
import bench.sample_defer_local
hook_ctx.uninstall()
return ct.elapsed


def bench_defer_imports_global() -> float:
with CatchTime() as ct: # noqa: SIM117
with defer_imports.install_import_hook(uninstall_after=True, apply_all=True):
import bench.sample_defer_global
with CatchTime() as ct:
hook_ctx = defer_imports.install_import_hook(apply_all=True)
import bench.sample_defer_global
hook_ctx.uninstall()
return ct.elapsed


Expand Down
23 changes: 12 additions & 11 deletions bench/generate_samples.py
Original file line number Diff line number Diff line change
Expand Up @@ -568,33 +568,34 @@

def main() -> None:
bench_path = Path("bench").resolve()
tests_path = bench_path.with_name("tests")

# ---- regular imports
regular_contents = "\n".join((_PYRIGHT_IGNORE_DIRECTIVES, _GENERATED_BY_COMMENT, _STDLIB_IMPORTS))
regular_path = bench_path / "sample_regular.py"
regular_path.write_text(regular_contents, encoding="utf-8")
sample_reg_contents = "\n".join((_PYRIGHT_IGNORE_DIRECTIVES, _GENERATED_BY_COMMENT, _STDLIB_IMPORTS))
sample_reg_path = bench_path / "sample_regular.py"
sample_reg_path.write_text(sample_reg_contents, encoding="utf-8")

# ---- defer_imports-instrumented and defer_imports-hooked imports (global)
shutil.copy(regular_path, regular_path.with_name("sample_defer_global.py"))
shutil.copy(sample_reg_path, sample_reg_path.with_name("sample_defer_global.py"))

# ---- defer_imports-instrumented and defer_imports-hooked imports (local)
defer_imports_contents = _CONTEXT_MANAGER_TEMPLATE.format(
sample_defer_contents = _CONTEXT_MANAGER_TEMPLATE.format(
import_stmt="import defer_imports",
ctx_manager="with defer_imports.until_use:",
)
defer_imports_path = bench_path / "sample_defer_local.py"
defer_imports_path.write_text(defer_imports_contents, encoding="utf-8")
sample_defer_path = bench_path / "sample_defer_local.py"
sample_defer_path.write_text(sample_defer_contents, encoding="utf-8")

# ---- defer_imports-influenced imports (local), but for a test in the tests directory
shutil.copy(defer_imports_path, bench_path.with_name("tests") / "sample_stdlib_imports.py")
shutil.copy(sample_defer_path, tests_path / "sample_stdlib_imports.py")

# ---- slothy-hooked imports
slothy_contents = _CONTEXT_MANAGER_TEMPLATE.format(
sample_slothy_contents = _CONTEXT_MANAGER_TEMPLATE.format(
import_stmt="from slothy import lazy_importing",
ctx_manager="with lazy_importing():",
)
slothy_path = bench_path / "sample_slothy.py"
slothy_path.write_text(slothy_contents, encoding="utf-8")
sample_slothy_path = bench_path / "sample_slothy.py"
sample_slothy_path.write_text(sample_slothy_contents, encoding="utf-8")


if __name__ == "__main__":
Expand Down
20 changes: 15 additions & 5 deletions src/defer_imports/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,17 +37,20 @@
def _lazy_import_module(name: str, package: typing.Optional[str] = None) -> types.ModuleType:
"""Lazily import a module. Has the same signature as ``importlib.import_module()``.
This is purely for limited internal usage, especially since it has not been evaluated for thread safety.
This is for limited internal usage, especially since it intentionally does not handle certain edge cases and has
not been evaluated for thread safety.
Notes
-----
Based on importlib code as well as recipes found in the Python 3.12 importlib docs.
"""

# 1. Resolve the name.
absolute_name = importlib.util.resolve_name(name, package)
if absolute_name in sys.modules:
return sys.modules[absolute_name]

# 2. Find the module's parent if it exists.
path = None
if "." in absolute_name:
parent_name, _, child_name = absolute_name.rpartition(".")
Expand All @@ -56,6 +59,7 @@ def _lazy_import_module(name: str, package: typing.Optional[str] = None) -> type
assert parent_module.__spec__ is not None
path = parent_module.__spec__.submodule_search_locations

# 3. Find the module spec.
for finder in sys.meta_path:
spec = finder.find_spec(absolute_name, path)
if spec is not None:
Expand All @@ -64,17 +68,21 @@ def _lazy_import_module(name: str, package: typing.Optional[str] = None) -> type
msg = f"No module named {absolute_name!r}"
raise ModuleNotFoundError(msg, name=absolute_name)

if spec.loader is None:
# 4. Wrap the module loader.
if spec.loader is not None:
spec.loader = loader = importlib.util.LazyLoader(spec.loader)
else:
msg = "missing loader"
raise ImportError(msg, name=spec.name)

spec.loader = loader = importlib.util.LazyLoader(spec.loader)
# 5. Execute and return the module
# 5.1. Account for the module replacing itself in sys.modules.
module = importlib.util.module_from_spec(spec)
sys.modules[absolute_name] = module
loader.exec_module(module)

if path is not None:
setattr(parent_module, child_name, module) # pyright: ignore [reportPossiblyUnboundVariable]
setattr(parent_module, child_name, sys.modules[absolute_name]) # pyright: ignore [reportPossiblyUnboundVariable]

return sys.modules[absolute_name]

Expand Down Expand Up @@ -198,7 +206,8 @@ def _sliding_window(
Examples
--------
>>> tokens = list(tokenize.generate_tokens(io.StringIO("def func(): ...").readline))
>>> source = "def func(): ..."
>>> tokens = list(tokenize.generate_tokens(io.StringIO(source).readline))
>>> [" ".join(item.string for item in window) for window in _sliding_window(tokens, 2)]
['def func', 'func (', '( )', ') :', ': ...', '... ', ' ']
"""
Expand Down Expand Up @@ -260,6 +269,7 @@ def _calc___package__(globals: coll_abc.MutableMapping[str, typing.Any]) -> typi

msg = f"__package__ != __spec__.parent ({package!r} != {spec.parent!r})"
warnings.warn(msg, category, stacklevel=3)

return package
elif spec is not None:
return spec.parent
Expand Down

0 comments on commit 2b7349b

Please sign in to comment.