Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PERF: refactor atef.bin cli entrypoint to defer imports #256

Closed
wants to merge 3 commits into from

Conversation

tangkong
Copy link
Contributor

@tangkong tangkong commented Sep 17, 2024

Description

  • takes a knife to atef.bin to defer imports until absolutely necessary

In more detail:

  • moves all build_arg_parser type functions to a module separate from the subcommand logic
  • atef.bin.main now grabs those parsers to create the top-level parser, avoiding the meaty functional imports
  • the main sub-commands are given an additional layer of wrapping, and atef.bin.main:main now calls a partial function to get the atef.bin.<subcommand>:main function, which then gets fed the parsed kwargs.

Motivation and Context

I keep vendoring this cli hook and never improving it. This was more of a proof to myself that it could work. If we like it I can start using it elsewhere or even maybe cookiecutter it.

I should really work on more important tasks but I refactored it in my brain during a post-dinner walk and wanted to put it to paper

How Has This Been Tested?

Before

$ time atef config -h
...
real    0m8.487s
user    0m6.721s
sys     0m1.158s

After

$ time atef config -h
...
real    0m0.518s
user    0m0.184s
sys     0m0.075s

Aside from this, lots of interactive testing to make sure all the entrypoints are working.

Where Has This Been Documented?

This PR

Pre-merge checklist

  • Code works interactively
  • Code contains descriptive docstrings, including context and API
  • New/changed functions and methods are covered in the test suite where possible
  • Code has been checked for threading issues (no blocking tasks in GUI thread)
  • Test suite passes locally
  • Test suite passes on GitHub Actions
  • Ran docs/pre-release-notes.sh and created a pre-release documentation page
  • Pre-release docs include context, functional descriptions, and contributors as appropriate

@tangkong tangkong requested a review from ZLLentz September 17, 2024 04:04
Copy link
Member

@ZLLentz ZLLentz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems reasonable to me and is a big speedup due to some of the slower imports in some of the code paths. My only in-line review comment is a typo check.

I have mixed feelings about the various variants/options for a subparser scheme like this one. The previous arrangement had some performance problems but functionally made clear separations between the implementations of the different subcommands. Part of me wonders if there was a way to keep the subparsers a little more separate without needing to change so much, something like:

atef.bin.subcmd

import argparse

def build_parser():
    ...

def main(**kwargs):
    from .subcmd_main import subcmd_main
    subcmd_main(**kwargs)

And then the overall structure doesn't change too much and we still import a bunch of subparsers but the heavy imports are deferred and the subparsers aren't collocated.

But maybe there are even cleaner ways as well.

This is all just musing. I'm happy to merge with the typo fixed.


Maintenance
-----------
- improves the performance of the CLI entrypoint, defering functional imports as longas possible
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- improves the performance of the CLI entrypoint, defering functional imports as longas possible
- improves the performance of the CLI entrypoint, deferring functional imports as long as possible

@tangkong
Copy link
Contributor Author

Hmm I might like that formulation more. I'll give that a whirl

@tangkong
Copy link
Contributor Author

superceded by #257

@tangkong tangkong closed this Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants