Skip to content

Commit

Permalink
Merge bitcoin#30991: test: enable running independent functional test…
Browse files Browse the repository at this point in the history
… sub-tests

409d0d6 test: enable running individual independent functional test methods (ismaelsadeeq)

Pull request description:

  - Some test methods in the functional test framework are independent and do not require any prior context or setup in `run_test`.
  - This commit adds a new option for running these specific methods within a test file, allowing them to be executed individually without running the entire test suite.
  - Using this option reduces the time you need to wait before the test you are interested in starts executing.
  - The functionality added by this PR can be achieved manually by commenting out code, but having a pragmatic option to do this is more convenient.

  Note: Running test methods that require arguments or context will fail.

  **Example Usage**:
  ```zsh
  build/test/functional/feature_reindex.py --test_methods continue_reindex_after_shutdown
  ```

  ```zsh
  build/test/functional/feature_config_args.py --test_methods test_log_buffer test_args_log test_connect_with_seednode
  ```

ACKs for top commit:
  maflcko:
    review ACK 409d0d6
  rkrux:
    reACK 409d0d6
  ryanofsky:
    Code review ACK 409d0d6. This seems like a good step towards making it easy to run independent tests quickly. I think ideally there would be some naming convention or @ annotation added to test methods that can run independently, so the test framework could provide more functionality like being able to list test methods, being able to show command lines to quickly reproduce problems when tests fails, and calling test methods automatically instead of requiring individual tests to call them. But these ideas are all compatible with the new `--test_methods` option

Tree-SHA512: b0daac7c3b322e6fd9b946962335d8279e8cb004ff76f502c8d597b9c4b0073840945be198a79d44c5aaa64bda421429829d5c84ceeb8c6139eb6ed079a35878
  • Loading branch information
ryanofsky committed Dec 2, 2024
2 parents 1927674 + 409d0d6 commit ebe4cac
Showing 1 changed file with 14 additions and 1 deletion.
15 changes: 14 additions & 1 deletion test/functional/test_framework/test_framework.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,11 @@ def main(self):

try:
self.setup()
self.run_test()
if self.options.test_methods:
self.run_test_methods()
else:
self.run_test()

except JSONRPCException:
self.log.exception("JSONRPC error")
self.success = TestStatus.FAILED
Expand All @@ -155,6 +159,13 @@ def main(self):
exit_code = self.shutdown()
sys.exit(exit_code)

def run_test_methods(self):
for method_name in self.options.test_methods:
self.log.info(f"Attempting to execute method: {method_name}")
method = getattr(self, method_name)
method()
self.log.info(f"Method '{method_name}' executed successfully.")

def parse_args(self, test_file):
previous_releases_path = os.getenv("PREVIOUS_RELEASES_DIR") or os.getcwd() + "/releases"
parser = argparse.ArgumentParser(usage="%(prog)s [options]")
Expand Down Expand Up @@ -194,6 +205,8 @@ def parse_args(self, test_file):
help="use BIP324 v2 connections between all nodes by default")
parser.add_argument("--v1transport", dest="v1transport", default=False, action="store_true",
help="Explicitly use v1 transport (can be used to overwrite global --v2transport option)")
parser.add_argument("--test_methods", dest="test_methods", nargs='*',
help="Run specified test methods sequentially instead of the full test. Use only for methods that do not depend on any context set up in run_test or other methods.")

self.add_options(parser)
# Running TestShell in a Jupyter notebook causes an additional -f argument
Expand Down

0 comments on commit ebe4cac

Please sign in to comment.