Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplify FAM test command #16980

Merged
merged 1 commit into from
Nov 22, 2024
Merged

simplify FAM test command #16980

merged 1 commit into from
Nov 22, 2024

Conversation

evgeni
Copy link
Member

@evgeni evgeni commented Nov 21, 2024

Problem Statement

The command we use to execute FAM tests is complicated and uses shell chaining which is not well readable

Solution

  • we don't need to export vars, setting them should be sufficient
  • make can change directories for us, no need to cd

Related Issues

- we don't need to export vars, setting them should be sufficient
- make can change directories for us, no need to cd
@evgeni evgeni requested a review from a team as a code owner November 21, 2024 15:21
@evgeni
Copy link
Member Author

evgeni commented Nov 21, 2024

trigger: test-robottelo
pytest: tests/foreman/sys/test_fam.py
env:
    BROKER_SSH_BACKEND: hussh

@Satellite-QE
Copy link
Collaborator

PRT Result

Build Number: 9435
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/sys/test_fam.py --external-logging
Test Result : =========== 17 failed, 93 passed, 352 warnings in 4954.49s (1:22:34) ===========

@Satellite-QE Satellite-QE added the PRT-Failed Indicates that latest PRT run is failed for the PR label Nov 21, 2024
@evgeni
Copy link
Member Author

evgeni commented Nov 21, 2024

trigger: test-robottelo
pytest: tests/foreman/sys/test_fam.py -vv
env:
    BROKER_SSH_BACKEND: hussh

@Satellite-QE
Copy link
Collaborator

PRT Result

Build Number: 9440
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/sys/test_fam.py -vv --external-logging
Test Result : =========== 17 failed, 93 passed, 443 warnings in 5366.39s (1:29:26) ===========

@evgeni
Copy link
Member Author

evgeni commented Nov 21, 2024

trigger: test-robottelo
pytest: tests/foreman/sys/test_fam.py -vv

@Satellite-QE
Copy link
Collaborator

PRT Result

Build Number: 9442
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/sys/test_fam.py -vv --external-logging
Test Result : =========== 16 failed, 94 passed, 441 warnings in 7910.99s (2:11:50) ===========

@jameerpathan111 jameerpathan111 added CherryPick PR needs CherryPick to previous branches AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing Stream Introduced in or relating directly to Satellite Stream/Master 6.16.z Introduced in or relating directly to Satellite 6.16 labels Nov 22, 2024
@jameerpathan111 jameerpathan111 merged commit fefe1ea into master Nov 22, 2024
10 of 13 checks passed
@jameerpathan111 jameerpathan111 deleted the fam-command branch November 22, 2024 06:48
github-actions bot pushed a commit that referenced this pull request Nov 22, 2024
- we don't need to export vars, setting them should be sufficient
- make can change directories for us, no need to cd

(cherry picked from commit fefe1ea)
jameerpathan111 pushed a commit that referenced this pull request Nov 22, 2024
simplify FAM test command (#16980)

- we don't need to export vars, setting them should be sufficient
- make can change directories for us, no need to cd

(cherry picked from commit fefe1ea)

Co-authored-by: Evgeni Golov <[email protected]>
evgeni added a commit to evgeni/robottelo that referenced this pull request Dec 13, 2024
- we don't need to export vars, setting them should be sufficient
- make can change directories for us, no need to cd

(cherry picked from commit fefe1ea)
evgeni added a commit to evgeni/robottelo that referenced this pull request Dec 13, 2024
- we don't need to export vars, setting them should be sufficient
- make can change directories for us, no need to cd

(cherry picked from commit fefe1ea)
jameerpathan111 pushed a commit that referenced this pull request Dec 13, 2024
* disable pytest plugin autoloading when running FAM tests (#16975)

pytest by default tries to load all plugins it finds on a system.
pulpcore ships with own pytest plugins, but we do not install their
dependencies (as we don't want to run pulpcore tests), which leads to
pytest failing to load those plugins and breaking overall execution:

    Traceback (most recent call last):
      File "/usr/bin/pytest-3.11", line 33, in <module>
        sys.exit(load_entry_point('pytest==7.2.0', 'console_scripts', 'pytest')())
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 190, in console_main
        code = main()
               ^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 148, in main
        config = _prepareconfig(args, plugins)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 329, in _prepareconfig
        config = pluginmanager.hook.pytest_cmdline_parse(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
        return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
        return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_callers.py", line 55, in _multicall
        gen.send(outcome)
      File "/usr/lib/python3.11/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
        config: Config = outcome.get_result()
                         ^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
        raise ex[1].with_traceback(ex[2])
      File "/usr/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
        res = hook_impl.function(*args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1058, in pytest_cmdline_parse
        self.parse(args)
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1346, in parse
        self._preparse(args, addopts=addopts)
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1229, in _preparse
        self.pluginmanager.load_setuptools_entrypoints("pytest11")
      File "/usr/lib/python3.11/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
        plugin = ep.load()
                 ^^^^^^^^^
      File "/usr/lib64/python3.11/importlib/metadata/__init__.py", line 202, in load
        module = import_module(match.group('module'))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
      File "/usr/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 168, in exec_module
        exec(co, module.__dict__)
      File "/usr/lib/python3.11/site-packages/pulp_ansible/pytest_plugin.py", line 3, in <module>
        import numpy as np
    ModuleNotFoundError: No module named 'numpy'

Disable the autoloading by setting the PYTEST_DISABLE_PLUGIN_AUTOLOAD
environment variable.

(cherry picked from commit 6a91dd9)

* simplify FAM test command (#16980)

- we don't need to export vars, setting them should be sufficient
- make can change directories for us, no need to cd

(cherry picked from commit fefe1ea)

* create a fake ntp module for FAM tests (#16997)

* use sat.put() instead of sat.execute(echo) to place puppet files

* allow passing in custom module code

* create a fake ntp module

the one in the repo is too old

(cherry picked from commit 39769a5)

* better logging of fam failures (#16967)

(cherry picked from commit 04976cc)
jameerpathan111 pushed a commit that referenced this pull request Dec 13, 2024
* Create fake galaxy.yml to make the FAM Makefile happy (#16936)

The data in the file is unused, but not being able to load it produces
errors in the logs and is confusing when searching for an actual problem
during testing.

(cherry picked from commit c45e681)

* disable pytest plugin autoloading when running FAM tests (#16975)

pytest by default tries to load all plugins it finds on a system.
pulpcore ships with own pytest plugins, but we do not install their
dependencies (as we don't want to run pulpcore tests), which leads to
pytest failing to load those plugins and breaking overall execution:

    Traceback (most recent call last):
      File "/usr/bin/pytest-3.11", line 33, in <module>
        sys.exit(load_entry_point('pytest==7.2.0', 'console_scripts', 'pytest')())
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 190, in console_main
        code = main()
               ^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 148, in main
        config = _prepareconfig(args, plugins)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 329, in _prepareconfig
        config = pluginmanager.hook.pytest_cmdline_parse(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
        return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
        return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_callers.py", line 55, in _multicall
        gen.send(outcome)
      File "/usr/lib/python3.11/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
        config: Config = outcome.get_result()
                         ^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
        raise ex[1].with_traceback(ex[2])
      File "/usr/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
        res = hook_impl.function(*args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1058, in pytest_cmdline_parse
        self.parse(args)
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1346, in parse
        self._preparse(args, addopts=addopts)
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1229, in _preparse
        self.pluginmanager.load_setuptools_entrypoints("pytest11")
      File "/usr/lib/python3.11/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
        plugin = ep.load()
                 ^^^^^^^^^
      File "/usr/lib64/python3.11/importlib/metadata/__init__.py", line 202, in load
        module = import_module(match.group('module'))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
      File "/usr/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 168, in exec_module
        exec(co, module.__dict__)
      File "/usr/lib/python3.11/site-packages/pulp_ansible/pytest_plugin.py", line 3, in <module>
        import numpy as np
    ModuleNotFoundError: No module named 'numpy'

Disable the autoloading by setting the PYTEST_DISABLE_PLUGIN_AUTOLOAD
environment variable.

(cherry picked from commit 6a91dd9)

* simplify FAM test command (#16980)

- we don't need to export vars, setting them should be sufficient
- make can change directories for us, no need to cd

(cherry picked from commit fefe1ea)

* create a fake ntp module for FAM tests (#16997)

* use sat.put() instead of sat.execute(echo) to place puppet files

* allow passing in custom module code

* create a fake ntp module

the one in the repo is too old

(cherry picked from commit 39769a5)

* better logging of fam failures (#16967)

(cherry picked from commit 04976cc)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.16.z Introduced in or relating directly to Satellite 6.16 AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing CherryPick PR needs CherryPick to previous branches PRT-Failed Indicates that latest PRT run is failed for the PR Stream Introduced in or relating directly to Satellite Stream/Master
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants