Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

create a fake ntp module for FAM tests #16997

Merged
merged 3 commits into from
Nov 25, 2024
Merged

create a fake ntp module for FAM tests #16997

merged 3 commits into from
Nov 25, 2024

Conversation

evgeni
Copy link
Member

@evgeni evgeni commented Nov 25, 2024

Problem Statement

The ntp module in the fixtures is too old, but we also only need a few params from it.

Solution

Drop the install code and use the fake code to create a module as the tests expect

Related Issues

@evgeni evgeni requested a review from a team as a code owner November 25, 2024 10:55
@evgeni evgeni added CherryPick PR needs CherryPick to previous branches 6.16.z Introduced in or relating directly to Satellite 6.16 labels Nov 25, 2024
@evgeni
Copy link
Member Author

evgeni commented Nov 25, 2024 

trigger: test-robottelo
pytest: tests/foreman/sys/test_fam.py::test_positive_run_modules_and_roles[smart_class_parameter_override_value] tests/foreman/sys/test_fam.py::test_positive_run_modules_and_roles[smart_class_parameter] tests/foreman/sys/test_fam.py::test_positive_run_modules_and_roles[config_group]

@Satellite-QE
Copy link
Collaborator

PRT Result

Build Number: 9452
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/sys/test_fam.py::test_positive_run_modules_and_roles[smart_class_parameter_override_value] tests/foreman/sys/test_fam.py::test_positive_run_modules_and_roles[smart_class_parameter] tests/foreman/sys/test_fam.py::test_positive_run_modules_and_roles[config_group] --external-logging
Test Result : ================== 3 passed, 30 warnings in 987.52s (0:16:27) ==================

@Satellite-QE Satellite-QE added the PRT-Passed Indicates that latest PRT run is passed for the PR label Nov 25, 2024
@evgeni
Copy link
Member Author

evgeni commented Nov 25, 2024

Before this change, smart_class_parameter_override_value and smart_class_parameter were failing \o/

@jameerpathan111 jameerpathan111 added the AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing label Nov 25, 2024
@jameerpathan111 jameerpathan111 merged commit 39769a5 into SatelliteQE:master Nov 25, 2024
11 of 12 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 25, 2024
@evgeni @web-flow
create a fake ntp module for FAM tests (#16997)
7fcb005 
* use sat.put() instead of sat.execute(echo) to place puppet files

* allow passing in custom module code

* create a fake ntp module

the one in the repo is too old

(cherry picked from commit 39769a5)
@Satellite-QE Satellite-QE mentioned this pull request Nov 25, 2024
Merged
ekohl
ekohl reviewed Nov 25, 2024
View reviewed changes
Copy link
Contributor

@ekohl ekohl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had the same problem in bats, back in 2020 and switched over to theforeman/motd in theforeman/forklift@db0b263. The reason we have a real module there is that we want to verify it actually applied changes to the filesystem. Doesn't robottelo do some real end-to-end testing with verifying the report?

I see there is a tar file in this repository but I can't see what actually uses that file.

@evgeni
Copy link
Member Author

evgeni commented Nov 25, 2024

We had the same problem in bats, back in 2020 and switched over to theforeman/motd in theforeman/forklift@db0b263. The reason we have a real module there is that we want to verify it actually applied changes to the filesystem. Doesn't robottelo do some real end-to-end testing with verifying the report?

This are the FAM tests. We are verifying that FAM can instruct Foreman to import a module and assign things from it. We do not apply anything to actual hosts.

@ekohl
Copy link
Contributor

ekohl commented Nov 25, 2024

Right, but since you were making changes I wanted to better understand the testing.

evgeni added a commit to evgeni/robottelo that referenced this pull request Dec 13, 2024
@evgeni
create a fake ntp module for FAM tests (SatelliteQE#16997)
ea54db8 
* use sat.put() instead of sat.execute(echo) to place puppet files

* allow passing in custom module code

* create a fake ntp module

the one in the repo is too old

(cherry picked from commit 39769a5)
evgeni added a commit to evgeni/robottelo that referenced this pull request Dec 13, 2024
@evgeni
create a fake ntp module for FAM tests (SatelliteQE#16997)
0b18341 
* use sat.put() instead of sat.execute(echo) to place puppet files

* allow passing in custom module code

* create a fake ntp module

the one in the repo is too old

(cherry picked from commit 39769a5)
jameerpathan111 pushed a commit that referenced this pull request Dec 13, 2024
@evgeni
cherry-picks of FAM test fixes for 6.15.z (#17164)
2bb7b1e 
* disable pytest plugin autoloading when running FAM tests (#16975)

pytest by default tries to load all plugins it finds on a system.
pulpcore ships with own pytest plugins, but we do not install their
dependencies (as we don't want to run pulpcore tests), which leads to
pytest failing to load those plugins and breaking overall execution:

    Traceback (most recent call last):
      File "/usr/bin/pytest-3.11", line 33, in <module>
        sys.exit(load_entry_point('pytest==7.2.0', 'console_scripts', 'pytest')())
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 190, in console_main
        code = main()
               ^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 148, in main
        config = _prepareconfig(args, plugins)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 329, in _prepareconfig
        config = pluginmanager.hook.pytest_cmdline_parse(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
        return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
        return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_callers.py", line 55, in _multicall
        gen.send(outcome)
      File "/usr/lib/python3.11/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
        config: Config = outcome.get_result()
                         ^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
        raise ex[1].with_traceback(ex[2])
      File "/usr/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
        res = hook_impl.function(*args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1058, in pytest_cmdline_parse
        self.parse(args)
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1346, in parse
        self._preparse(args, addopts=addopts)
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1229, in _preparse
        self.pluginmanager.load_setuptools_entrypoints("pytest11")
      File "/usr/lib/python3.11/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
        plugin = ep.load()
                 ^^^^^^^^^
      File "/usr/lib64/python3.11/importlib/metadata/__init__.py", line 202, in load
        module = import_module(match.group('module'))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
      File "/usr/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 168, in exec_module
        exec(co, module.__dict__)
      File "/usr/lib/python3.11/site-packages/pulp_ansible/pytest_plugin.py", line 3, in <module>
        import numpy as np
    ModuleNotFoundError: No module named 'numpy'

Disable the autoloading by setting the PYTEST_DISABLE_PLUGIN_AUTOLOAD
environment variable.

(cherry picked from commit 6a91dd9)

* simplify FAM test command (#16980)

- we don't need to export vars, setting them should be sufficient
- make can change directories for us, no need to cd

(cherry picked from commit fefe1ea)

* create a fake ntp module for FAM tests (#16997)

* use sat.put() instead of sat.execute(echo) to place puppet files

* allow passing in custom module code

* create a fake ntp module

the one in the repo is too old

(cherry picked from commit 39769a5)

* better logging of fam failures (#16967)

(cherry picked from commit 04976cc)
jameerpathan111 pushed a commit that referenced this pull request Dec 13, 2024
@evgeni
cherry-picks of FAM test fixes for 6.14.z (#17165)
9278652 
* Create fake galaxy.yml to make the FAM Makefile happy (#16936)

The data in the file is unused, but not being able to load it produces
errors in the logs and is confusing when searching for an actual problem
during testing.

(cherry picked from commit c45e681)

* disable pytest plugin autoloading when running FAM tests (#16975)

pytest by default tries to load all plugins it finds on a system.
pulpcore ships with own pytest plugins, but we do not install their
dependencies (as we don't want to run pulpcore tests), which leads to
pytest failing to load those plugins and breaking overall execution:

    Traceback (most recent call last):
      File "/usr/bin/pytest-3.11", line 33, in <module>
        sys.exit(load_entry_point('pytest==7.2.0', 'console_scripts', 'pytest')())
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 190, in console_main
        code = main()
               ^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 148, in main
        config = _prepareconfig(args, plugins)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 329, in _prepareconfig
        config = pluginmanager.hook.pytest_cmdline_parse(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_hooks.py", line 265, in __call__
        return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_manager.py", line 80, in _hookexec
        return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_callers.py", line 55, in _multicall
        gen.send(outcome)
      File "/usr/lib/python3.11/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
        config: Config = outcome.get_result()
                         ^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/pluggy/_result.py", line 60, in get_result
        raise ex[1].with_traceback(ex[2])
      File "/usr/lib/python3.11/site-packages/pluggy/_callers.py", line 39, in _multicall
        res = hook_impl.function(*args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1058, in pytest_cmdline_parse
        self.parse(args)
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1346, in parse
        self._preparse(args, addopts=addopts)
      File "/usr/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1229, in _preparse
        self.pluginmanager.load_setuptools_entrypoints("pytest11")
      File "/usr/lib/python3.11/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
        plugin = ep.load()
                 ^^^^^^^^^
      File "/usr/lib64/python3.11/importlib/metadata/__init__.py", line 202, in load
        module = import_module(match.group('module'))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
      File "/usr/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 168, in exec_module
        exec(co, module.__dict__)
      File "/usr/lib/python3.11/site-packages/pulp_ansible/pytest_plugin.py", line 3, in <module>
        import numpy as np
    ModuleNotFoundError: No module named 'numpy'

Disable the autoloading by setting the PYTEST_DISABLE_PLUGIN_AUTOLOAD
environment variable.

(cherry picked from commit 6a91dd9)

* simplify FAM test command (#16980)

- we don't need to export vars, setting them should be sufficient
- make can change directories for us, no need to cd

(cherry picked from commit fefe1ea)

* create a fake ntp module for FAM tests (#16997)

* use sat.put() instead of sat.execute(echo) to place puppet files

* allow passing in custom module code

* create a fake ntp module

the one in the repo is too old

(cherry picked from commit 39769a5)

* better logging of fam failures (#16967)

(cherry picked from commit 04976cc)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ekohl ekohl ekohl left review comments

@jameerpathan111 jameerpathan111 jameerpathan111 approved these changes

Assignees

@evgeni evgeni

Labels
6.16.z Introduced in or relating directly to Satellite 6.16 AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing CherryPick PR needs CherryPick to previous branches PRT-Passed Indicates that latest PRT run is passed for the PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@evgeni @Satellite-QE @ekohl @jameerpathan111