rhui: do not bootstrap target client on aws

[MichalHe]

Bootstrapping target RHUI client on 7>8 AWS is not possible as the RHEL8 RHUI client now depends on virtually the entire RHEL8 RPM stack.

This patch turns the bootstrapping off. Leapp will rely only on the files bundled in the leapp-rhui- package.

[github-actions]

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

• review please @oamg/developers to notify leapp developers of the review request
• /packit copr-build to submit a public copr build using packit

Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build. If you need a different version of leapp from PR#42, use /packit test oamg/leapp#42

It is possible to schedule specific on-demand tests as well. Currently 2 test sets are supported, beaker-minimal and kernel-rt, both can be used to be run on all upgrade paths or just a couple of specific ones.
To launch on-demand tests with packit:

• /packit test --labels kernel-rt to schedule kernel-rt tests set for all upgrade paths
• /packit test --labels beaker-minimal-8.9to9.3,kernel-rt-8.9to9.3 to schedule kernel-rt and beaker-minimal test sets for 8.9->9.3 upgrade path

[Deprecated] To launch on-demand regression testing public members of oamg organization can leave the following comment:

• /rerun to schedule basic regression tests using this pr build and latest upstream leapp build as artifacts
• /rerun 42 to schedule basic regression tests using this pr build and leapp*PR42* as artifacts
• /rerun-sst to schedule sst tests using this pr build and latest upstream leapp build as artifacts
• /rerun-sst 42 to schedule sst tests using this pr build and leapp*PR42* as artifacts

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra.

[alexxa]

copr builds

• 7to8: leapp-upgrade-el7toel8-deps-0.19.0-0.20240215140250854304.pr1178.74.g3883b3fe.el7.noarch leapp-upgrade-el7toel8-0.19.0-0.20240215140250854304.pr1178.74.g3883b3fe.el7.noarch)
  ** leapp-rhui-aws-1.0.9-1.el7.noarch.rpm
  ** leapp-rhui-aws-sap-e4s-1.0.9-1.el7.noarch.rpm
  ** leapp-rhui-azure-1.0.0-12.el7.noarch.rpm
  ** leapp-rhui-azure-sap-1.0.0-12.el7.noarch.rpm
  ** leapp-rhui-google-1.0.0-4.el7.noarch.rpm
  ** leapp-rhui-google-sap-1.0.0-4.el7.noarch.rpm
• 8to9: leapp-upgrade-el7toel8-deps-0.19.0-0.20240215140250854304.pr1178.74.g3883b3fe.el8.noarch leapp-upgrade-el7toel8-0.19.0-0.20240215140250854304.pr1178.74.g3883b3fe.el8.noarch
  ** same as above for el8

	AWS	Azure (v12)	GCP
vanilla 7.9 to 8.8	pass	pass after extending /var	pass but with --nogpgcheck
sap-haus 7.9 to 8.8	pass	pass	pass but with --nogpgcheck
sap-apps 7.9 to 8.8	n/a	pass	n/a
vanilla 8.8 to 9.2	failed_		failed
sap-haus 8.8 to 9.2	failed_
sap-apps 8.8 to 9.2	n/a		n/a

8to9 fail:

Risk Factor: high (error)
Title: Actor target_userspace_creator unexpectedly terminated with exit code: 1
Summary: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/leapp/repository/actor_definition.py", line 74, in _do_run
    actor_instance.run(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/leapp/actors/__init__.py", line 289, in run
    self.process(*args)
  File "/etc/leapp/repos.d/system_upgrade/common/actors/targetuserspacecreator/actor.py", line 58, in process
    userspacegen.perform()
  File "/usr/lib/python3.6/site-packages/leapp/utils/deprecation.py", line 42, in process_wrapper
    return target_item(*args, **kwargs)
  File "/etc/leapp/repos.d/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py", line 1243, in perform
    setup_target_rhui_access_if_needed(context, indata)
  File "/etc/leapp/repos.d/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py", line 1217, in setup_target_rhui_access_if_needed
    dest = get_copy_location_from_copy_in_task(context, copy_task)
  File "/etc/leapp/repos.d/system_upgrade/common/actors/targetuserspacecreator/libraries/userspacegen.py", line 858, in get_copy_location_from_copy_in_task
    dest_in_container = os.path.join(context_basepath, copy_task.dst)
  File "/usr/lib64/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NspawnActions

brew builds

	AWS	Azure	GCP
vanilla 7.9to8.8	fail	pass after extending /var	pass but with --nogpgcheck
sap-haus 7.9to8.8	fail	pass
sap-apps 7.9to8.8	n/a	pass	n/a
vanilla 8.8to9.2	pass	pass
sap-haus 8.8to9.2	nothing provides leapp-framework >= 5.0 needed by leapp-upgrade-el8toel9-0.20.0-1.el8.noarch	nothing provides leapp-framework >= 5.0 needed by leapp-upgrade-el8toel9-0.20.0-1.el8.noarch	nothing provides leapp-framework >= 5.0 needed by leapp-upgrade-el8toel9-0.20.0-1.el8.noarch
sap-apps 8.8to9.2	n/a	nothing provides leapp-framework >= 5.0 needed by leapp-upgrade-el8toel9-0.20.0-1.el8.noarch	n/a

[Rezney]

@alexxa Thanks for the thorough testing. The traceback you shared is a valid issue and we will take a look at it. For the nothing provides leapp-framework >= 5.0 needed by leapp-upgrade-el8toel9-0.20.0-1.el8.noarch, I believe you need to use corresponding "leapp" package. Either from master or the final build.

[fernflower]

/packit test --labels kernel-rt-8.9to9.3

[alexxa]

• 7to8
  leapp-upgrade-el7toel8.noarch 0:0.19.0-0.20240219085502630737.pr1178.74.gd65c7470.el7
  leapp-upgrade-el7toel8-deps.noarch 0:0.19.0-0.20240219085502630737.pr1178.74.gd65c7470.el7
  leapp.noarch 0:0.17.0-100.20240213170309274358.master.0.g1c9e2ee.el7
  leapp-deps.noarch 0:0.17.0-100.20240213170309274358.master.0.g1c9e2ee.el7
  pciutils.x86_64 0:3.5.1-3.el7
  python2-leapp.noarch 0:0.17.0-100.20240213170309274358.master.0.g1c9e2ee.el7

  leapp-rhui-aws-1.0.7-1.el7.noarch.rpm (production!!!)
  leapp-rhui-aws-sap-e4s-1.0.7-1.el7.noarch.rpm (production!!!)
  leapp-rhui-azure-1.0.0-12.el7.noarch.rpm
  leapp-rhui-azure-sap-1.0.0-12.el7.noarch.rpm
  leapp-rhui-google-1.0.0-4.el7.noarch.rpm
  leapp-rhui-google-sap-1.0.0-4.el7.noarch.rpm

• 8to9
  ** same as above for el8

	AWS	Azure	GCP
vanilla 7.9 to 8.8	pass	pass after /var extend	pass with --nogpgcheck
sap-haus 7.9 to 8.8	pass	pass	pass with --nogpgcheck
sap-apps 7.9 to 8.8	n/a	pass	n/a
vanilla 8.8 to 9.2	pass	FAIL	pass
sap-haus 8.8 to 9.2	pass	pass	pass
sap-apps 8.8 to 9.2	n/a	pass	n/a

GCP --nogpgcheck issue on 7to8, on sap example:

Packages installed

leapp-0.17.0-100.20240213170309274358.master.0.g1c9e2ee.el7.noarch
leapp-upgrade-el7toel8-deps-0.19.0-0.20240219085502630737.pr1178.74.gd65c7470.el7.noarch
python2-leapp-0.17.0-100.20240213170309274358.master.0.g1c9e2ee.el7.noarch
leapp-upgrade-el7toel8-0.19.0-0.20240219085502630737.pr1178.74.gd65c7470.el7.noarch
leapp-deps-0.17.0-100.20240213170309274358.master.0.g1c9e2ee.el7.noarch
leapp-rhui-google-sap-1.0.0-4.el7.noarch

error:

Risk Factor: high (inhibitor)
Title: Detected unknown GPG keys for target system repositories
Summary: Some of the target repositories require GPG keys that are not installed in the current RPM DB or are not stored in the /etc/leapp/repos.d/system_upgrade/common/files/rpm-gpg/8 directory. Leapp is not able to guarantee validity of such gpg keys and manual review is required, so any spurious keys are not imported in the system during the in-place upgrade. The following additional gpg keys are required to be imported during the upgrade:
    - https://packages.cloud.google.com/yum/doc/yum-key.gpg
Remediation: [hint] Check the path to the listed GPG keys is correct, the keys are valid and import them into the host RPM DB or store them inside the /etc/leapp/repos.d/system_upgrade/common/files/rpm-gpg/8 directory prior the upgrade. If you want to proceed the in-place upgrade without checking any RPM signatures, execute leapp with the `--nogpgcheck` option.
Key: e7e3f34a530e861b14f1cdd759cb62f92677066f
----------------------------------------
Risk Factor: high
Title: Difference in Python versions and support in RHEL 8
Summary: In RHEL 8, there is no 'python' command. Python 3 (backward incompatible) is the primary Python version and Python 2 is available with limited support and limited set of packages. If you no longer require Python 2 packages following the upgrade, please remove them. Read more here: https://red.ht/rhel-8-python
Related links:
    - Difference in Python versions and support in RHEL 8: https://red.ht/rhel-8-python
Remediation: [hint] Please run "alternatives --set python /usr/bin/python3" after upgrade
Key: 0c98585b1d8d252eb540bf61560094f3495351f5
----------------------------------------
Risk Factor: high
Title: Packages not signed by Red Hat found on the system
Summary: The following packages have not been signed by Red Hat and may be removed during the upgrade process in case Red Hat-signed packages to be removed during the upgrade depend on them:
- epel-release
- gce-disk-expand
- google-cloud-cli
- google-cloud-sap-agent
- google-compute-engine
- google-compute-engine-oslogin
- google-guest-agent
- google-osconfig-agent
- leapp
- leapp-deps
- leapp-upgrade-el7toel8
- leapp-upgrade-el7toel8-deps
- python2-leapp

@Rezney asked to check production GCP, on vanilla example:

leapp-upgrade-el7toel8-deps-0.19.0-1.el7_9.noarch
leapp-deps-0.16.0-1.el7_9.noarch
leapp-rhui-google-1.0.0-2.el7.noarch
python2-leapp-0.16.0-1.el7_9.noarch
leapp-0.16.0-1.el7_9.noarch
leapp-upgrade-el7toel8-0.19.0-1.el7_9.noarch

Risk Factor: high (inhibitor)
Title: Detected unknown GPG keys for target system repositories
Summary: Some of the target repositories require GPG keys that are not installed in the current RPM DB or are not stored in the /etc/leapp/repos.d/system_upgrade/common/files/rpm-gpg/8 directory. Leapp is not able to guarantee validity of such gpg keys and manual review is required, so any spurious keys are not imported in the system during the in-place upgrade. The following additional gpg keys are required to be imported during the upgrade:
    - https://packages.cloud.google.com/yum/doc/yum-key.gpg
Remediation: [hint] Check the path to the listed GPG keys is correct, the keys are valid and import them into the host RPM DB or store them inside the /etc/leapp/repos.d/system_upgrade/common/files/rpm-gpg/8 directory prior the upgrade. If you want to proceed the in-place upgrade without checking any RPM signatures, execute leapp with the `--nogpgcheck` option.
Key: e7e3f34a530e861b14f1cdd759cb62f92677066f
----------------------------------------
Risk Factor: high
Title: Packages not signed by Red Hat found on the system
Summary: The following packages have not been signed by Red Hat and may be removed during the upgrade process in case Red Hat-signed packages to be removed during the upgrade depend on them:
- epel-release
- gce-disk-expand
- google-cloud-cli
- google-compute-engine
- google-compute-engine-oslogin
- google-guest-agent
- google-osconf

azure vanilla 8to9, ask me for that VM if needed, @Rezney has keys added already and IP shared in Slack, error:

Microsoft Azure RPMs for Red Hat Enterprise Lin  11 kB/s | 1.9 kB     00:00    
No match for argument: dnf-command(config-manager)
No match for argument: dnf
No match for argument: kpatch-dnf

============================================================
                           ERRORS                           
============================================================

2024-02-19 12:24:58.687304 [ERROR] Actor: target_userspace_creator
Message: Unable to install RHEL 9 userspace packages.
Summary:
    Details: Command ['systemd-nspawn', '--register=no', '--quiet', '--keep-unit', '--capability=all', '-D', '/var/lib/leapp/scratch/mounts/root_/system_overlay', '--setenv=LEAPP_HOSTNAME=monday-rhel-88to92-copr.internal.cloudapp.net', '--setenv=LEAPP_EXPERIMENTAL=0', '--setenv=LEAPP_UNSUPPORTED=0', '--setenv=LEAPP_NO_RHSM=1', '--setenv=LEAPP_NO_RHSM_FACTS=1', '--setenv=LEAPP_UPGRADE_PATH_TARGET_RELEASE=9.2', '--setenv=LEAPP_UPGRADE_PATH_FLAVOUR=default', '--setenv=LEAPP_IPU_IN_PROGRESS=8to9', '--setenv=LEAPP_EXECUTION_ID=d31bdf77-abbc-4eb2-adac-635426b2077c', '--setenv=LEAPP_COMMON_TOOLS=:/etc/leapp/repos.d/system_upgrade/common/tools:/etc/leapp/repos.d/system_upgrade/el8toel9/tools', '--setenv=LEAPP_COMMON_FILES=:/etc/leapp/repos.d/system_upgrade/common/files:/etc/leapp/repos.d/system_upgrade/el8toel9/files', 'dnf', 'install', '-y', '--setopt=module_platform_id=platform:el9', '--setopt=keepcache=1', '--releasever', '9.2', '--installroot', '/el9target', '--disablerepo', '*', '--enablerepo', 'rhui-microsoft-azure-rhel9', 'dnf-command(config-manager)', 'rhui-azure-rhel9', 'dnf', 'kpatch-dnf', '--disableplugin', 'subscription-manager'] failed with exit code 1.
    Stderr: Host and machine ids are equal (c30654a8e7de4cd68b942774c4e1ccca): refusing to link journals
            Error: Unable to find a match: dnf-command(config-manager) dnf kpatch-dnf

[alexxa]

@Rezney it hit me with GCP it can be due to epel-repo. I recall last time we discussed it with GCP. When removed, it all goes fine for 7to8

[alexxa]

after updating the RHUI cleint rpm on failed Azure VM, the issue is gone. @MichalHe thanks