Allow to run test at build time

[schaefi]

This change runs the unit tests at build time of the package. It increases the build time a lot because the test run is expensive. I created this pull request as a draft and open for a conversation if we want this. The Debian package maintainer @glaubitz runs the test at build time

[Conan-Kudo]

What's with "breaking system packages"? Also pip can't work in Logo or OBS like this.

Why not just ensure test dependencies are specified as BuildRequires?

[schaefi]

Ok so here is the story:

running pytest without an installed kiwi fails at a million places because kiwi cannot be found as a module. One would think setting PYTHONPATH properly would fix this but it simply does not work. @glaubitz can tell you a looong frustrating story about this.

Long story short simply installing kiwi via pip into the users pip cache fixes all of this trouble and allows python and pytest to find kiwi. It's also the same approach when installing kiwi from pip. As you moved kiwi to poetry and we have standard wheel package without any extra script code or magic, the install of the .whl is a stable way to make kiwi available to the system, no matter if it's an obs worker or a chroot or a debian gbp chroot, or any other system

Also pip can't work in Logo or OBS like this

This is not correct because the way pip is used here will not pull in any dependencies. All dependencies are satisfied by the proper BuildRequires as you can see it from the patch here in this PR

So the call of pip will see all deps resolved, and will simply install the .whl in a way that python can consume it.

What's with "breaking system packages"

SUSE doesn't want users to use pip and want to force them to install packages. Because of that reason the default behavior is that the tool tells you that you should install a package and if you think this is not correct use the --break-system-packages option. That's why I have to call it that way. I'm not sure if this option is generally available in pip but if not we need to differentiate the call between the distros

Overall I was also hesitant to call pip to make kiwi available to python but with the standard .whl package and all deps resolved via the packaging it seems the simplest and most stable way to allow to run the tests against the kiwi code from source. Setting up a venv via poetry or other tools always failed because that pulls in all deps from the network which does not work in obs and other buildsystems.

I have spent many hours and tried many different paths. So far this few lines at least worked local and in obs.

[schaefi]

I'm still not sure if we want to run the tests with every package build, but @glaubitz had a good point when he said that we will not find issues due to python dependency updates done in the distributions. We ensure that no upstream release reached pypi without passing all tests, but all those tests run as github action on some Ubuntu system fetching dependencies from pip and not from the distribution. So it probably makes sense to let the tests run at package build time too

[Conan-Kudo]

How did you try to run the tests before this pip method?

[Conan-Kudo]

In Fedora, we have a %pytest macro that's defined effectively like so:

PATH="%{buildroot}%{_bindir}:$PATH" PYTHONPATH="${PYTHONPATH:-%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}}" PYTHONDONTWRITEBYTECODE=1 pytest

[schaefi]

That does not work for me

osc build --clean openSUSE_Tumbleweed x86_64

[   31s] + cd test/unit
[   31s] + PATH=/home/abuild/rpmbuild/BUILDROOT/python-kiwi-10.0.22-0.x86_64/usr/bin:/usr/local/bin:/usr/bin:/bin
[   31s] + PYTHONPATH='/home/abuild/rpmbuild/BUILDROOT/python-kiwi-10.0.22-0.x86_64%{pyth
[   31s] on3_sitearch:/home/abuild/rpmbuild/BUILDROOT/python-kiwi-10.0.22-0.x86_64/usr/lib/python3.11/site-packages}'
[   31s] + PYTHONDONTWRITEBYTECODE=1
[   31s] + pytest
[   33s] ============================= test session starts ==============================
[   33s] platform linux -- Python 3.11.9, pytest-7.4.4, pluggy-1.3.0
[   33s] rootdir: /home/abuild/rpmbuild/BUILD/kiwi-10.0.22
[   33s] configfile: setup.cfg
[   33s] collected 0 items / 180 errors
[   33s] 
[   33s] ==================================== ERRORS ====================================
[   33s] ________________ ERROR collecting test/unit/api_helper_test.py _________________
[   33s] ImportError while importing test module '/home/abuild/rpmbuild/BUILD/kiwi-10.0.22/test/unit/api_helper_test.py'.
[   33s] Hint: make sure your test modules/packages have valid Python names.
[   33s] Traceback:
[   33s] /usr/lib64/python3.11/importlib/__init__.py:126: in import_module
[   33s]     return _bootstrap._gcd_import(name[level:], package, level)
[   33s] api_helper_test.py:3: in <module>
[   33s]     from kiwi.api_helper import (
[   33s] E   ModuleNotFoundError: No module named 'kiwi'

... and a lot more of these

[schaefi]

How did you try to run the tests before this pip method?

we did not run them in the package build so far. If you have enabled that in your Fedora spec file I would be really interested in how you make it to work. As I said I tried for days without any success and @glaubitz also didn't found a method that worked for Debian

[schaefi]

I added a %check section to cover this part and added a comment to the spec file why the pip way was the only way to install into userland such that importlib finds the kiwi namespace.

I still could not find another way. @Conan-Kudo if you have a better idea please advise