Result of query filtering by arch seems to differ according to arch of the platform code is run on

[AdamWill]

We have this releng script called spam-o-matic which attempts to list broken deps in a compose per arch (it's basically a repoclosure test).

Here's the key code:

def get_unresolved_deps(dbo, arch):
    # Modified from repoclosure plugin from dnf-plugins-core
    unresolved = {}

    deps = set()
    available = dbo.sack.query().available()
    if dbo.conf.best:
        available = available.latest()
    if arch is not None:
        available = available.filter(arch=[arch, "noarch"])
    pkgs = set()
    for pkgs_filtered in available:
        pkgs.add(pkgs_filtered)

    for pkg in pkgs:
        unresolved[pkg] = set()
        for req in pkg.requires:
            reqname = str(req)
            # XXX: https://bugzilla.redhat.com/show_bug.cgi?id=1186721
            if reqname.startswith("solvable:") or \
                    reqname.startswith("rpmlib("):
                continue
            deps.add(req)
            unresolved[pkg].add(req)

    available.apply()
    unresolved_deps = set(x for x in deps if not available.filter(provides=x))

    unresolved_transition = {k: set(x for x in v if x in unresolved_deps)
                             for k, v in unresolved.items()}
    return {k: v for k, v in unresolved_transition.items() if v}

Now for some reason, this is behaving differently when run on the 'native' arch (e.g. running this query for x86_64 on an x86_64 machine) than when run on a 'non-native' arch (e.g. running this query for ppc64le on an x86_64 machine), as discussed here.

You can try out the script by downloading it, then getting enough repodata to make it happy. I created a directory called 20220920.n.0 with this layout:

[adamw@toolbox scripts (main *%)]$ ls -R 20220920.n.0/
20220920.n.0/:
ppc64le  x86_64

20220920.n.0/ppc64le:
os

20220920.n.0/ppc64le/os:
repodata

20220920.n.0/ppc64le/os/repodata:
232ade33f124be5f85ad190a6c77cb215d05e3a6b10c67ade927a1d96ebff042-primary.sqlite.xz
23c00c2eb8bdce8c6c2e8ba8dbf41b6abefc62df0575423ac971e1588298913e-filelists.xml.zck
27fb6206b3f8ed589da91f74848f712761bc4f83b259a27f0313355eb8cfc5e2-primary.xml.gz
3629caefc3e92f978bacc9f62ad641797a1fdb63688c6bdeeb262a2e039bcb25-comps-Everything.ppc64le.xml
5835fcf46d24f5cc7133c2b792caeea4f2b3e9367e9a98add8db543133e79944-primary.xml.zck
7fc5d8f0307dee66412763a718f2ea6ccb5230f5093992f5f47866acd47ef6b3-other.xml.gz
82ec318d02e9cec69a1634fb90ce0c8bf79b1055856fc263b0d04efbee251386-comps-Everything.ppc64le.xml.zck
8cb85d3d821ea592930c120b4702277bd6cd8358866be7722f8e200e80543e66-other.sqlite.xz
9353b37cfa17340af22718e458cff6db0bde092305905d21a11695de48fbc075-filelists.xml.gz
9733c4da1221d51bb6a253a1730d69046e3dffbc21166ca5c27c72e0041ac38d-other.xml.zck
e2fc2b1ce662178ff91ebb573ad8d6a3b9968172cae15039d29739c120749984-comps-Everything.ppc64le.xml.xz
repomd.xml

20220920.n.0/x86_64:
os

20220920.n.0/x86_64/os:
repodata

20220920.n.0/x86_64/os/repodata:
1b6f31e343e34f04394c4c176416a0208485d189b36164882d05db7bedf747de-other.xml.zck
5d4071cf957b9dab48fd4560154672001241713d264a6ed4046626f1a600b1f5-other.sqlite.xz
664051aab26039b06f56d2f7d54a30037b4da90c1f4362ca5816e4f19e186e5b-comps-Everything.x86_64.xml.xz
6b4bbd7960ce9cdf1fe24e630320f8ffbb45b12653239a0f9edf3ead2be0d4f2-primary.xml.zck
88c925581a16f6ccbc9bd011b1f1760d564aa4fe4b34dd0955bb15d70a939540-other.xml.gz
93903cce85fd4b4422aba4d66856e588dfb68c0342b0b8d72763da1c349e41ba-filelists.sqlite.xz
97540ee9775293c7b4ebb852385dcd9def595242344a459e1bd24b9ca5498f4b-primary.sqlite.xz
97d353dba225394e8780e72060ba9b9bb77029495d475b267979e3aafb031353-filelists.xml.gz
a8c63799a742d98327001c5ee752de75d9d43645925fad34916ba4b5e3833532-primary.xml.gz
ae66a3262bcbcc0d9b36a46c1fcd3bdd2bc8496c5849451eb7be10fba50e12b2-comps-Everything.x86_64.xml.zck
bfc08eebde588175e2b12e7b2c13bfb1ed68dfcb62acc9da1f2b373f8ed2db5d-filelists.xml.zck
c09c78e2c8c7b13a8ae5441106b36140ecfa43ae37dc73ee1f4bcc0b1272a2ca-comps-Everything.x86_64.xml
repomd.xml

the files are grabbed from under https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20220920.n.0/compose/Everything/ . Then you can run e.g. ./spam-o-matic --nomail --only-arches=ppc64le 20220920.n.0 to query just for ppc64le, or ./spam-o-matic --nomail --only-arches=x86_64 20220920.n.0 to query just for x86_64.

If you run it on an x86_64 system, the results for x86_64 will be correct but the results for ppc64le will be completely wrong, basically it thinks nothing at all is provided.

It seems to me like things go wrong at unresolved_deps = set(x for x in deps if not available.filter(provides=x)). That available.filter(provides=x) query just seems to not work right when it's not run on the 'native' arch. For instance, if I hack up the code to do this right after available.apply():

print("XXX LIBM PROVIDES OF GLIBC:")
glibc = [pkg for pkg in available if pkg.name == "glibc"][0]
for provide in glibc.provides:
    if "libm" in provide.name:
        print(provide.name)
print("XXX DEP TEST FOR libm.so.6(GLIBC_2.27)(64bit)")
gotprovs = available.filter(provides="libm.so.6(GLIBC_2.27)(64bit)")
if gotprovs:
    for pkg in gotprovs:
        print("PROVIDER: " + pkg.name)
else:
    print("NO PROVIDER!")

then running that on x86_64 for x86_64 arch prints:

XXX LIBM PROVIDES OF GLIBC:
libm.so.6()(64bit)
libm.so.6(GLIBC_2.2.5)(64bit)
libm.so.6(GLIBC_2.29)(64bit)
libm.so.6(GLIBC_2.27)(64bit)
libm.so.6(GLIBC_2.35)(64bit)
libm.so.6(GLIBC_2.23)(64bit)
libmvec.so.1()(64bit)
libmvec.so.1(GLIBC_2.22)(64bit)
libmvec.so.1(GLIBC_2.35)(64bit)
libm.so.6(GLIBC_2.32)(64bit)
libm.so.6(GLIBC_2.25)(64bit)
libm.so.6(GLIBC_2.26)(64bit)
libm.so.6(GLIBC_2.18)(64bit)
libm.so.6(GLIBC_2.28)(64bit)
libm.so.6(GLIBC_2.15)(64bit)
libm.so.6(GLIBC_2.24)(64bit)
libm.so.6(GLIBC_2.31)(64bit)
libm.so.6(GLIBC_2.4)(64bit)
libmemusage.so()(64bit)
XXX DEP TEST FOR libm.so.6(GLIBC_2.27)(64bit)
PROVIDER: glibc

but running it on x86_64 for ppc64le arch prints:

XXX LIBM PROVIDES OF GLIBC:
libm.so.6()(64bit)
libm.so.6(GLIBC_2.17)(64bit)
libm.so.6(GLIBC_2.29)(64bit)
libm.so.6(GLIBC_2.27)(64bit)
libm.so.6(GLIBC_2.35)(64bit)
libm.so.6(GLIBC_2.32)(64bit)
libm.so.6(GLIBC_2.23)(64bit)
libm.so.6(GLIBC_2.26)(64bit)
libm.so.6(GLIBC_2.18)(64bit)
libm.so.6(GLIBC_2.25)(64bit)
libm.so.6(GLIBC_2.28)(64bit)
libm.so.6(GLIBC_2.24)(64bit)
libm.so.6(GLIBC_2.31)(64bit)
libmemusage.so()(64bit)
XXX DEP TEST FOR libm.so.6(GLIBC_2.27)(64bit)
NO PROVIDER!

i.e. the available query definitely has an entry for a glibc package that provides libm.so.6(GLIBC_2.27)(64bit) in both cases, but for some reason, available.filter(provides="libm.so.6(GLIBC_2.27)(64bit)") does not find it when run in the 'not-native arch' case. It does find it in the 'native arch' case.

This really seems like a bug to me, I can't find any indication that this is intended behaviour, and it does not feel like intended behaviour.

[AdamWill]

This is with dnf-4.13.0-3.fc37.noarch and libdnf-0.67.0-4.fc37.x86_64, btw. On the ppc64le test box I have dnf-4.13.0-1.fc36.noarch and libdnf-0.67.0-2.fc36.ppc64le.

[hroncok]

I've recently encountered a similar problem and had to set dbo.conf.arch to the arch I was looking for. Does that make a difference for you?

[AdamWill]

aha! yes, that does seem to help.

I actually went looking for exactly such a setting, but the docs page does not mention its existence at all. I didn't bother just blindly trying it anyway. I guess I should have done. :D

Well, I guess the page links to https://dnf.readthedocs.io/en/latest/conf_ref.html#conf-main-options-label , and that does mention it. None of this is really obvious, though, nor why you should need to set that in order for this to work...

[m-blaha]

Setting the dbo.conf.arch is an API equivalent of using --forcearch=ppc64le option in dnf CLI. The thing is that dnf needs to know the architecture (e.g. for variables substitutions in repository URLs) the default arch value is autodetected (using uname() syscall).
But sure, the documentation can always be better and we should have mentioned that arch option is not only for installing packages but also for repodata retrieving and querying packages for foreign architectures.
Also we didn't want to repeat all the config options in class dnf.conf.Conf documentation so we went with those links to relevant sections of dnf.conf documentation.