Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

goal: Do not add lower versions to upgrade jobs #878

Closed
wants to merge 2 commits into from

Conversation

m-blaha
Copy link
Member

@m-blaha m-blaha commented Sep 7, 2023

For UPGRADE* actions, do not add packages that are older than the
currently installed versions to the libsolv job. This decreases the
number of potential solutions and helps resolve some edge cases.

See https://issues.redhat.com/browse/RHEL-1448 for one of those edge
cases.

Test: rpm-software-management/ci-dnf-stack#1380

@m-blaha m-blaha force-pushed the mblaha/upgrade-versions branch from f62ab7f to 3423249 Compare September 7, 2023 12:20
m-blaha added a commit to rpm-software-management/ci-dnf-stack that referenced this pull request Sep 7, 2023
@m-blaha m-blaha requested a review from j-mracek September 7, 2023 14:10
@m-blaha m-blaha marked this pull request as ready for review September 7, 2023 14:17
// Remove packages of versions lower than the installed ones.
// Substract downgrades instead of filter_upgrades() to keep
// installed packages in the query.
rpm::PackageQuery tmp_downgrades(query);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am quite curious, why you want to keep packages with the same version as installed in the query?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I don't :) I've just overlooked it. Thanks for noticing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recall that "dnf upgrade" reinstalls packages which were originally installed unsigned from a local build and then they appeared properly signed in a repository. That could warrant leaving same versions in the query.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recall that "dnf upgrade" reinstalls packages which were originally installed unsigned from a local build and then they appeared properly signed in a repository. That could warrant leaving same versions in the query.

I need to check this. We do not have a test for it, I suppose?

Copy link
Member Author

@m-blaha m-blaha Sep 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ppisar it seems that you are refering to behaviour described in this libsolv issue: openSUSE/libsolv#287
The trigger for re-installation was not whether the package was or was not signed, but more generally that the package was changed (for example has different build time).
This behavior was changed in dnf4 here: rpm-software-management/libdnf#786 (and the same behavior we have in dnf5).
So dnf4 upgrade does not reinstall package with the same nevra but different buildtime for quite a long time. But distrosync command should trigger the reinstallation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that libsolv ticket describes my experience. If DNF stopped doing it, the then j-mracek's proposal is fine.

@j-mracek j-mracek self-assigned this Sep 7, 2023
@m-blaha m-blaha force-pushed the mblaha/upgrade-versions branch from 3423249 to bbcab54 Compare September 7, 2023 15:43
/// to the installed ones. Similar to filter_upgrades(), but retains also
/// installed packages and packages whose names are not installed.
static void remove_older_versions(libdnf5::rpm::PackageQuery & query) {
libdnf5::rpm::PackageQuery installed(query.get_base());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to initialize query without excluded packages

libdnf5::rpm::PackageQuery installed(query.get_base(), rpm::PackageQuery::ExcludeFlags::IGNORE_EXCLUDES)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, fixed.

/// installed packages and packages whose names are not installed.
static void remove_older_versions(libdnf5::rpm::PackageQuery & query) {
libdnf5::rpm::PackageQuery installed(query.get_base());
installed.filter_installed();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very weak suggestion because it does not change the result.
What about to add additional filter - latest()? Installonly packages will be handled in the next query only once.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added. (although I did not measure impact on performance here).

@m-blaha m-blaha force-pushed the mblaha/upgrade-versions branch 2 times, most recently from 05935ea to c39f7fc Compare September 8, 2023 06:45
j-mracek
j-mracek previously approved these changes Sep 8, 2023
@m-blaha
Copy link
Member Author

m-blaha commented Sep 8, 2023

Looks like some tests with repo priorities started failing. I'll investigate.

@j-mracek
Copy link
Contributor

j-mracek commented Sep 8, 2023

We have a problem, if we do this we will have a problem with repo priority test cases. Basically we have two options, we can keep all packages from proiority repo and then apply removal of old packages (logic from check update)

@m-blaha m-blaha marked this pull request as draft September 8, 2023 08:56
For UPGRADE* actions, do not add packages that are older than the
currently installed versions to the libsolv job. This decreases the
number of potential solutions and helps resolve some edge cases.

See https://issues.redhat.com/browse/RHEL-1448 for one of those edge
cases.
The `minimal` parameter makes sense only for upgrade* actions, and
filtering of the earliest EVR needs to be performed after the versions
lower than the installed ones have been removed.
@m-blaha m-blaha force-pushed the mblaha/upgrade-versions branch from c39f7fc to a87ab64 Compare September 8, 2023 09:18
@m-blaha
Copy link
Member Author

m-blaha commented Sep 8, 2023

We have a problem, if we do this we will have a problem with repo priority test cases. Basically we have two options, we can keep all packages from proiority repo and then apply removal of old packages (logic from check update)

I've added code to pre-select upgrade candidates considering also repository priorities. However, I'm starting to feel that we might be exceeding our responsibilities by intervening in matters that should be handled by libsolv. To be honest, I'm leaning towards closing this PR and simply documenting in the upgrade code why including all versions of the package in the libsolv job is important.

@j-mracek j-mracek dismissed their stale review September 8, 2023 09:34

Discovered a problem

@ppisar
Copy link
Contributor

ppisar commented Sep 8, 2023

I agree we are stepping on libsolve's toes. Maybe --best can provide a similar user experience.

@m-blaha
Copy link
Member Author

m-blaha commented Sep 8, 2023

@ppisar unfortunately, the --best does not help here and the result remains the same.
Unversioned obsoletes are obstacle, that's hard to get around. I don't see other solution than rename the package and start again from scratch.

@m-blaha m-blaha closed this Nov 3, 2023
@m-blaha m-blaha deleted the mblaha/upgrade-versions branch November 8, 2023 12:14
@m-blaha m-blaha restored the mblaha/upgrade-versions branch November 8, 2023 12:14
@m-blaha m-blaha deleted the mblaha/upgrade-versions branch February 19, 2024 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants