Skip to content

Commit

Permalink
hermetic: rename the feature from "isolated → hermetic"
Browse files Browse the repository at this point in the history
This has been discussed within the Mock/Copr and Konflux teams, and the
new term seems to better reflect what Mock actually does.  It also
avoids confusion with the "rpmbuild isolation" that Mock has always
implemented.
  • Loading branch information
praiskup committed Sep 19, 2024
1 parent be4034c commit 39c06b1
Show file tree
Hide file tree
Showing 16 changed files with 146 additions and 126 deletions.
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
Feature: Mock 5.7+ supports isolated builds
Feature: Mock 5.7+ supports hermetic builds

@isolated_build
Scenario: Isolated build against a DNF5 distribution
@hermetic_build
Scenario: Hermetic build against a DNF5 distribution
Given an unique mock namespace
When deps for python-copr-999-1.src.rpm are calculated against fedora-rawhide-x86_64
And a local repository is created from lockfile
And an isolated build is retriggered with the lockfile and repository
And a hermetic build is retriggered with the lockfile and repository
Then the build succeeds
And the produced lockfile is validated properly

@isolated_build
Scenario: Isolated build against a DNF4 distribution
@hermetic_build
Scenario: Hermetic build against a DNF4 distribution
Given an unique mock namespace
# Temporary image, until we resolve https://issues.redhat.com/browse/CS-2506
And next mock call uses --config-opts=bootstrap_image=quay.io/mock/behave-testing-c9s-bootstrap option
And next mock call uses --config-opts=bootstrap_image_ready=True option
When deps for mock-test-bump-version-1-0.src.rpm are calculated against centos-stream+epel-9-x86_64
And a local repository is created from lockfile
And an isolated build is retriggered with the lockfile and repository
And a hermetic build is retriggered with the lockfile and repository
Then the build succeeds
And the produced lockfile is validated properly
6 changes: 3 additions & 3 deletions behave/features/steps/other.py
Original file line number Diff line number Diff line change
Expand Up @@ -206,14 +206,14 @@ def step_impl(context):
lockfile = mock_run["lockfile"]

context.local_repo = tempfile.mkdtemp(prefix="mock-tests-local-repo-")
cmd = ["mock-isolated-repo", "--lockfile", lockfile, "--output-repo",
cmd = ["mock-hermetic-repo", "--lockfile", lockfile, "--output-repo",
context.local_repo]
assert_that(run(cmd)[0], equal_to(0))


@when('an isolated build is retriggered with the lockfile and repository')
@when('a hermetic build is retriggered with the lockfile and repository')
def step_impl(context):
context.mock.isolated_build()
context.mock.hermetic_build()


@then('the produced lockfile is validated properly')
Expand Down
10 changes: 5 additions & 5 deletions behave/testlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,22 +123,22 @@ def calculate_deps(self, srpm, chroot):
"lockfile": os.path.join(self.resultdir, "buildroot_lock.json")
})

def isolated_build(self):
def hermetic_build(self):
"""
From the previous calculate_deps() run, perform isolated build
From the previous calculate_deps() run, perform hermetic build
"""
mock_calc = self.context.mock_runs["calculate-build-deps"][-1]
out, err = run_check(self.basecmd + [
"--isolated-build", mock_calc["lockfile"], self.context.local_repo,
"--hermetic-build", mock_calc["lockfile"], self.context.local_repo,
mock_calc["srpm"]
])
self.context.mock_runs["rebuild"].append({
"status": 0,
"out": out,
"err": err,
})
# We built into an isolated-build.cfg!
self.context.chroot = "isolated-build"
# We built into a hermetic-build.cfg!
self.context.chroot = "hermetic-build"

def clean(self):
""" Clean chroot, but keep dnf/yum caches """
Expand Down
12 changes: 10 additions & 2 deletions docs/Plugin-BuildrootLock.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This plugin generates an additional build artifact—the buildroot *lockfile*
The *lockfile* describes both the list of buildroot sources (e.g., a list of
installed RPMs, bootstrap image info, etc.) and a set of Mock configuration
options. Using this information, Mock can later reproduce the buildroot
preparation (see the [Isolated Builds feature page](feature-isolated-builds)).
preparation (see the [Hermetic Builds feature page](feature-hermetic-builds)).

This plugin is **disabled** by default but is automatically enabled with the
`--calculate-build-dependencies` option. You can enable it (for all builds) by
Expand All @@ -36,4 +36,12 @@ installed together with the Mock RPM package:

Currently, we do not provide a compatibility promise. Only the exact same
version of Mock that produced the file is guaranteed to read and process it.
For more information, see [Isolated Builds](feature-isolated-builds).
For more information, see [Hermetic Builds](feature-hermetic-builds).

Also, in the future we plan to switch to a standardized tooling so we operate
with a standardized format, too. For more info see the [DNF5 feature
request][discussion], [rpm-lockfile-prototype][] and [libpkgmanifest][].

[discussion]: https://github.com/rpm-software-management/dnf5/issues/833
[rpm-lockfile-prototype]: https://github.com/konflux-ci/rpm-lockfile-prototype
[libpkgmanifest]: https://github.com/rpm-software-management/libpkgmanifest
132 changes: 72 additions & 60 deletions docs/feature-isolated-builds.md → docs/feature-hermetic-builds.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
---
layout: default
title: Isolated builds with Mock
title: Hermetic builds with Mock
---

Isolated builds with Mock
Hermetic builds with Mock
=========================

Mock (v5.7+) supports isolated RPM builds, sometimes referred to as "hermetic"
Mock (v5.7+) supports hermetic RPM builds, sometimes referred to as "isolated"
or "offline" builds. For more details, see the
[SLSA "isolated" definition][SLSA].
[SLSA "hermetic" definition][SLSA future].

Quick start
-----------

For the impatient, the TL;DR steps of the HOWTO are as follows:
For the impatient, the TL;DR steps are as follows:

# we want to build this package
srpm=your-package.src.rpm
Expand All @@ -28,29 +28,32 @@ For the impatient, the TL;DR steps of the HOWTO are as follows:
lockfile=/var/lib/mock/fedora-rawhide-x86_64/result/buildroot_lock.json

# create a local RPM repository (+ download bootstrap image)
mock-isolated-repo --lockfile "$lockfile" --output-repo "$repo"
mock-hermetic-repo --lockfile "$lockfile" --output-repo "$repo"

# perform the isolated build!
mock --isolated-build "$lockfile" "$repo" "$srpm"
# perform the hermetic build!
mock --hermetic-build "$lockfile" "$repo" "$srpm"

What an "isolated build" is..
What an "hermetic build" is..
-----------------------------

The term "isolated build" is often used in different contexts, even within
Mock's terminology. Historically, when we said that "Mock isolates the build,"
The term "isolated build" is often used in different contexts within
Mock's terminology. Historically, when we said that "Mock isolates the build,"
we typically meant that Mock creates a *buildroot* (also referred to as a *build
directory* or *build chroot*) and runs the (Turing-complete, and thus
potentially insecure) *RPM build* process (i.e., a call to `/usr/bin/rpmbuild`)
inside it. In this sense, Mock "isolates" the RPM build process from the rest
of the system, or protects the system from potential mishaps. However, the
**buildroot preparation** process was never "isolated" in this manner—only the
*RPM build* was. Even the *RPM build* "isolation" was always performed on a
*RPM build* was. Also, the *RPM build* "isolation" was always performed on a
best-effort basis. For more details, see [Mock's Scope](index).

When we now talk about making builds and the corresponding built artifacts
safer, more predictable, and more reproducible, we refer to the [SLSA
isolation][SLSA] definition. This involves using Mock in an *isolated*
environment, free from unintended external influence.
This document focuses on making builds and their corresponding artifacts safer,
more predictable, and more reproducible. When we refer to *isolation*, we are
specifically referencing the [SLSA platform isolation][SLSA]. SLSA outlines
various security levels, and for the future, it introduces the concept of
[*hermetic builds*][SLSA future]. This is where Mock steps in, enabling builds
to be performed in a *hermetic* environment, free from unintended external
influences.

Mock itself doesn't aim to provide this level of *isolation*. Mock is still
just a tool that runs in "some" build environment to perform the `SRPM → RPM`
Expand All @@ -60,40 +63,42 @@ may be (un)intentionally altered. Therefore, the preparation of the environment
to **run Mock** and the **isolation** itself is the responsibility of a
different tool (for example, `podman run --privileged --network=none`).

So, what does Mock `--isolated-build` do if it doesn't isolate?
Essentially, it just does less work than it usually does! It optimizes out any
action (primarily during the *buildroot* preparation) that would rely on
"external" factors—specifically, it never expects Internet connectivity.
So, what does Mock `--hermetic-build` do if it doesn't isolate? Essentially, it
just does less work than it usually does! It optimizes out any action
(primarily during the *buildroot* preparation) that would rely on "external"
factors—specifically, it never expects Internet connectivity.
However, for the eventual build to succeed, **something else** still needs to
perform these omitted actions. Every single component required for *buildroot*
preparation must be prepared in advance for the `mock --isolated-build`
call (within **the** properly *isolated* environment, of course).
perform these omitted actions. Every single component/artifact required for
*buildroot* preparation must be prepared in advance for the `mock
--hermetic-build` call (within the properly *isolated* or *hermetic*
environment, of course).


Challenges
----------

You’ve probably noticed that what used to be a simple command—like
`mock -r "$chroot" "$srpm"`—has now become a more complicated set of commands.

This complexity arises because the *buildroot* in Mock is always prepared by
installing a set of RPMs (Mock calls DNF, DNF calls RPM, ...), which normally
requires a network connection.

Additionally, it’s not always guaranteed that the DNF/RPM variant on the build
host (e.g., an EPEL 8 host) is sufficient or up-to-date for building the target
distribution (e.g., the newest Fedora Rawhide). Therefore, we need network
access [to obtain the appropriate bootstrap tooling](Feature-bootstrap).

The [dynamic build dependencies][] further complicate the process. Without
them, we could at least make the `/bin/rpmbuild` fully offline—but with them,
it’s not so simple. Mock needs to interrupt the ongoing *RPM build* process,
resolve additional `%generate_buildrequires` (installing more packages on
demand), restart the *RPM build*, interrupt it again, and so on. This process
also requires a network connection!
host is sufficient or up-to-date for building the target distribution (e.g.,
building the newest *Fedora Rawhide* packages on *EPEL 8* host). Therefore, we
need network access [to obtain the appropriate bootstrap
tooling](Feature-bootstrap).

[Dynamic build dependencies][] add further complexity to the process. Without
them, we could potentially make the `/bin/rpmbuild` process fully offline—but
with their inclusion, it becomes much more challenging. Mock must interrupt the
ongoing *RPM build* process, resolve additional `%generate_buildrequires`
(installing more packages on demand), restart the *RPM build*, and potentially
repeat this cycle. This process also requires an (intermittent) network
connection!

All of this is further complicated by the goal of making the *buildroot* as
*minimal* as possible—the fewer packages installed, the better. We can’t even
*minimal* as possible—the fewer packages installed, the better. We can’t even
afford to install DNF into the buildroot, and as you’ve probably realized, we
definitely don’t want to blindly install all available RPMs.

Expand All @@ -102,7 +107,7 @@ The solution
------------

To address the challenges, we needed to separate the online
(`--calculate-build-dependencies`) and offline (`--isolated-build`) tasks
(`--calculate-build-dependencies`) and offline (`--hermetic-build`) tasks
that Mock performs.

1. **Online Tasks:** These need to be executed first. We let Mock prepare the
Expand All @@ -113,34 +118,35 @@ that Mock performs.
The format of lockfile is defined by provided JSON Schema file(s), see
documentation for the [buildroot_lock plugin](Plugin-BuildrootLock).

**Note:** The *buildroot* preparation also includes the installation of
dynamic build dependencies! Therefore, we **have to start an RPM build**.
Although we don’t finish the build (we terminate it once the
`%generate_buildrequires` is resolved, before reaching the `%build` phase,
etc.), it must be initiated.
**Note:** The *buildroot* preparation includes the installation of dynamic
build dependencies! That's why we have to **initiate** `rpmbuild`.
But we don’t **finish** the buildwe terminate it once the
`%generate_buildrequires` section is resolved, before reaching the `%build`
phase.

2. **Offline Repository Creation:** With the *lockfile* from the previous step,
we can easily retrieve the referenced components from the network. The Mock
project provides an example implementation for this step in the
`mock-isolated-repo(1)` utility. This tool downloads all the referenced
`mock-hermetic-repo(1)` utility. This tool downloads all the referenced
components from the internet and places them into a single local
directory—let's call it an *offline repository*.

**Note:** This step doesn’t necessarily have to be done by the Mock project
itself. The *lockfile* is concise enough for further processing and
itself. The *lockfile* is concise enough for further processing and
validation (e.g., ensuring the set of RPMs and the buildroot image come from
trusted sources) and could be parsed by build-system-specific tools like
[cachi2][] (potentially in the future).

3. **Offline Build:** With the *srpm* and the *offline repository*, we can
instruct Mock to restart the build using the `--isolated-build
LOCKFILE OFFLINE_REPO SRPM` command. The *lockfile* is still needed at this
instruct Mock to restart the build using the `--hermetic-build
LOCKFILE OFFLINE_REPO SRPM` command. The *lockfile* is still needed at this
stage because it contains some of the configuration options used in step 1
that must be inherited by the current Mock call.

This step creates a new *buildroot #2* using the pre-downloaded RPMs in the
*offline repository* (installing them all at once) and then (re)starts the
RPM build process.
RPM build process. This `rpmbuild` run **finishes** though, and provides the
binary RPM artifacts as usually.

You might notice that some steps are performed twice, specifically downloading
the RPMs (steps 1 and 2) and running the RPM build (steps 1 and 3). This
Expand All @@ -155,16 +161,17 @@ Also, while you can calmly experiment with
mock --calculate-build-dependencies -r fedora-rawhide-x86_64 "$srpm"
mock --no-clean -r fedora-rawhide-x86_64 "$srpm"

and it is very close to the TL;DR variant, such an approach is not the same
thing! The *buildroot #1* **was not** prepared by Mock in **isolated**
environment.
This approach might seem similar to the TL;DR version, but it's not the same!
There is no *buildroot #1* and *buildroot #2*, only one buildroot. And that one
was prepared while Mock was online, meaning that something could **have
influenced** the environment preparation, and the subsequent **build**.

Limitations
-----------

- Let us stress out that this feature itself, while related or at least a bit
helpful, doesn't provide reproducible builds. For reproducible builds, build
systems need to take in account state of host machine, the full
helpful for, doesn't provide reproducible builds. For reproducible builds,
build systems need to take in account state of host machine, the full
software/hardware stack. There's still a big influence of external factors!

- We rely heavily on
Expand All @@ -176,24 +183,29 @@ Limitations
the implementation, as we don't need to recall the set of commands (or list of
packages to install into) needed for bootstrap preparation.

- It is known fact that *normal builds* and *isolated builds* may result in
- It is known fact that *normal builds* and *hermetic builds* may result in
slightly different outputs (at least in theory). This issue relates to the
topic of *reproducible builds*. Normally, the *buildroot* is installed using
several DNF commands (RPM transactions), whereas the *isolated* build installs
all dependencies in a single DNF command (RPM transaction). While this
difference might cause the outputs of *normal* and *isolated* builds to vary
(in theory, because the chroot depends on RPM installation order), it OTOH
introduces more determinism!
several DNF commands (RPM transactions), whereas the *hermetic build* installs
all dependencies in a single DNF command (single RPM transaction). While this
difference might cause the outputs of *normal* and *hermetic* builds to vary
(in theory, because the chroot shape depends on the complex RPM installation
order), the *hermetic* variant introduces more determinism!

- The *lockfile* provides a list of the required RPMs, referenced by URLs.
These URLs point to the corresponding RPM repositories (online) from which
they were installed in step 1. However, in many cases, RPMs are downloaded
from `metalink://` or `mirrorlist://` repositories, meaning the URL might be
selected non-deterministically, and the specific mirrors chosen could be
rather ephemeral. For this reason, users should—for isolated builds—avoid
using mirrored repositories (as in the case of Koji builders) or avoid making
large delays between step 1 and step 2.
rather ephemeral. For this reason, users should—for *hermetic* builds, for
now—avoid using mirrored repositories (and prefer Koji buildroots only) and
avoid making large delays between step 1 and step 2. Especially that, at the
time of writing this document, we know about [two][bug1] [bugs][bug2] that
will complicate the *lockfile* generation.

[SLSA]: https://slsa.dev/spec/v1.0/requirements
[SLSA future]: https://slsa.dev/spec/v1.0/future-directions
[dynamic build dependencies]: https://github.com/rpm-software-management/mock/issues/1359
[cachi2]: https://github.com/containerbuildsystem/cachi2
[bug1]: https://github.com/rpm-software-management/dnf/issues/2130
[bug2]: https://github.com/rpm-software-management/dnf5/issues/1673
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ Every plugin has a corresponding wiki page with docs.
* [package managers](Feature-package-managers) - supported package managers
* [rhel chroots](Feature-rhelchroots) - builds for RHEL
* [GPG keys and SSL](feature-gpg-and-ssl) - how to get your GPG keys and SSL certificates to buildroot
* [Isolated (offline) Builds](feature-isolated-builds) - doing offline builds with Mock
* [Hermetic (offline) Builds](feature-hermetic-builds) - doing hermetic builds with Mock

## Using Mock outside your git sandbox

Expand Down
Loading

0 comments on commit 39c06b1

Please sign in to comment.