Skip to content

Commit

Permalink
export_buildroot_image: new plugin for OCI image exports
Browse files Browse the repository at this point in the history
The --enable-plugin=export_buildroot_image option automatically wraps
the <chroot>/root directory as OCI-image-as-tarball file, and stores it
into Mock's resultdir.

We exclude the /builddir/build directory from the generated image; it
could contain RPMs, SRPMs, extracted sources, etc.

Relates: rpm-software-management#1482
  • Loading branch information
praiskup committed Dec 12, 2024
1 parent 061d3b6 commit cb90110
Show file tree
Hide file tree
Showing 6 changed files with 133 additions and 1 deletion.
62 changes: 62 additions & 0 deletions docs/Plugin-Export-Buildroot-Image.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
layout: default
title: Plugin export_buildroot_image
---

This plugin allows you to (on demand) export the Mock chroot as an OCI image in
local archive format (tarball). This tarball can provide additional convenience
for local build reproducibility. See the example below for details.

By default, this plugin is **disabled**. You can enable it using the
`--enable-plugin export_buildroot_image` option in `--rebuild` mode.

This plugin has been added in Mock v6.0.

## Example use-case

First, let's start a standard Mock build, but enable the OCI archive generator:

$ mock -r fedora-rawhide-x86_64 --enable-plugin export_buildroot_image \
/tmp/quick-package/dummy-pkg-20241212_1114-1.src.rpm
... mock installs all build-deps, and does other chroot tweaks ...
Start: producing buildroot as OCI image
... mock performs the rpmbuild ...
INFO: Results and/or logs in: /var/lib/mock/fedora-rawhide-x86_64/result
Finish: run

The archive has been saved in the result directory:

$ ls /var/lib/mock/fedora-rawhide-x86_64/result/*.tar
/var/lib/mock/fedora-rawhide-x86_64/result/buildroot-oci.tar

Then, you can try re-running the build without Mock, like this:

$ chmod a+r /tmp/quick-package/dummy-pkg-20241212_1114-1.src.rpm
$ podman run --rm -ti \
-v /tmp/quick-package/dummy-pkg-20241212_1114-1.src.rpm:/dummy-pkg.src.rpm:z \
oci-archive:/var/lib/mock/fedora-rawhide-x86_64/result/buildroot-oci.tar \
rpmbuild --rebuild /dummy-pkg.src.rpm

Installing /dummy-pkg.src.rpm
setting SOURCE_DATE_EPOCH=1401926400
Executing(%mkbuilddir): /bin/sh -e /var/tmp/rpm-tmp.XIm441
...
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.pqJ9hu
...
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.iaeMZG
...
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.SHktaE
...
Processing files: dummy-pkg-20241212_1114-1.fc42.x86_64
...
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.E71FWH
...
+ exit 0

**Warning:** This method of reproducing a Mock build is not recommended for
production use. During a normal/full Mock rebuild, Mock ensures the buildroot
is fully up-to-date. Using just plain `rpmbuild` within Podman may result in
outdated files, different structure in the kernel-driven filesystems like
`/proc`, `/dev`, and `/sys`, different SELinux assumptions, permissions, etc.
Proceed with caution, and be prepared to encounter some differences (and perhaps
different build failures).
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ See a [separate document](Mock-Core-Configs).
* [ccache](Plugin-CCache) - compiler cache plugin
* [chroot_scan](Plugin-ChrootScan) - allows you to retrieve build artifacts from buildroot (e.g. additional logs, coredumps)
* [compress_logs](Plugin-CompressLogs) - compress logs
* [export_buildroot_image](Plugin-Export-Buildroot-Image) - export prepared build chroot as an OCI image
* [hw_info](Plugin-HwInfo) - prints HW information of builder
* [lvm_root](Plugin-LvmRoot) - caching buildroots using LVM
* [mount](Plugin-Mount) - allows you to mount directories into chroot
Expand Down
1 change: 1 addition & 0 deletions mock/mock.spec
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ Recommends: dnf-utils
Recommends: btrfs-progs
Suggests: qemu-user-static
Suggests: procenv
Recommends: buildah
Recommends: podman
Recommends: fuse-overlayfs

Expand Down
4 changes: 3 additions & 1 deletion mock/py/mockbuild/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
'ccache', 'selinux', 'package_state', 'chroot_scan',
'lvm_root', 'compress_logs', 'sign', 'pm_request',
'hw_info', 'procenv', 'showrc', 'rpkg_preprocessor',
'rpmautospec', 'buildroot_lock']
'rpmautospec', 'buildroot_lock', 'export_buildroot_image']

def nspawn_supported():
"""Detect some situations where the systemd-nspawn chroot code won't work"""
Expand Down Expand Up @@ -234,6 +234,8 @@ def setup_default_config_opts():
'process-distgit',
]
},
'export_buildroot_image_enable': False,
'export_buildroot_image_opts': {},
}

config_opts['environment'] = {
Expand Down
63 changes: 63 additions & 0 deletions mock/py/mockbuild/plugins/export_buildroot_image.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
"""
Generate OCI from prepared build chroot.
Use given OCI image as build chroot (TODO).
"""

import os
import mockbuild.util
from mockbuild.trace_decorator import getLog

requires_api_version = "1.1"


def init(plugins, conf, buildroot):
""" The obligatory plugin entry point """
OCIAsBuildroot(plugins, conf, buildroot)


class OCIAsBuildroot:
"""
OCIAsBuildroot plugin (class).
"""
def __init__(self, plugins, conf, buildroot):
self.buildroot = buildroot
self.state = buildroot.state
self.conf = conf
plugins.add_hook("postdeps", self.produce_buildroot_image)

def do(self, cmd):
""" Execute command on host (as root) """
getLog().info("Executing %s", ' '.join(cmd))
mockbuild.util.do(cmd, returnOutput=True, returnStderr=True)

def _produce_image_as_root(self):
name = f"mock-container-{self.buildroot.config['root']}"
tarball = os.path.join(self.buildroot.resultdir, "buildroot-oci.tar")
chroot = self.buildroot.make_chroot_path()

# Add the whole chroot directory into the container
self.do(["buildah", "from", "--name", name, "scratch"])
self.do(["buildah", "add", name, chroot, "/"])

# Keep just /builddir directory, make it correctly owned
self.do(["buildah", "run", name, "rm", "-r",
self.buildroot.config["chroothome"] + "/build"])
self.do(["buildah", "run", name, "chown", "-R", "mockbuild:mock",
self.buildroot.config["chroothome"]])

# When starting container, switch to mockbuild user directly
self.do(["buildah", "config", "--user", "mockbuild:mock", name])

# Export the image as OCI archive, and remove the WIP container
self.do(["buildah", "commit", "--format", "oci", name,
"oci-archive:" + tarball])
self.do(["buildah", "rm", name])

def produce_buildroot_image(self):
""" Generate OCI image from buildroot using Buildah """
try:
self.state.start("producing buildroot as OCI image")
with self.buildroot.uid_manager.elevated_privileges():
self._produce_image_as_root()
finally:
self.state.finish("producing buildroot as OCI image")
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
A new plugin, `export_buildroot_image`, has been added. This plugin can export
the Mock chroot as an OCI archive once all the build dependencies have been
installed (when the chroot is ready-made for runnign `/bin/rpmbuild -bb`).

0 comments on commit cb90110

Please sign in to comment.