AARCH64 images for RHEL family

[noama-nv]

Hi,

im struggling building/adding support for AARCH64 images for RHEL family
I have access to ARM server and RHEL cloud/iso images

looking for some guidance/tips

[noama-nv]

rhel9.pkr.hcl

packer {
  required_version = ">= 1.7.0"
  required_plugins {
    qemu = {
      version = "~> 1.0"
      source  = "github.com/hashicorp/qemu"
    }
  }
}

variable "filename" {
  type        = string
  default     = "rhel9.tar.gz"
  description = "The filename of the tarball to produce"
}

variable "rhel9_iso_path" {
  type    = string
  default = "${env("RHEL9_ISO_PATH")}"
}

# Use --baseurl to specify the exact url for AppStream repo
variable "ks_appstream_repos" {
  type    = string
  default = "--baseurl='file:///run/install/repo/AppStream'"
}

variable ks_proxy {
  type    = string
  default = "${env("KS_PROXY")}"
}

variable "timeout" {
  type        = string
  default     = "1h"
  description = "Timeout for building the image"
}

locals {
  ks_proxy = var.ks_proxy != "" ? "--proxy=${var.ks_proxy}" : ""
}

source "qemu" "rhel9" {
  boot_command     = ["<up><tab> ", "inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/rhel9.ks ", "console=ttyS0 inst.cmdline", "<enter>"]
  boot_wait        = "3s"
  communicator     = "none"
  disk_size        = "4G"
  headless         = true
  iso_checksum     = "none"
  iso_url          = var.rhel9_iso_path
  memory           = 2048
  qemu_binary    = "qemu-system-aarch64"
  qemuargs         = [
    ["-serial", "stdio"],
    ["-cpu", "host"],
    ["-machine", "type=virt,accel=kvm"]
  ]
  shutdown_timeout = var.timeout
  http_content = {
    "/rhel9.ks" = templatefile("${path.root}/http/rhel9.ks.pkrtpl.hcl",
      {
        KS_PROXY           = local.ks_proxy,
        KS_APPSTREAM_REPOS = var.ks_appstream_repos,
      }
    )
  }

}

build {
  sources = ["source.qemu.rhel9"]

  post-processor "shell-local" {
    inline = [
      "SOURCE=${source.name}",
      "OUTPUT=${var.filename}",
      "source ../scripts/fuse-nbd",
      "source ../scripts/fuse-tar-root",
      "rm -rf output-${source.name}",
    ]
    inline_shebang = "/bin/bash -e"
  }
}

PACKER_LOG=1 make ISO=/root/snap/firefox/common/Downloads/rhel-9.4-aarch64-dvd.iso

==> qemu.rhel9: Overriding default Qemu arguments with qemuargs template option...
2024/09/24 10:53:38 packer-plugin-qemu_v1.1.0_x5.0_linux_arm64 plugin: 2024/09/24 10:53:38 Executing /usr/bin/qemu-system-aarch64: []string{"-serial", "stdio", "-vnc", "127.0.0.1:77", "-smp", "1", "-device", "virtio-net,netdev=user.0", "-m", "2048M", "-cpu", "host", "-machine", "type=virt,accel=kvm", "-drive", "file=output-rhel9/packer-rhel9,if=virtio,cache=writeback,discard=ignore,format=qcow2", "-drive", "file=/root/snap/firefox/common/Downloads/rhel-9.4-aarch64-dvd.iso,media=cdrom", "-boot", "once=d", "-name", "packer-rhel9", "-netdev", "user,id=user.0"}
2024/09/24 10:53:38 packer-plugin-qemu_v1.1.0_x5.0_linux_arm64 plugin: 2024/09/24 10:53:38 Started Qemu. Pid: 1097596
2024/09/24 10:53:38 packer-plugin-qemu_v1.1.0_x5.0_linux_arm64 plugin: 2024/09/24 10:53:38 Qemu stderr: qemu-system-aarch64: no function defined to set boot device list for this architecture
2024/09/24 10:53:38 packer-plugin-qemu_v1.1.0_x5.0_linux_arm64 plugin: 2024/09/24 10:53:38 failed to unlock port lockfile: close tcp 127.0.0.1:5977: use of closed network connection

[noama-nv]

Manage to build the image with lots of modifications but post processing is now failing

packer {
  required_version = ">= 1.7.0"
  required_plugins {
    qemu = {
      version = "~> 1.0"
      source  = "github.com/hashicorp/qemu"
    }
  }
}

variable "filename" {
  type        = string
  default     = "rhel9.tar.gz"
  description = "The filename of the tarball to produce"
}

variable "rhel9_iso_path" {
  type    = string
  default = "${env("RHEL9_ISO_PATH")}"
}

# Use --baseurl to specify the exact url for AppStream repo
variable "ks_appstream_repos" {
  type    = string
  default = "--baseurl='file:///run/install/repo/AppStream'"
}

variable ks_proxy {
  type    = string
  default = "${env("KS_PROXY")}"
}

variable "timeout" {
  type        = string
  default     = "1h"
  description = "Timeout for building the image"
}

locals {
  ks_proxy = var.ks_proxy != "" ? "--proxy=${var.ks_proxy}" : ""
}

source "qemu" "rhel9" {
  iso_checksum     = "none"
  iso_url          = var.rhel9_iso_path
  #shutdown_command   = "/sbin/shutdown -hP now"
  accelerator        = "kvm"
  #ssh_username       = "root"
  #ssh_password       = "password"
  #ssh_timeout        = "3600s"
  communicator     = "none"
  cpus               = 2
  firmware           = "/usr/share/AAVMF/AAVMF_CODE.fd"
  use_pflash         = false
  disk_interface     = "virtio-scsi"
  disk_size          = "10G"
  disk_cache         = "unsafe"
  disk_discard       = "unmap"
  disk_detect_zeroes = "unmap"
  disk_compression   = true
  format             = "qcow2"
  headless           = false
  machine_type       = "virt,gic-version=max"
  memory             = 2048
  net_device         = "virtio-net"
  qemu_binary        = "qemu-system-aarch64"
  #vm_name            = "rhel9.qcow2"
  boot_wait          = "10s"
  boot_command       = [
    "c<wait>",
    "linux /images/pxeboot/vmlinuz",
    " inst.stage2=hd:LABEL=RHEL-9-4-0-BaseOS-aarch64 ro",
    " inst.text biosdevname=0 net.ifnames=0",
    " inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/rhel9.ks",
    "<enter>",
    "initrd /images/pxeboot/initrd.img<enter>",
    "boot<enter><wait>"
  ]
  qemuargs = [
    ["-cpu", "max"],
    ["-boot", "strict=on"],
    ["-monitor", "none"],
    ["-usb"],
  ]
  http_content = {
    "/rhel9.ks" = templatefile("${path.root}/http/rhel9_aa64.ks.pkrtpl.hcl",
      {
        KS_PROXY           = local.ks_proxy,
        KS_APPSTREAM_REPOS = var.ks_appstream_repos,
      }
    )
  }
}

build {
  sources = ["source.qemu.rhel9"]

  post-processor "shell-local" {
    inline = [
      "SOURCE=${source.name}",
      "OUTPUT=${var.filename}",
      "source ../scripts/fuse-nbd",
      "source ../scripts/fuse-tar-root",
      "rm -rf output-${source.name}",
    ]
    inline_shebang = "/bin/bash -e"
  }
}

KS script

cdrom
poweroff
firewall --enabled --service=ssh
firstboot --disable
#ignoredisk --only-use=sda
lang en_US.UTF-8
keyboard us
network --device eth0 --bootproto=dhcp
firewall --enabled --service=ssh
selinux --enforcing
timezone UTC --isUtc
#bootloader --location=mbr --driveorder="sda" --timeout=1
bootloader --location=mbr
rootpw --plaintext password

repo --name="AppStream" ${KS_APPSTREAM_REPOS} ${KS_PROXY}

zerombr
clearpart --all --initlabel
#part / --size=1 --grow --asprimary --fstype=ext4
part /boot/efi --size=200 --fstype=efi
part /boot --size=512 --fstype=xfs
part / --size=8000 --fstype=xfs

%post --erroronfail
# workaround anaconda requirements and clear root password
passwd -d root
passwd -l root

# Clean up install config not applicable to deployed environments.
for f in resolv.conf fstab; do
    rm -f /etc/$f
    touch /etc/$f
    chown root:root /etc/$f
    chmod 644 /etc/$f
done

rm -f /etc/sysconfig/network-scripts/ifcfg-[^lo]*

# Kickstart copies install boot options. Serial is turned on for logging with
# Packer which disables console output. Disable it so console output is shown
# during deployments
sed -i 's/^GRUB_TERMINAL=.*/GRUB_TERMINAL_OUTPUT="console"/g' /etc/default/grub
sed -i '/GRUB_SERIAL_COMMAND="serial"/d' /etc/default/grub
sed -ri 's/(GRUB_CMDLINE_LINUX=".*)\s+console=ttyS0(.*")/\1\2/' /etc/default/grub
sed -i 's/GRUB_ENABLE_BLSCFG=.*/GRUB_ENABLE_BLSCFG=false/g' /etc/default/grub

dnf clean all
%end

%packages
@core
bash-completion
cloud-init
# cloud-init only requires python3-oauthlib with MAAS. As such upstream
# removed this dependency.
python3-oauthlib
rsync
tar
# grub2-efi-x64 ships grub signed for UEFI secure boot. If grub2-efi-x64-modules
# is installed grub will be generated on deployment and unsigned which breaks
# UEFI secure boot.
grub2-efi-aa64
efibootmgr
shim-aa64
dosfstools
lvm2
mdadm
device-mapper-multipath
iscsi-initiator-utils
-plymouth
# Remove Intel wireless firmware
-i*-firmware
%end

fail log

==> qemu.rhel9: Waiting for shutdown...
2024/09/24 17:02:24 packer-plugin-qemu_v1.1.0_x5.0_linux_arm64 plugin: 2024/09/24 17:02:24 VM shut down.
2024/09/24 17:02:24 packer-plugin-qemu_v1.1.0_x5.0_linux_arm64 plugin: 2024/09/24 17:02:24 Executing qemu-img: []string{"convert", "-c", "-O", "qcow2", "output-rhel9/packer-rhel9", "output-rhel9/packer-rhel9.convert"}
==> qemu.rhel9: Converting hard drive...
2024/09/24 17:03:46 packer-plugin-qemu_v1.1.0_x5.0_linux_arm64 plugin: 2024/09/24 17:03:46 stdout:
2024/09/24 17:03:46 packer-plugin-qemu_v1.1.0_x5.0_linux_arm64 plugin: 2024/09/24 17:03:46 stderr:
2024/09/24 17:03:46 packer-plugin-qemu_v1.1.0_x5.0_linux_arm64 plugin: 2024/09/24 17:03:46 failed to unlock port lockfile: close tcp 127.0.0.1:5929: use of closed network connection
2024/09/24 17:03:46 [INFO] (telemetry) ending qemu.rhel9
2024/09/24 17:03:46 [INFO] (telemetry) Starting post-processor shell-local
==> qemu.rhel9: Running post-processor:  (type shell-local)
2024/09/24 17:03:46 packer-post-processor-shell-local plugin: [INFO] (shell-local): Prepending inline script with #!/bin/bash -e
2024/09/24 17:03:46 packer-post-processor-shell-local plugin: [INFO] (shell-local): starting local command: /bin/sh -c PACKER_BUILDER_TYPE='qemu' PACKER_BUILD_NAME='rhel9' PACKER_HTTP_ADDR='10.0.2.2:8668' PACKER_HTTP_IP='10.0.2.2' PACKER_HTTP_PORT='8668'  /tmp/packer-shell2275901966
==> qemu.rhel9 (shell-local): Running local shell script: /tmp/packer-shell2275901966
2024/09/24 17:03:46 packer-post-processor-shell-local plugin: [INFO] (shell-local communicator): Executing local shell command [/bin/sh -c PACKER_BUILDER_TYPE='qemu' PACKER_BUILD_NAME='rhel9' PACKER_HTTP_ADDR='10.0.2.2:8668' PACKER_HTTP_IP='10.0.2.2' PACKER_HTTP_PORT='8668'  /tmp/packer-shell2275901966]
==> qemu.rhel9 (shell-local): ++ PACKER_OUTPUT=output-rhel9/packer-rhel9
==> qemu.rhel9 (shell-local): ++ IMG_FMT=qcow2
==> qemu.rhel9 (shell-local): ++ '[' '!' -f output-rhel9/packer-rhel9 ']'
==> qemu.rhel9 (shell-local): ++ sync -f output-rhel9/packer-rhel9
==> qemu.rhel9 (shell-local): +++ mktemp -d /tmp/packer-maas-XXXX
==> qemu.rhel9 (shell-local): ++ TMP_DIR=/tmp/packer-maas-TqqX
==> qemu.rhel9 (shell-local): ++ trap cleanup EXIT
==> qemu.rhel9 (shell-local): ++ sleep 5
==> qemu.rhel9 (shell-local): ++ qemu-nbd --socket=/tmp/packer-maas-TqqX/qemu-img.sock --format=qcow2 --shared=10 output-rhel9/packer-rhel9
==> qemu.rhel9 (shell-local): ++ mkdir -p /tmp/packer-maas-TqqX/root
==> qemu.rhel9 (shell-local): + source ../scripts/fuse-tar-root
    qemu.rhel9 (shell-local): Mounting root partition...
==> qemu.rhel9 (shell-local): ++ ROOT_PARTITION=1
==> qemu.rhel9 (shell-local): ++ ROOT_MOUNT=/tmp/packer-maas-TqqX/root
==> qemu.rhel9 (shell-local): ++ DETECT_BLS_BOOT=0
==> qemu.rhel9 (shell-local): ++ BOOT_MOUNT=/tmp/packer-maas-TqqX/boot
==> qemu.rhel9 (shell-local): ++ echo 'Mounting root partition...'
==> qemu.rhel9 (shell-local): ++ mount_part 1 /tmp/packer-maas-TqqX/root
==> qemu.rhel9 (shell-local): ++ PART=1
==> qemu.rhel9 (shell-local): ++ MOUNTPOINT=/tmp/packer-maas-TqqX/root
==> qemu.rhel9 (shell-local): ++ FUSEDRV=fuse2fs
==> qemu.rhel9 (shell-local): ++ DEV=/tmp/packer-maas-TqqX/p1
==> qemu.rhel9 (shell-local): ++ mkdir -p /tmp/packer-maas-TqqX/root
==> qemu.rhel9 (shell-local): ++ mkdir -p /tmp/packer-maas-TqqX/p1
==> qemu.rhel9 (shell-local): ++ retries=0
==> qemu.rhel9 (shell-local): ++ '[' -f /tmp/packer-maas-TqqX/p1/nbd ']'
==> qemu.rhel9 (shell-local): ++ sleep 1
==> qemu.rhel9 (shell-local): ++ nbdfuse /tmp/packer-maas-TqqX/p1 --command nbdkit -s nbd socket=/tmp/packer-maas-TqqX/qemu-img.sock --filter=partition partition=1
==> qemu.rhel9 (shell-local): ++ (( ++retries > 10 ))
==> qemu.rhel9 (shell-local): ++ '[' -f /tmp/packer-maas-TqqX/p1/nbd ']'
==> qemu.rhel9 (shell-local): ++ case "${FUSEDRV}" in
==> qemu.rhel9 (shell-local): ++ fuse2fs /tmp/packer-maas-TqqX/p1/nbd /tmp/packer-maas-TqqX/root -o fakeroot
    qemu.rhel9 (shell-local): /tmp/packer-maas-TqqX/p1/nbd: Bad magic number in super-block.
    qemu.rhel9 (shell-local): Please run e2fsck -fy /tmp/packer-maas-TqqX/p1/nbd.
==> qemu.rhel9 (shell-local): + cleanup
==> qemu.rhel9 (shell-local): + for dev in "${TMP_DIR}"/p*/ "${TMP_DIR}"/disk/ "${TMP_DIR}"/boot/
==> qemu.rhel9 (shell-local): + dev=/tmp/packer-maas-TqqX/p1
==> qemu.rhel9 (shell-local): + '[' -d /tmp/packer-maas-TqqX/p1 ']'
==> qemu.rhel9 (shell-local): + fusermount -u -z /tmp/packer-maas-TqqX/p1
==> qemu.rhel9 (shell-local): + grep -qs '/tmp/packer-maas-TqqX/p1 ' /proc/mounts
==> qemu.rhel9 (shell-local): + for dev in "${TMP_DIR}"/p*/ "${TMP_DIR}"/disk/ "${TMP_DIR}"/boot/
==> qemu.rhel9 (shell-local): + dev=/tmp/packer-maas-TqqX/disk
==> qemu.rhel9 (shell-local): + '[' -d /tmp/packer-maas-TqqX/disk ']'
==> qemu.rhel9 (shell-local): + for dev in "${TMP_DIR}"/p*/ "${TMP_DIR}"/disk/ "${TMP_DIR}"/boot/
==> qemu.rhel9 (shell-local): + dev=/tmp/packer-maas-TqqX/boot
==> qemu.rhel9 (shell-local): + '[' -d /tmp/packer-maas-TqqX/boot ']'
==> qemu.rhel9 (shell-local): + rm -rf /tmp/packer-maas-TqqX
2024/09/24 17:03:52 packer-post-processor-shell-local plugin: error: Script exited with non-zero exit status: 2. Allowed exit codes are: [0]
2024/09/24 17:03:52 [INFO] (telemetry) ending shell-local
2024/09/24 17:03:52 Deleting original artifact for build 'qemu.rhel9'

* Post-processor failed: Script exited with non-zero exit status: 2. Allowed exit codes are: [0]
==> Wait completed after 5 minutes 57 seconds
2024/09/24 17:03:52 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2024/09/24 17:03:52 machine readable: qemu.rhel9,error []string{"1 error(s) occurred:\n\n* Post-processor failed: Script exited with non-zero exit status: 2. Allowed exit codes are: [0]"}

* Post-processor failed: Script exited with non-zero exit status: 2. Allowed exit codes are: [0]
Build 'qemu.rhel9' errored after 5 minutes 57 seconds: 1 error(s) occurred:
==> Builds finished but no artifacts were created.

2024/09/24 17:03:52 [INFO] (telemetry) Finalizing.
* Post-processor failed: Script exited with non-zero exit status: 2. Allowed exit codes are: [0]

==> Wait completed after 5 minutes 57 seconds

==> Some builds didn't complete successfully and had errors:
--> qemu.rhel9: 1 error(s) occurred:

* Post-processor failed: Script exited with non-zero exit status: 2. Allowed exit codes are: [0]

need some help here
tried to set ROOT_PARTITION=3 but had similar output

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[kojiwell]

I'm running into the same issue with Rocky 9. @noama-nv, did you find a way to solve it?

[kojiwell]

@noama-nv , I was able to build a Rocky 9 arm64 image with the changes on this diff:
https://github.com/canonical/packer-maas/compare/main...kojiwell:packer-maas:add_rocky9_arm64?expand=1

You just have to add changes for the efi partition eveywhere in the build process, which extends the work a bit more than expected. I hope this will help.

However, as you know, this is still a work-in-progress due to the issue described here: Rocky 9 aarch64 deployment fails because of “grub2-install: error: This utility should not be used for EFI platforms because it does not support UEFI Secure Boot.”

[noama-nv]

seems like you advance more then me, ill look into your changes and try it also

i was not able to even build the image the finalize stage was failing

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[gambl3]

I used the changes in the diff above and was able to get RHEL 9.5 and Rocky 9 to build successfully. I'm currently looking into the issue from the discourse discussion as I hit the same grub2-efi error.

The function below found in /usr/lib/python3/dist-packages/curtin/command/install_grub.py may be able to solve the grub2-efi issue by adding --force. I'm still working to test this but I figured I'd share where I'm at in my work to get RHEL/Rocky arm64 to deploy with MAAS.

def gen_uefi_install_commands(grub_name, grub_target, grub_cmd, update_nvram,
distroinfo, devices, target): if grub_cmd == GRUB_MULTI_INSTALL:
....omitted for brevity...
# grub-multi-install is called with no arguments
install_cmds.append([grub_cmd])
elif grub_cmd:
install_cmds.append(
[grub_cmd, '--target=%s' % grub_target,
'--efi-directory=%s' % efidir, '--bootloader-id=%s' % bootid,
'--recheck', '--force'] + ([] if update_nvram else ['--no-nvram']))

Update: This got me past the grub2-efi error