OSTree extlinux.conf on Nvidia Jetson Xavier #2255
Comments
|
I've got a patch that implements |
|
@wmanley , nice! Sounds like something I'm going to need. Any special instructions for using it, once it is available? My day is just getting started and I'd love to test it out. I'm guessing I'll need to compile ostree for my build machine (that makes the image) and for the Xavier (to handle upgrades)? Let me know when you have it up and I'll try it out, immediately. Thank you! |
|
I've raised #2256 with my bootloader patches. You'll also need to add: To your |
|
@wmanley , related to this goal, but unrelated to OSTree... when I attempt to flash the Xavier device, flash.sh pushes into the rootfs a new copy of the kernel, as well as many other files. Do you have an existing work-around to maintain the integrity of the rootfs while the device is being flashed? I also seem to have an issue, when I manually set the /boot/extlinux/extlinux.conf boot options that I can't get it to boot with the kernel/initrd/devicetree provided by OSTree... only with the /boot/Image that is inserted during the flash.sh process. Do you have a script to work around this issue? |
|
@wmanley , I must be missing a step, somewhere. Here is exactly the steps I have taken: git clone https://github.com/stb-tester/ostree.git
cd ostree
git checkout u-boot-extlinux-compatibilityAt this point, I created the following Dockerfile to build OSTree: FROM ubuntu:20.04 as builder
RUN mkdir -p /root/prj
WORKDIR /root/prj
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y build-essential dh-autoreconf bison e2fslibs-dev \
libgtk2.0-dev liblzma-dev libgpgme-dev libfuse-dev \
libsoup2.4-dev && \
rm -rf /var/lib/apt/lists/*
COPY . .
RUN mkdir -p out && \
env NOCONFIGURE=1 ./autogen.sh && \
./configure --prefix=/usr && \
make && \
make install DESTDIR=/root/prj/out
FROM scratch
COPY --from=builder /root/prj/out/ /I built and installed with: DOCKER_BUILDKIT=1 docker build --output=./out/ .
sudo cp -r ./out/usr/* /usr/After having this version of OSTree installed, I ran the same setup script I placed in the issue (no changes to anything, yet) with the anticipation that I would discover a syslinux.cfg in rootfs/boot/loader/. I did not see this file. Am I missing a step or argument? |
I've not been using flashing. I copy ostree to a running system and convert it from the inside, so I can't help here. |
|
@wmanley , am I missing a step (or compiler dependency, flag, etc...) to create the syslinux.cfg? Or is it required that a syslinux.cfg exists before OSTree will take it over? |
It's required that |
|
I've managed to get a deployment to boot into OSTree. When I have it automated, I'll post the resulting script, here, for anyone it might help (and then close the issue). |
|
So, I got it working (mostly). Final script is below. There are still some issues:
The script does require an SSH public key to exist so that you can SSH into the device after flashing. Because of these issues, I'm not sure if the pull request is fully complete, @wmanley (namely, the need for "\tKERNEL" and that the created syslinux.cfg isn't bootable). #!/bin/bash
set -euo pipefail # Exits script on any error.
# To run this script a few base packages are required. Run the following command
# to install this base packages:
#
# apt install -y build-essential debootstrap qemu-user-static ostree
#
# Requires file Jetson_Linux_R32.4.4_aarch64.tbz2 to be in the same directory as
# this setup.sh script.
HOMEDIR="/home/$USER"
echo "Will be using SSH key from the following home directory: $HOMEDIR"
# Start fresh and remove any existing builds. Existing root filesystems may have
# the immutable flag set, so the flag needs to be cleared before the directory
# can be removed.
sudo chattr -i out/Linux_for_Tegra/rootfs/ostree/deploy/mine/deploy/* || /bin/true
sudo rm -rf out || /bin/true
mkdir out
cd out
# Extract the Jetson tools
tar -xf ../Jetson_Linux_R32.4.4_aarch64.tbz2
cd Linux_for_Tegra
# Build the filesystem.
sudo qemu-debootstrap --arch=arm64 bionic rootfs
sudo /bin/bash -c "echo \"deb http://ports.ubuntu.com/ubuntu-ports bionic main universe\" > rootfs/etc/apt/sources.list"
sudo chroot rootfs /bin/bash -c "apt update && apt install -y libdrm-common libdrm2 libegl-mesa0 libegl1 libgbm1 libglapi-mesa libglvnd0 libwayland-client0 libwayland-server0 libx11-xcb1 libxau6 libxcb-dri2-0 libxcb-dri3-0 libxcb-present0 libxcb-sync1 libxcb-xfixes0 libxcb1 libxdmcp6 libxshmfence1 multiarch-support device-tree-compiler fontconfig fontconfig-config fonts-dejavu-core libcairo2 libdatrie1 libevdev2 libfontconfig1 libfreetype6 libgles2 libgraphite2-3 libgudev-1.0-0 libharfbuzz0b libinput-bin libinput10 libjpeg-turbo8 libmtdev1 libpango-1.0-0 libpangoft2-1.0-0 libpixman-1-0 libpng16-16 libpython-stdlib libpython2.7-minimal libpython2.7-stdlib libthai-data libthai0 libunwind8 libwacom-bin libwacom-common libwacom2 libwayland-cursor0 libwayland-egl1-mesa libx11-6 libx11-data libxcb-render0 libxcb-shm0 libxext6 libxkbcommon0 libxrender1 python python-minimal python2.7 python2.7-minimal gstreamer1.0-plugins-base iso-codes libasound2-data libcdparanoia0 libogg0 libopus0 liborc-0.4-0 libtheora0 libvisual-0.4-0 libvorbis0a libvorbisenc2 libgstreamer-plugins-bad1.0-0 libasound2 libpangocairo-1.0-0 openssh-server network-manager ostree && rm -rf /var/lib/apt/lists/*"
sudo mkdir -p rootfs/root/.ssh && sudo cp $HOMEDIR/.ssh/id_rsa.pub rootfs/root/.ssh/authorized_keys
sudo ./apply_binaries.sh
sudo rm -f rootfs/lib/systemd/system/nv-oem-config*.*
sudo rm -f rootfs/usr/sbin/nv-oem-config-firstboot
sudo rm -f rootfs/etc/systemd/nv-oem-config*.sh
sudo rm -f rootfs/etc/nv-oem-config.conf.*
sudo rm rootfs/README.txt
# Adapt filesystem to OSTree
# - /bin -> /usr/bin
sudo mv rootfs/bin/* rootfs/usr/bin/ && sudo rmdir rootfs/bin && sudo ln -s /usr/bin rootfs/bin
# - /sbin -> /usr/sbin
sudo mv rootfs/sbin/* rootfs/usr/sbin/ && sudo rmdir rootfs/sbin && sudo ln -s /usr/sbin rootfs/sbin
# - /home -> /var/home
sudo mkdir rootfs/var/home && sudo rmdir rootfs/home && sudo ln -s /var/home rootfs/home
# - /lib -> /usr/lib
sudo mv rootfs/lib/aarch64-linux-gnu/* rootfs/usr/lib/aarch64-linux-gnu/ && sudo rmdir rootfs/lib/aarch64-linux-gnu
sudo mv rootfs/lib/systemd/* rootfs/usr/lib/systemd/ && sudo rmdir rootfs/lib/systemd
sudo mv rootfs/lib/* rootfs/usr/lib/ && sudo rmdir rootfs/lib && sudo ln -s /usr/lib rootfs/lib
# - /opt -> /var/opt
sudo mv rootfs/opt/* rootfs/var/opt/ && sudo rmdir rootfs/opt && sudo ln -s /var/opt rootfs/opt
# - /root -> /var/roothome
sudo mv rootfs/root rootfs/var/roothome && sudo ln -s /var/roothome rootfs/root
# - /media -> /run/media
sudo rmdir rootfs/media && sudo ln -s /run/media rootfs/media
# - /etc -> /usr/etc
sudo mv rootfs/etc rootfs/usr/etc
# Setup OSTree-Systemd tmpfiles.
sudo bash -c 'cat <<EOF > rootfs/usr/lib/tmpfiles.d/ostree.conf
L /var/home - - - - ../sysroot/home
d /var/roothome 0700 root root -
d /var/opt 0755 root root -
d /var/local 0755 root root -
d /run/media 0755 root root -
EOF'
# Move Linux, initrd and devicetree to /usr/lib/modules/${version}
modules="rootfs/usr/lib/modules/4.9.140-tegra"
sudo mv rootfs/boot/tegra194-p2888-0001-p2822-0000.dtb ${modules}/devicetree
sudo mv rootfs/boot/Image ${modules}/vmlinuz
sudo mv rootfs/boot/initrd ${modules}/initramfs.img
# Clear /dev so the OS can be commited.
sudo rm -rf rootfs/dev/*
# Create a repo and commit the OSTree image.
sudo mkdir repo
sudo ostree init --repo=repo --mode=archive
sudo ostree --repo=repo commit --branch=mine rootfs
# Create a fresh rootfs.
sudo mv rootfs orig_rootfs
sudo mkdir rootfs
# Create the OSTree image.
sudo ostree admin init-fs rootfs
sudo ostree --sysroot=rootfs admin os-init mine
sudo ostree --repo=rootfs/ostree/repo remote add mine http://10.0.1.4:8080 --no-gpg-verify
sudo ostree --repo=rootfs/ostree/repo pull-local --remote=mine repo mine
sudo mkdir rootfs/boot/extlinux
sudo mv orig_rootfs/var rootfs/
sudo mv orig_rootfs/boot/extlinux/extlinux.conf rootfs/boot/extlinux/
# Create links
sudo rm -rf rootfs/bin || /bin/true
sudo rm -rf rootfs/sbin || /bin/true
sudo rm -rf rootfs/home || /bin/true
sudo rm -rf rootfs/lib || /bin/true
sudo rm -rf rootfs/opt || /bin/true
sudo rm -rf rootfs/root || /bin/true
sudo rm -rf rootfs/media || /bin/true
sudo rm -rf rootfs/etc || /bin/true
sudo ln -s /usr/bin rootfs/bin
sudo ln -s /usr/sbin rootfs/sbin
sudo ln -s /var/home rootfs/home
sudo ln -s /usr/lib rootfs/lib
sudo ln -s /var/opt rootfs/opt
sudo ln -s /var/roothome rootfs/root
sudo ln -s /run/media rootfs/media
sudo ln -s /usr/etc rootfs/etc
# Set up expected extlinux.conf.
sudo bash -c '
cat <<EOF > rootfs/boot/extlinux/extlinux.conf
TIMEOUT 30
DEFAULT primary
MENU TITLE L4T boot options
LABEL primary
MENU LABEL primary kernel
KERNEL /boot/Image
INITRD /boot/initrd
APPEND quiet
EOF
'
# Complete OS creation.
sudo ostree --sysroot=rootfs admin deploy --os=mine mine:mine
cd rootfs
usr_dir=`echo ostree/deploy/mine/deploy/*/usr`
cd ..
sudo ln -s "/$usr_dir" rootfs/usr
# Fix extlinux.conf (syslinux.cfg).
orig="rootfs/boot/loader/syslinux.cfg"
tmpfile="extlinux.conf"
cat <<EOF > $tmpfile
TIMEOUT 30
DEFAULT primary
MENU TITLE L4T boot options
LABEL primary
MENU LABEL primary kernel
EOF
cat $orig | grep ostree | grep -v LABEL >> $tmpfile
echo " APPEND quiet" >> $tmpfile
sudo cp $tmpfile $orig
sudo mv $tmpfile rootfs/boot/extlinux/extlinux.conf
# Inform user of next step.
echo "========================================================================="
echo "To flash device run:"
echo " cd out/Linux_for_Tegra"
echo " sudo ./flash.sh jetson-agx-xavier-devkit mmcblk0p1"
echo "To complete install after flashing:"
echo " rm /boot/extlinux/extlinux.conf"
echo " ln -s /boot/loader/syslinux.cfg /boot/extlinux/extlinux.conf"
echo "========================================================================="
|
|
I thought I got it working, but when I try to deploy, I get a message about not being booted into an OSTree OS (though ostree admin status gives information about the deployment, which is why I thought everything was working) and that I need to select an OS. Selecting 'mine' as the OS (based on the script, above) and manually updating the /boot/extlinux/extlinux.conf and rebooting does not result in the changes from the changeset being applied (adding a single file to /usr/bin/). I'm clearly missing something. If I drop the symlink for /usr, the OS won't boot. What am I missing or misunderstanding? How did you get yours working from inside the booted OS @wmanley ? |
|
For what it is worth, I haven't actually been fully successful getting this device to boot into an OSTree-based OS. I thought I had for a time, as 'ostree admin status' returned valid output, but when I tried to deploy or upgrade, it would tell me I wasn't booted into an OSTree system. After running 'ostree admin deploy mine:mine --os=mine' on the live system and rebooting, I get the following message: I do see entries in /boot/loader/entries/ and I match the configuration (manually) in my /boot/extlinux/extlinux.conf file before booting. The only thing I can point to as a possible problem is the following that I get during boot in the system logs: I'm getting this with a manual build of OSTree 2020.8. |
My goal is to create an image that can be flashed to an Nvidia Jetson Xavier device that is based on OSTree. I use the script, below, to create the rootfs. According to the Nvidia Documentation the device requires the kernel, devicetree and initrd to be referenced by /boot/extlinux/extlinux.conf. After running the below script, this file does not exist.
My questions are:
The text was updated successfully, but these errors were encountered: