@@ -4,38 +4,42 @@ Booting Ubuntu live ISOs via U-Boot
===================================
U-Boot can replace GRUB as the bootloader on an Ubuntu live ISO.
-``scripts/ubuntu-iso-to-uboot.py`` rewrites the ISO so the appended EFI
-system partition holds a U-Boot EFI application and the ISO 9660 tree
-carries a :doc:`Boot Loader Specification Type #1 </usage/bls>` entry
-pointing at the casper kernel and initrd that Ubuntu already places on
-the disc.
+``scripts/ubuntu-iso-to-uboot.py`` rewrites the ISO so the appended EFI system
+partition holds a U-Boot EFI application and the ISO 9660 tree carries
+a :doc:`Boot Loader Specification Type #1 </usage/bls>` entry pointing at the
+casper kernel and initrd that Ubuntu already places on the disc.
-All other boot records (BIOS El Torito, grub2 MBR, GPT layout) are
-preserved verbatim by xorriso's ``-boot_image any replay``, and casper
-still finds its squashfs by disc label at runtime.
+All other boot records (BIOS El Torito, grub2 MBR, GPT layout) are preserved
+verbatim by xorriso's ``-boot_image any replay``, and casper still finds its
+squashfs by disc label at runtime.
Host prerequisites
------------------
::
- sudo apt install xorriso mtools dosfstools qemu-system-x86 ovmf
+ sudo apt install xorriso mtools dosfstools squashfs-tools fakeroot \\
+ qemu-system-x86 ovmf
+
+``squashfs-tools`` and ``fakeroot`` are only needed when the script modifies the
+install squashfs (see :ref:`target-bls-setup`); pass ``-N`` to skip that step
+if you'd rather not install them.
Building the U-Boot EFI application
-----------------------------------
The ``efi-x86_app64`` target enables ``CONFIG_BOOTMETH_BLS=y``,
-``CONFIG_FS_ISOFS=y`` and ``CONFIG_JOLIET=y`` by default, so no Kconfig
-tweaks are required. Build with::
+``CONFIG_FS_ISOFS=y`` and ``CONFIG_JOLIET=y`` by default, so no Kconfig tweaks
+are required. Build with::
make O=/tmp/b/efi-x86_app64 efi-x86_app64_defconfig
make O=/tmp/b/efi-x86_app64 -j$(nproc)
-The output is ``/tmp/b/efi-x86_app64/u-boot-app.efi``, a PE32+ x86_64
-EFI application.
+The output is ``/tmp/b/efi-x86_app64/u-boot-app.efi``, a PE32+ x86_64 EFI
+application.
-If ``rustc`` is not installed, also disable the rust example build
-before the main ``make``::
+If ``rustc`` is not installed, also disable the rust example build before the
+main ``make``::
scripts/config --file /tmp/b/efi-x86_app64/.config \\
-d RUST_EXAMPLES -d EXAMPLES
@@ -44,8 +48,7 @@ before the main ``make``::
Rewriting an Ubuntu ISO
-----------------------
-Fetch the ISO (desktop and server images both work) and run the
-helper::
+Fetch the ISO (desktop and server images both work) and run the helper::
curl -LO https://releases.ubuntu.com/24.04.1/ubuntu-24.04.1-desktop-amd64.iso
@@ -55,39 +58,51 @@ helper::
The script:
-1. Reads the input ISO's boot record with ``xorriso -report_el_torito``
- to pick up the volume label and the EFI system partition GUID.
+1. Reads the input ISO's boot record with ``xorriso -report_el_torito`` to pick
+ up the volume label and the EFI system partition GUID, and parses
+ ``/boot/grub/grub.cfg`` to inherit the kernel command line from the source
+ ISO's own default entry (``--- quiet splash`` on Ubuntu 24.04.1).
2. Builds a small FAT ESP (4 MiB by default) containing just
``/EFI/BOOT/BOOTX64.EFI`` -- the U-Boot EFI application.
-3. Writes a new ISO with
- ``xorriso -indev ... -outdev ... -boot_image any replay
- -append_partition 2 ... -map entry.conf /loader/entry.conf``,
- which replaces the original ESP and adds ``/loader/entry.conf`` to
- the ISO 9660 tree. The kernel and initrd stay in ``/casper/`` on the
- ISO 9660 tree; U-Boot reads them directly via its isofs driver.
+3. Writes a new ISO with ``xorriso -indev ... -outdev ... -boot_image any replay
+ -append_partition 2 ... -map entry.conf /loader/entry.conf``, which replaces
+ the original ESP and adds ``/loader/entry.conf`` to the ISO 9660 tree. The
+ kernel and initrd stay in ``/casper/`` on the ISO 9660 tree; U-Boot reads
+ them directly via its isofs driver.
4. Strips the shim, GRUB and MOK manager binaries
- (``/EFI/boot/{bootx64,grubx64,mmx64}.efi``) from the ISO 9660 tree.
- The UEFI firmware loads ``BOOTX64.EFI`` from the appended ESP, so
- the ISO 9660 copies are unused dead weight. The BIOS El Torito
- image under ``/boot/grub/`` is left in place, so legacy-BIOS boot
- still chains into GRUB as before.
+ (``/EFI/boot/{bootx64,grubx64,mmx64}.efi``) from the ISO 9660 tree. The UEFI
+ firmware loads ``BOOTX64.EFI`` from the appended ESP, so the ISO 9660 copies
+ are unused dead weight. The BIOS El Torito image under ``/boot/grub/`` is
+ left in place, so legacy-BIOS boot still chains into GRUB as before.
+5. Wires BLS maintenance into the target system that a later ``Install Ubuntu``
+ produces; see :ref:`target-bls-setup`.
Relevant options:
* ``-u PATH`` -- the U-Boot EFI application (required).
* ``-o PATH`` -- the output ISO (required).
* ``-a ARGS`` -- override the kernel command line written to
- ``loader/entry.conf``. The default is
- ``console=ttyS0,115200 console=tty0 --- quiet``, which logs the
- kernel to serial and video. Duplicate the ``console=`` arguments
- after ``---`` as well if you want casper and the running system
- logged to serial too.
-* ``-k PATH`` / ``-i PATH`` -- override the kernel and initrd paths
- inside the ISO if a distribution uses something other than
- ``casper/vmlinuz`` and ``casper/initrd``.
-* ``-s MiB`` -- force an ESP size; the default is 4 MiB, enough for the
- U-Boot binary.
+ ``loader/entry.conf``. When not given, the arguments after the first
+ ``linux /casper/vmlinuz`` entry in the source ISO's ``/boot/grub/grub.cfg``
+ are used verbatim, so the rewritten ISO boots exactly as the original. Pass
+ something like ``console=ttyS0,115200 console=tty0 --- quiet splash`` when
+ you want the kernel logged to serial too.
+* ``-k PATH`` / ``-i PATH`` -- override the kernel and initrd paths inside the
+ ISO if a distribution uses something other than ``casper/vmlinuz`` and
+ ``casper/initrd``.
+* ``-s MiB`` -- force an ESP size; the default is 4 MiB, enough for the U-Boot
+ binary.
* ``-t TITLE`` -- override the BLS entry title.
+* ``-N`` / ``--no-target-bls`` -- skip the :ref:`target-bls-setup` step and
+ produce a plain rewritten ISO. Useful when you do not want ``squashfs-tools``
+ / ``fakeroot`` on the host or cannot afford the extra ~3 minutes the squashfs
+ repack adds.
+* ``-I PATH`` / ``--install-squashfs PATH`` -- path inside the ISO of the
+ install squashfs to modify for interactive installs. Defaults to
+ ``casper/minimal.squashfs``; set to an empty string to skip the squashfs step
+ while still shipping the autoinstall snippet.
+* ``-v`` -- show progress markers and subprocess output (xorriso, mksquashfs,
+ mkfs.vfat). Off by default.
Testing under QEMU + OVMF
-------------------------
@@ -113,8 +128,83 @@ The expected boot trace on the serial console is roughly::
Starting kernel ...
The bootflow appears on ``part_1`` -- the ISO 9660 partition -- because
-``entry.conf`` lives in the ISO 9660 tree and U-Boot reads kernel and
-initrd from the same partition via isofs.
+``entry.conf`` lives in the ISO 9660 tree and U-Boot reads kernel and initrd
+from the same partition via isofs.
+
+.. _target-bls-setup:
+
+Keeping BLS entries fresh after install
+---------------------------------------
+
+A rewritten live ISO boots via U-Boot + BLS, but once the user clicks
+*Install Ubuntu*, apt-managed kernel updates still go through ``update-grub``
+on the target. They do not refresh ``/boot/loader/entries/``, so the installed
+system keeps booting the kernel the installer unpacked rather than whatever
+``apt install linux-image-*`` lays down later.
+
+By default the script reaches both possible install paths to seed BLS entries
+for the installer's kernel:
+
+1. **Autoinstall path.** ``/autoinstall.yaml`` at the ISO 9660 root carries
+ ``late-commands`` that, after subiquity has finished copying the system to
+ disk, write a BLS entry per kernel directly to
+ ``/boot/efi/loader/entries/<v>.conf`` on the install target, copying the
+ kernel and initrd alongside on the ESP so the entry can reference them by
+ ESP-relative paths. A ``/loader/entry.conf`` fallback is written too,
+ matching the format the live ISO already uses. Plain interactive installs
+ ignore ``/autoinstall.yaml`` entirely, so shipping the file is harmless when
+ autoinstall is not requested.
+
+ The ``kernel-install`` round-trip used by other distros is bypassed: inside
+ curtin's chroot, ``/boot/efi`` is a plain directory rather than the ESP
+ mountpoint, and ``systemd``'s ``90-loaderentry.install`` plugin silently
+ exits when it cannot find a real ESP under ``$BOOT_ROOT``.
+
+ Entries land on the ESP rather than on ``/boot/loader/entries/`` on the
+ rootfs because the bootflow iterator advances per-partition. On the install
+ target the ESP is the lower-numbered partition; without an entry there the
+ EFI bootmeth would match ``/EFI/BOOT/BOOTX64.EFI`` before the BLS bootmeth
+ ever reached the rootfs further down, chain-loading U-Boot into itself.
+ Putting the entry on the ESP lets the BLS bootmeth win on that partition
+ before the EFI bootmeth has a turn. A separately maintained
+ ``/boot/loader/entries/`` on the rootfs would still be picked up (the BLS
+ bootmeth scans every partition); this just guarantees a working entry on
+ the partition the iterator reaches first.
+
+2. **Interactive path.** The script unpacks the default install squashfs
+ (``casper/minimal.squashfs``), drops in a ``oneshot`` systemd unit plus its
+ helper script, and repacks. On first boot of the installed system the unit
+ runs::
+
+ apt-get update
+ apt-get install -y systemd-boot-efi
+ mkdir -p /boot/loader/entries
+ for k in /usr/lib/modules/*; do
+ kernel-install add "$(basename "$k")" "$k/vmlinuz" || true
+ done
+ touch /var/lib/ubuntu-iso-to-uboot-bls-setup.done
+
+ The unit is gated on ``ConditionPathExists=!/cdrom`` so it stays dormant
+ inside the live session, and the done-file stops it re-running on subsequent
+ boots. It needs a network on first boot for the apt call; if that is not
+ available it fails gracefully and retries on the next boot.
+
+For interactive installs, every subsequent ``apt install linux-image-*`` on the
+target updates ``/boot/loader/entries/`` automatically via the
+``kernel-install`` hooks, and U-Boot's BLS bootmeth keeps picking up the current
+kernel/initrd without regenerating the ISO. Autoinstall systems ship with one
+entry per kernel the installer unpacked but do not auto-refresh on later kernel
+updates -- the autoinstall path is intended to get the freshly installed system
+to its first boot under U-Boot, not to replace ``kernel-install``. Run the
+interactive path's first-boot unit (or ``kernel-install add`` manually)
+afterwards if you want apt-driven kernel updates to maintain BLS entries on an
+autoinstalled system.
+
+Pass ``-N`` / ``--no-target-bls`` to skip both steps and produce a plain
+rewritten ISO. The squashfs repack is the slow part of the build (about three
+extra minutes on the 24.04.1 desktop ISO); skipping it keeps the rewrite under
+a minute at the cost of needing a manual ``kernel-install add`` on the
+installed system.
See also
--------
@@ -122,3 +212,4 @@ See also
* :doc:`/usage/bls` -- the U-Boot Boot Loader Specification bootmeth.
* `Boot Loader Specification
<https://uapi-group.org/specifications/specs/boot_loader_specification/>`_.
+* ``kernel-install(8)`` -- systemd's kernel install hook.
@@ -76,9 +76,53 @@ sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'tools'))
from u_boot_pylib import command
from u_boot_pylib import tout
-REQUIRED_TOOLS = ('xorriso', 'mcopy', 'mmd', 'mkfs.vfat')
+REQUIRED_TOOLS = (
+ 'xorriso', 'mcopy', 'mmd', 'mkfs.vfat',
+ 'unsquashfs', 'mksquashfs', 'fakeroot',
+)
MIB = 1024 * 1024
+# First-boot systemd unit + helper script written into the install
+# squashfs so kernel-install manages BLS entries on the installed
+# system. ConditionPathExists=!/cdrom keeps the unit dormant inside
+# the live session, and the done-file stops it re-running on every
+# subsequent boot.
+FIRST_BOOT_SCRIPT = '''\
+#!/bin/sh
+# Set up BLS entries for future kernel updates
+set -e
+apt-get update
+apt-get install -y systemd-boot-efi
+mkdir -p /boot/loader/entries
+# Force kernel-install's BLS layout: with the default layout=auto
+# the heuristic can pick 'other' on Ubuntu and skip the
+# 90-loaderentry.install plugin
+grep -q '^layout=' /etc/kernel/install.conf 2>/dev/null \\
+ || echo layout=bls >> /etc/kernel/install.conf
+for k in /usr/lib/modules/*; do
+ v=$(basename "$k")
+ kernel-install add "$v" "/boot/vmlinuz-$v" || true
+done
+touch /var/lib/ubuntu-iso-to-uboot-bls-setup.done
+'''
+
+FIRST_BOOT_UNIT = '''\
+[Unit]
+Description=Set up BLS entries for future kernel updates
+ConditionPathExists=!/var/lib/ubuntu-iso-to-uboot-bls-setup.done
+ConditionPathExists=!/cdrom
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=oneshot
+ExecStart=/usr/local/sbin/ubuntu-iso-to-uboot-bls-setup
+RemainAfterExit=yes
+
+[Install]
+WantedBy=multi-user.target
+'''
+
def _quiet() -> bool:
"""True when tout is set below INFO (no progress chatter requested)."""
@@ -180,16 +224,17 @@ def build_esp(esp_path: Path, size_mib: int, uboot_efi: Path) -> None:
def repack_iso(
in_iso: Path, out_iso: Path, esp_img: Path, esp_guid: str,
- entry_conf: Path,
+ file_maps: list[tuple[Path, str]],
) -> None:
"""Stream the input ISO to a new ISO with the ESP and BLS entry replaced.
-boot_image any replay preserves every other boot record (BIOS El Torito,
grub2 MBR, GPT layout); only the bytes behind partition 2 are rewritten,
- plus /loader/entry.conf is added to the ISO 9660 tree, and the shim,
- GRUB and MokManager copies under /EFI/boot/ are removed since U-Boot
- supplies the UEFI boot path via the appended ESP. The BIOS El Torito
- path still uses /boot/grub/ so legacy boot continues to work.
+ plus each (src, dst) pair in @file_maps is added to the ISO 9660 tree,
+ and the shim, GRUB and MokManager copies under /EFI/boot/ are removed
+ since U-Boot supplies the UEFI boot path via the appended ESP. The
+ BIOS El Torito path still uses /boot/grub/ so legacy boot continues
+ to work.
-find is tolerant of missing files: if a distribution does not ship
one of these binaries, the call is a no-op.
@@ -200,17 +245,131 @@ def repack_iso(
"""
if out_iso.exists():
out_iso.unlink()
- _run(
+ cmd = [
'xorriso',
'-indev', str(in_iso),
'-outdev', str(out_iso),
'-boot_image', 'any', 'replay',
'-append_partition', '2', esp_guid, str(esp_img),
- '-map', str(entry_conf), '/loader/entry.conf',
+ ]
+ for src, dst in file_maps:
+ cmd += ['-map', str(src), dst]
+ cmd += [
'-find', '/EFI/boot', '-name', 'bootx64.efi', '-exec', 'rm', '--',
'-find', '/EFI/boot', '-name', 'grubx64.efi', '-exec', 'rm', '--',
'-find', '/EFI/boot', '-name', 'mmx64.efi', '-exec', 'rm', '--',
'-commit',
+ ]
+ _run(*cmd)
+
+
+def inject_first_boot_unit(
+ iso: Path, sqfs_in_iso: str, work: Path,
+) -> Path:
+ """Unpack the install squashfs, drop in a first-boot BLS-setup unit
+ plus its helper script, and repack.
+
+ The whole unpack/modify/repack cycle runs under one fakeroot
+ invocation so ownership survives round-tripping through a regular
+ user filesystem. Compression matches the original (xz) to keep the
+ resulting ISO close to the source in size; that does mean the
+ rewrite takes several minutes.
+
+ Returns the path to the modified squashfs.
+ """
+ extracted = work / 'orig.squashfs'
+ modified = work / 'modified.squashfs'
+ stage = work / 'sqfs-stage'
+ aux = work / 'aux'
+ aux.mkdir()
+ script_src = aux / 'ubuntu-iso-to-uboot-bls-setup'
+ script_src.write_text(FIRST_BOOT_SCRIPT)
+ script_src.chmod(0o755)
+ unit_src = aux / 'ubuntu-iso-to-uboot-bls-setup.service'
+ unit_src.write_text(FIRST_BOOT_UNIT)
+
+ tout.notice(f'=> Extracting {sqfs_in_iso}')
+ _run(
+ 'xorriso', '-osirrox', 'on', '-indev', str(iso),
+ '-extract', '/' + sqfs_in_iso.lstrip('/'), str(extracted),
+ )
+
+ tout.notice('=> Unpacking, injecting unit, repacking (xz, slow)')
+ shell = f'''
+set -e
+unsquashfs -d '{stage}' '{extracted}'
+install -D -m 755 -o root -g root '{script_src}' \\
+ '{stage}/usr/local/sbin/ubuntu-iso-to-uboot-bls-setup'
+install -D -m 644 -o root -g root '{unit_src}' \\
+ '{stage}/etc/systemd/system/ubuntu-iso-to-uboot-bls-setup.service'
+mkdir -p '{stage}/etc/systemd/system/multi-user.target.wants'
+ln -sf ../ubuntu-iso-to-uboot-bls-setup.service \\
+ '{stage}/etc/systemd/system/multi-user.target.wants/ubuntu-iso-to-uboot-bls-setup.service'
+chown -h root:root \\
+ '{stage}/etc/systemd/system/multi-user.target.wants/ubuntu-iso-to-uboot-bls-setup.service'
+mksquashfs '{stage}' '{modified}' -noappend -comp xz -no-progress
+'''
+ _run('fakeroot', 'sh', '-c', shell)
+ return modified
+
+
+def autoinstall_yaml() -> str:
+ """Return an autoinstall snippet that seeds BLS entries on the
+ installed ESP.
+
+ The Ubuntu installer (ubiquity/subiquity) reads /autoinstall.yaml from the
+ install media when autoinstall mode is invoked; the file is ignored in plain
+ interactive installs, so shipping it by default is harmless. When
+ autoinstall runs, the late-commands here write a BLS Type #1 entry to
+ /boot/efi/loader/entries/<v>.conf for every kernel the installer has
+ unpacked and copy the kernel and initrd alongside on the ESP, so U-Boot's
+ bootmeth_bls finds the entry on the ESP partition before the EFI bootmeth on
+ the same partition chain-loads U-Boot's BOOTX64.EFI fallback into a loop.
+
+ The kernel-install path is bypassed because curtin's chroot has /boot/efi as
+ a plain directory rather than the ESP mountpoint, so systemd-boot-efi's
+ 90-loaderentry.install plugin silently exits without writing entries.
+ """
+ return (
+ '#cloud-config\n'
+ 'autoinstall:\n'
+ ' version: 1\n'
+ ' late-commands:\n'
+ # Write BLS entries to the install target's ESP. The kernel
+ # and initrd are copied alongside on the ESP so the entry can
+ # reference them by ESP-relative paths. We place the entries
+ # on the ESP rather than /boot/loader/entries on the rootfs
+ # because U-Boot's bootflow iterator advances per-partition:
+ # the ESP is the lower-numbered partition and EFI bootmeth
+ # finds our BOOTX64.EFI fallback there before BLS reaches the
+ # rootfs further down. Putting the entries on the ESP lets
+ # BLS win on that partition before EFI has a turn. We also
+ # write a /loader/entry.conf (singular) fallback matching the
+ # format the live ISO already uses, so U-Boot picks the entry
+ # whether its scan prefers the entries/ directory or the
+ # singular file.
+ ''' - |
+ curtin in-target -- sh -c 'set -e;\
+ ESP=/boot/efi;\
+ mkdir -p "$ESP/loader/entries";\
+ uuid=$(findmnt -no UUID /);\
+ last_v="";\
+ for k in /usr/lib/modules/*; do\
+ v=$(basename "$k");\
+ cp "/boot/vmlinuz-$v" "$ESP/vmlinuz-$v";\
+ cp "/boot/initrd.img-$v" "$ESP/initrd.img-$v";\
+ printf "title Ubuntu (%s)\\n\
+linux /vmlinuz-%s\\n\
+initrd /initrd.img-%s\\n\
+options root=UUID=%s ro console=ttyS0,115200 console=tty0\\n"\
+ "$v" "$v" "$v" "$uuid"\
+ > "$ESP/loader/entries/$v.conf";\
+ last_v="$v";\
+ done;\
+ if [ -n "$last_v" ]; then\
+ cp "$ESP/loader/entries/$last_v.conf" "$ESP/loader/entry.conf";\
+ fi'
+'''
)
@@ -234,6 +393,15 @@ def main() -> None:
help='BLS entry title (default: derived from volume label)')
p.add_argument('-s', '--esp-size', type=int, default=None,
help='ESP size in MiB (default: 4 MiB)')
+ p.add_argument('-N', '--no-target-bls', action='store_true',
+ help='do not ship an autoinstall snippet or modify '
+ 'the install squashfs to wire kernel-install + '
+ 'BLS into the installed system')
+ p.add_argument('-I', '--install-squashfs',
+ default='casper/minimal.squashfs',
+ help='path inside the ISO of the install squashfs to '
+ 'modify for interactive installs '
+ '(default: %(default)s; set to empty to skip)')
p.add_argument('-v', '--verbose', action='store_true',
help='show progress markers and subprocess output')
args = p.parse_args()
@@ -275,8 +443,21 @@ def main() -> None:
f'options {cmdline}\n'
)
+ file_maps = [(entry, '/loader/entry.conf')]
+ if not args.no_target_bls:
+ ai = work / 'autoinstall.yaml'
+ ai.write_text(autoinstall_yaml())
+ file_maps.append((ai, '/autoinstall.yaml'))
+ if args.install_squashfs:
+ modified_sqfs = inject_first_boot_unit(
+ args.iso, args.install_squashfs, work,
+ )
+ file_maps.append(
+ (modified_sqfs, '/' + args.install_squashfs.lstrip('/')),
+ )
+
tout.notice(f'=> Repacking to {args.out}')
- repack_iso(args.iso, args.out, esp, esp_guid, entry)
+ repack_iso(args.iso, args.out, esp, esp_guid, file_maps)
size_mib = args.out.stat().st_size / MIB
tout.notice(f'=> Done: {args.out} ({size_mib:.1f} MiB)')