feat: Investigate NFIT parsing for uki pxe booting

[Itxaka]

So in UEFI 2.5 there is this NFIT Ram disk feature:

RAM Disks (such as a bootable ISO image) are loaded into memory, and the NFIT table is used to describe them to the operating system as NVDIMM-like regions.

This is not the typical NFIT use for real NVDIMMs, but a special UEFI-driven setup where a RAM disk acts like persistent memory, and the OS can access it via information in the NFIT table. This is mostly used to make the UEFI-loaded ISO image available to the OS at runtime, without requiring re-reading from storage.

So basically it exposes that in memory iso when you pxe boot into it so the OS can access it like if it was still attached.

I tested to http boot a kairos non uki iso and indeed, the device is available after boot and immucore finds it by label and mounts it under /run/initramfs/live and /run/rootfsbase automatically.

But I did the same with a uki iso and its now available. I tried several methods to be able to read the memory and mount it directly but /proc/mem is protected.

We should investigate if this is possible. Seems like the change is systemd-boot either cleaning up stuff or something like that. The one that worked was the normal efi iso which uses grub.

Testing:

• Create a normal non-uki iso and boot it from http directly (a simple server and manual entry in the firmware is good enough)
• Test that after mount /dev/pmem0 or similar exists and it has the iso on it
• Change the bootloader in the iso to be systemd-boot with a normal type1 entry pointing tot he kernel and initrd as normal
• Test this new change to see if it still exposes the iso
• If it does, then the problem is on loading EFI files vs loading kernel+iso
• If it doesnt then the problem is that systemd somehow erases that memory or misses passing some acpi tables?
• In any case, if grub works and systemd-boot doesnt, ask the systemd mailing list for help there. Maybe they are cleaning the acpi tables or not respecting that, maybe its just not taking them into account (noticve that systemd-boot is much simpler than grub, one of its selling points) so the feature may need to be added

What do we want from this?

Currently we have to store the pxe address with the iso into a efivar so the OS can see it and the agent can download and mount the remote iso when doing the install as when booting we dont have the iso available anymore
This would make things simpler as the iso would just appear theere and immucore would just mount it in the proper place. No efivar, no download, no nothing special needed from the installer side. Less things in the key enroller as well as it wont have to deal with creating the efivar and so on.

[Itxaka]

Fedora gets it automatically:

[Itxaka]

somehow our side doesnt, becuase immucore is stuck on waiting for sysroot

[Itxaka]

ok basically this works with a few modules added in dracut for good measure and with proper ISOs (no idea which Aurora version, but latest works as expected.

Seems like before, our isos were a bit wrong in construction, as they had 2 gaps on them AND one of the gpas had the same COS_LIVE label, so dracut would crap out becuase it would try to mount the fist COS_LIVE label, which is the gap

With an normal iso built with AuroraBoot 0.8.0 it works out of the box because the iso has a correct table.

Wrong iso:

Disk kairos.iso: 2396672 sectors, 1.1 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): A6A7CBC1-6666-425C-A26C-C949F145A905
Partition table holds up to 248 entries
Main partition table begins at sector 2 and ends at sector 63
First usable sector is 64, last usable sector is 2396504
Partitions will be aligned on 4-sector boundaries
Total free space is 1 sectors (512 bytes)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              64            1051   494.0 KiB   0700  Gap0
   2            1052            9243   4.0 MiB     EF00  EFI boot partition
   3            9244         2396503   1.1 GiB     0700  Gap1

Good iso

Disk kairos.iso: 2377344 sectors, 1.1 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): 339379AA-CA7B-47F6-B334-85549B3F4A62
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 2377310
Partitions will be aligned on 64-sector boundaries
Total free space is 61 sectors (30.5 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              64         2369087   1.1 GiB     8300  Linux filesystem
   2         2369088         2377279   4.0 MiB     EF00  EFI system partition

As you can see we had 2 gaps in there. This seems to not affect anything when booting from iso, but on http netboot it indeed does.

So EFI http non-uki support is in automatically working on our side :D

For UKI we migth need to add a few modules to autoload the stuff from immucore but it migth also work?

The idea now is to substitute the working efi iso from grub to systemd-boot and test if it also boots as expected or systemd-boot is doing something weird

[Itxaka]

This adds the missing modules to all initramfs on non-uki: kairos-io/kairos-init#99

[Itxaka]

some modules are missing on immucore and unfortunately on ubuntu we need to bring the linux-modules-extra which brings about 100Mb which is too much for UKI.

Im going to do a workaround in order to store the needed modules (2) and restore them after removing the package

[Itxaka]

umm I pushed the immucore part directyl to main by mistake but the change to support and insert the modules on boot for uki paths is this: kairos-io/immucore@00d9e81

[Itxaka]

This adds the workaround tog et the modules under ubuntu in kairos-init for uki booting: kairos-io/kairos-init#100

[Itxaka]

so now basically it needs the uki-serve comand to be simpler. No efi booting of the enroller or anything like that. I tested this with direct http booting will test tomorrow with pxe booting to see if it behaves the same (it should)

[Itxaka]

also test wiht fedora to see if they ship the modules by default or not

[Itxaka]

ok so I reverted the agent changes and improved the Auroraboot uki pxe support:

• simpler, no changes to agent
• immucore change to add the missing modules: kairos-io/immucore@00d9e81
• kairos-init to get the proper modules under ubuntu kairos-io/kairos-init@ad534e9
• Auroraboot to simplify the uki-pxe command: Simplify uki-pxe AuroraBoot#290

This works beautifully, you just pass the iso to aurora and boot and select http boot. The dhcp pxe server gets the requests, sends the data about aurora and serves the http iso.
First boot, it will autoenroll the keys like if you were using an iso, second one it will boot from the iso with secureboot enabled like its running from the iso. And when booted, it will enable the nfit modules and automount the ISO in the proper places for the install.

[Itxaka]

Also reverted the changes to agetn as they are not needed anymore, nice: kairos-io/kairos-agent#795

will now test with Fedora to see if any changes are needed or it just works.

[Itxaka]

Fedora seems to ship the modules out of the box in a normal init build so it will probably work out of the box:

$ docker run -it --rm fedora-init:42                                 
[root@753657ae6e55 /]# find / -name nfit
/usr/lib/modules/6.14.6-300.fc42.x86_64/kernel/drivers/acpi/nfit
[root@753657ae6e55 /]# find / -name nfit*
/usr/lib/modules/6.14.6-300.fc42.x86_64/kernel/drivers/acpi/nfit
/usr/lib/modules/6.14.6-300.fc42.x86_64/kernel/drivers/acpi/nfit/nfit.ko.xz
[root@753657ae6e55 /]# find / -name libnvdimm*
/usr/lib/modules/6.14.6-300.fc42.x86_64/kernel/drivers/nvdimm/libnvdimm.ko.xz
[root@753657ae6e55 /]# find / -name nd_pmem*
/usr/lib/modules/6.14.6-300.fc42.x86_64/kernel/drivers/nvdimm/nd_pmem.ko.xz
[root@753657ae6e55 /]# find / -name dax_pmem*
/usr/lib/modules/6.14.6-300.fc42.x86_64/kernel/drivers/dax/dax_pmem.ko.xz

[Itxaka]

Fedora doesnt seem to work and somehow it cant find the /sbin/init..... because its under /usr/sbin/init

Seems like under /sbin there is only the suc-upgrade?? I though it was a link now to /usr/sbin???

[Itxaka]

umm seems like somehow we are overriding the /usr/sbin dir when dropping the suc-upgrade on it....

As Fedora is not really supported under UKI I will try to fix it on kairos-init main branch for the 0.5.0 release, no need to backport any fixes IMHO.

[Itxaka]

ok everything is merged.

Aurora in main
Init in main
Immucore new version released
Agent new version released

So everything is in place for 3.5.0, the docs need updating though.

[Itxaka]

tested with Fedora 42 + latest init from main and it works as expected and loads the proper modules, mounts stuff around and install works beautifully. Will write docs and close this?

[Itxaka]

actually it fails to install due some deps...

[Itxaka]

docs: kairos-io/kairos-docs#416
Fix init libs: kairos-io/kairos-init#102