Simple Bare Metal Provisioning with iPXE

Two boots next to each other in the gras — When booting, it is important to have the right boot order.*

Intro

After getting to know Debian pre-seeding in an effort to make installing an OS more consistent, I knew I wanted to go one step further. The ideal setup process would be completely hands-off, by just booting the machine and having it install the OS automatically.

I roughly knew about something called PXE, which is this thing your server does if it doesn’t find a bootable disk and is also widely used in data centers. But I never really looked into it, because I never had the need for it. But now I had a use case and I wanted to know more about it.

What is PXE and iPXE

PXE stands for Preboot eXecution Environment and is a standard that allows a computer to boot from a network interface. It is a part of the BIOS or UEFI firmware and is used to boot a computer from a network card. It is a standard that is widely used in data centers to boot servers without the need for physical media, like a USB stick or a CD.

Every hardware vendor has their own implementation of PXE, but they all kinda follow the same standard. The PXE client sends a DHCP request to the network, the DHCP server responds with the IP address of the TFTP server and the path to the boot file. The PXE client then downloads the boot file from the TFTP server and boots the boot file.

iPXE is an open-source implementation of PXE that is more feature-rich than the PXE implementation of most hardware vendors. iPXE is able to boot from a lot of different sources, like HTTP, TFTP… iPXE is also able to chainload other bootloaders, like GRUB or the Windows bootloader. The big advantage of iPXE is that it is able to run scripts that can be used to automate the boot process.

One programm that uses iPXE as their base is netboot.xyz. netboot.xyz is able to boot a lot of different operating systems and tools, like Windows, Linux, Memtest, Clonezilla, GParted, etc by providing a menu to choose from a wide range of choices.

Setting up the DHCP & TFTP Server

I already had a OpenWRT router that was able to provide DHCP and TFTP services, so I just had to configure it to provide the correct boot file and the correct TFTP server.

After downloading the correct netboot.xyz image for my HPE Microsrvere Gen8, I uploaded it to the TFTP server and configured the DHCP server to provide the boot file and the TFTP server to all clients.

The setting I used for my DHCP server on OpenWRT

Starting off with netboot.xyz

I already managed to get netboot.xyz to PXE boot from my OpenWRT router, so I knew that it was possible to use it as a base for my own setup. I still had to manually select the debian image and provide the path to the pre-seed file, but it was a good starting point.

I first started off thinking I have to compile a netboot.xyz to provide a custom script. netboot.xyz also offers a docker image to run your own server with your own scripts, but that was a bit overkill for me, as I only wanted one script to run, which was an automated version of their Debian install.

The nteboot.xyz menu — The netboot.xyz menu

Setting up the iPXE script

After playing around for a while, I found out that netboot.xyz will try to chainload the following files in that order for you to set variables for the boot process: local-vars.ipxe -> HOSTNAME-${hostname}.ipxe -> MAC-${mac:hexraw}.ipxe -> menu.ipxe from your TFTP server. This is not documented, but is an amazing functionality.

Also nothing hinders you from providing a custom script in any of the files, as it is just a script that is run by iPXE. So I just had to provide a HOSTNAME-....ipxe file that would chainload the Debian install script with the pre-seed file. For this I took heavy inspiration from the netboot.xyz Debian script and just added the pre-seed file to the boot command.

After that my microserver.ipxe file looked like this:

#!ipxe

set debian_mirror http://deb.debian.org
set debian_base_dir debian

set os Debian
set os_arch amd64
set debian_version bookworm
set debian_mirror ${debian_mirror}
set mirrorcfg mirror/suite=${debian_version}
set dir ${debian_base_dir}/dists/${debian_version}/main/installer-${os_arch}/current/images/netboot/

set preseedurl tftp://${tftp-server}/preseed/microserver-preseed.cfg

set install_params auto=true priority=critical preseed/url=${preseedurl}
set dir ${dir}debian-installer/${os_arch}
set netcfg netcfg/choose_interface=eno1

imgfree
kernel ${debian_mirror}/${dir}/linux ${install_params} ${netcfg} ${mirrorcfg} initrd=initrd.magic
initrd ${debian_mirror}/${dir}/initrd.gz
echo
echo MD5sums:
md5sum linux initrd.gz

boot ||
echo Boot from ${debian_mirror}/${dir} failed
prompt --key 0x197e --timeout 2000 Press F12 to investigate || exit
shell

We keep the upstream Debian image source and just add the pre-seed file to the boot command. The pre-seed file is located in the TFTP server in the preseed directory and is named microserver-preseed.cfg.

After softlinking the microserver.ipxe file to the HOSTNAME-....ipxe file, I was able to boot my HPE Microserver Gen8 and install Debian without any user interaction after a few tries and fixes to the ipxe file.

Conclusion

In the end the process for setting up my machine consists of the following steps and contains the following components: A netboot.xyz image, a pre-seed file and a custom iPXE script, which are together far under 1MB in size.

Boot the machine
The network card sends a DHCP request to the DHCP server and loads the iPXE script from the TFTP server
The iPXE script sends another DHCP request to the DHCP server and loads the custom Debian install script from the TFTP server
The Debian installer sends a DHCP request to the DHCP server and loads the pre-seed file from the TFTP server
The Debian installer installs the basic OS with the pre-seed file

After setting up the DHCP and TFTP server and providing a custom iPXE script, I was able to boot my HPE Microserver Gen8 and install Debian without any user interaction. This is a huge step forward in my homelab setup, as I am now able to just boot a machine and have it install the OS without any user interaction, in a repeatable and consistent way.

I probably won’t use this all the time, but now I can sleep better knowing that all machines are installed the same way.

References

* Photo by Siora Photography on Unsplash