From eb13a9a552f9e50a66faba985521ba8e3483c447 Mon Sep 17 00:00:00 2001 From: Kleidi Bujari Date: Tue, 17 Dec 2024 23:03:27 -0500 Subject: Push configuration out of flake Projects are not necessarily all exported by the system flake, and should freely support other derivations. The new "mod" directory will be used for most projects, and it can be imported by machine configurations. That said, the tooling around flakes is pretty good and will still be used until the dependency is lessened. --- mod/web/blog/content/log/_index.md | 7 ++ .../blog/content/log/deterministic-hostnames.md | 80 ++++++++++++++++++++++ mod/web/blog/content/log/nohup.md | 26 +++++++ 3 files changed, 113 insertions(+) create mode 100644 mod/web/blog/content/log/_index.md create mode 100644 mod/web/blog/content/log/deterministic-hostnames.md create mode 100644 mod/web/blog/content/log/nohup.md (limited to 'mod/web/blog/content/log') diff --git a/mod/web/blog/content/log/_index.md b/mod/web/blog/content/log/_index.md new file mode 100644 index 0000000..10f2311 --- /dev/null +++ b/mod/web/blog/content/log/_index.md @@ -0,0 +1,7 @@ ++++ +title = "Log" +sort_by = "date" +template = "blog.html" +page_template = "post.html" +description = "notes too short to post" ++++ diff --git a/mod/web/blog/content/log/deterministic-hostnames.md b/mod/web/blog/content/log/deterministic-hostnames.md new file mode 100644 index 0000000..43dc7d8 --- /dev/null +++ b/mod/web/blog/content/log/deterministic-hostnames.md @@ -0,0 +1,80 @@ +--- +title: "Deterministic and unique network hostnames" +date: "2024-11-10" +--- + +As part of building out a Kubernetes cluster, I wanted to build and distribute a +single OS image to create stateless worker nodes. Using network booting, and +some clever tricks to differentiate nodes, we can create a scaleable and +efficient farm of workers for a cluster that don't even need disks. + +The idea came from a plan to build a cluster using the +[compute blade](https://computeblade.com/), and a few Raspberry Pi SBCs I +already own. Running the cluster from an SD card is not recommended due to the +not-so-great reliability of the flash used by most manufacturers, so I wanted to +try PXE booting each Pi to save money rather than purchasing an SSD for each +one. The compute blades do support an NVMe disk, but I plan to use those for a +storage cluster later, so they need to remain empty. + +## Base image + +Alpine Linux has been my preferred server OS for a long time. It provides a very +lightweight base system, and bundles an excellent bootstrapping system, +[apkovl](https://wiki.alpinelinux.org/wiki/Alpine_local_backup), that allows the +user to save a set of customisations to an system as an overlay to a stock +Alpine live image. In other words, we can create our image once, save the +changes as an `apkovl.tar.gz` file, and apply the same changes to a base system +on boot. This file can even be provided as a +[kernel parameter](https://wiki.alpinelinux.org/wiki/PXE_boot#Guide_to_options) +and will be fetched from a remote webserver automatically! + +Since the image and configuration will be shipped to the node via the network, +an added benefit of using Alpine is its tiny space consumption. I'm not using +enough nodes for this to really matter, but it's a cool optimization regardless. + +## Differentiating the nodes + +One of the main goals of this project is that there should be no persistent +storage required outside the boot image itself. Since every node will download +and generate the same root file-system on startup, the first problem that arises +is how the nodes will identify themselves both on the network and the cluster, +given that it's not possible to name them ahead of time. In other words, any +given node has to generate a unique hostname that won't collide with other +workers, and that will be the same each time that node boots. + +Since these nodes will not have a predefined name, we have to rely on +characteristics of the hardware to differentiate each one. The hardware MAC +address is perfect for this, since it's unique to to each node and will not be +wiped away after the node reboots. On a system like Linux that exposes its +hardware through a _sysfs_, we can find a file containing the address at +`/sys/class/net/eth0/address`. I don't really like the idea of attaching the +literal MAC address of the node to its network hostname, since it's a security +risk, and a bit too verbose. Instead, we can transform it into something safer +using a `sha1sum`, which is already present on our Alpine base system: + +```console +sha1sum /sys/class/net/eth0/address | head -c 6 | awk '{print "worker-" $0}' +``` + +### Applying the new name + +Ideally, the node should apply its generated hostname before reaching out for an +address over DHCP or joining the cluster. We can make sure it happens before any +traffic is sent out by adding a `pre-up` command to the right interface in +`/etc/network/interfaces`: + +``` + +... + +auto eth0 +iface eth0 inet dhcp + pre-up sha1sum /sys/class/net/eth0/address | head -c 6 | awk '{print "worker-" $0}' > /etc/hostname + +... +``` + +The VM I tested with looks outputs `worker-e2fae8`. Pretty clean result, and if +you want to know the physical node that maps to each hostname, you can take note +of the MAC address beforehand and generate the same hash on another computer to +match them up. diff --git a/mod/web/blog/content/log/nohup.md b/mod/web/blog/content/log/nohup.md new file mode 100644 index 0000000..64f7983 --- /dev/null +++ b/mod/web/blog/content/log/nohup.md @@ -0,0 +1,26 @@ +--- +title: "Spawning background processes" +date: "2024-11-17" +--- + +Working in a terminal, +I often pair my editor with a background process watching files. +Before reaching for terminal multiplexers, +see if you can get away with simple tty job control. +Spawn the background process, +still attached to the terminal instance: + +``` +program args & +``` + +Also redirect its output to a file, +for when the process writes to the tty from the background: + +``` +nohup program args & +``` + +Extra reading: + +- -- cgit v1.3.1