From b847b87823dfd7e1b7870c67ea32199bc607f852 Mon Sep 17 00:00:00 2001 From: Kleidi Bujari Date: Wed, 18 Dec 2024 20:17:41 -0500 Subject: Enable github actions Moved personal blog to be built in this tree. Also enables github actions to host the site on pages. --- mod/web/blog/content/log/_index.md | 7 -- .../blog/content/log/deterministic-hostnames.md | 80 ------------------ mod/web/blog/content/log/nohup.md | 26 ------ mod/web/blog/content/posts/_index.md | 4 +- .../blog/content/posts/deterministic-hostnames.md | 80 ++++++++++++++++++ mod/web/blog/content/posts/nohup.md | 26 ++++++ mod/web/blog/static/avatar.png | Bin 1800 -> 0 bytes mod/web/blog/static/style.css | 85 ------------------- mod/web/blog/templates/base.html | 92 +++++++++++++++++---- mod/web/blog/templates/blog.html | 22 ----- mod/web/blog/templates/index.html | 52 +++--------- mod/web/blog/templates/post.html | 4 +- 12 files changed, 199 insertions(+), 279 deletions(-) delete mode 100644 mod/web/blog/content/log/_index.md delete mode 100644 mod/web/blog/content/log/deterministic-hostnames.md delete mode 100644 mod/web/blog/content/log/nohup.md create mode 100644 mod/web/blog/content/posts/deterministic-hostnames.md create mode 100644 mod/web/blog/content/posts/nohup.md delete mode 100644 mod/web/blog/static/avatar.png delete mode 100644 mod/web/blog/static/style.css delete mode 100644 mod/web/blog/templates/blog.html (limited to 'mod') diff --git a/mod/web/blog/content/log/_index.md b/mod/web/blog/content/log/_index.md deleted file mode 100644 index 10f2311..0000000 --- a/mod/web/blog/content/log/_index.md +++ /dev/null @@ -1,7 +0,0 @@ -+++ -title = "Log" -sort_by = "date" -template = "blog.html" -page_template = "post.html" -description = "notes too short to post" -+++ diff --git a/mod/web/blog/content/log/deterministic-hostnames.md b/mod/web/blog/content/log/deterministic-hostnames.md deleted file mode 100644 index 43dc7d8..0000000 --- a/mod/web/blog/content/log/deterministic-hostnames.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "Deterministic and unique network hostnames" -date: "2024-11-10" ---- - -As part of building out a Kubernetes cluster, I wanted to build and distribute a -single OS image to create stateless worker nodes. Using network booting, and -some clever tricks to differentiate nodes, we can create a scaleable and -efficient farm of workers for a cluster that don't even need disks. - -The idea came from a plan to build a cluster using the -[compute blade](https://computeblade.com/), and a few Raspberry Pi SBCs I -already own. Running the cluster from an SD card is not recommended due to the -not-so-great reliability of the flash used by most manufacturers, so I wanted to -try PXE booting each Pi to save money rather than purchasing an SSD for each -one. The compute blades do support an NVMe disk, but I plan to use those for a -storage cluster later, so they need to remain empty. - -## Base image - -Alpine Linux has been my preferred server OS for a long time. It provides a very -lightweight base system, and bundles an excellent bootstrapping system, -[apkovl](https://wiki.alpinelinux.org/wiki/Alpine_local_backup), that allows the -user to save a set of customisations to an system as an overlay to a stock -Alpine live image. In other words, we can create our image once, save the -changes as an `apkovl.tar.gz` file, and apply the same changes to a base system -on boot. This file can even be provided as a -[kernel parameter](https://wiki.alpinelinux.org/wiki/PXE_boot#Guide_to_options) -and will be fetched from a remote webserver automatically! - -Since the image and configuration will be shipped to the node via the network, -an added benefit of using Alpine is its tiny space consumption. I'm not using -enough nodes for this to really matter, but it's a cool optimization regardless. - -## Differentiating the nodes - -One of the main goals of this project is that there should be no persistent -storage required outside the boot image itself. Since every node will download -and generate the same root file-system on startup, the first problem that arises -is how the nodes will identify themselves both on the network and the cluster, -given that it's not possible to name them ahead of time. In other words, any -given node has to generate a unique hostname that won't collide with other -workers, and that will be the same each time that node boots. - -Since these nodes will not have a predefined name, we have to rely on -characteristics of the hardware to differentiate each one. The hardware MAC -address is perfect for this, since it's unique to to each node and will not be -wiped away after the node reboots. On a system like Linux that exposes its -hardware through a _sysfs_, we can find a file containing the address at -`/sys/class/net/eth0/address`. I don't really like the idea of attaching the -literal MAC address of the node to its network hostname, since it's a security -risk, and a bit too verbose. Instead, we can transform it into something safer -using a `sha1sum`, which is already present on our Alpine base system: - -```console -sha1sum /sys/class/net/eth0/address | head -c 6 | awk '{print "worker-" $0}' -``` - -### Applying the new name - -Ideally, the node should apply its generated hostname before reaching out for an -address over DHCP or joining the cluster. We can make sure it happens before any -traffic is sent out by adding a `pre-up` command to the right interface in -`/etc/network/interfaces`: - -``` - -... - -auto eth0 -iface eth0 inet dhcp - pre-up sha1sum /sys/class/net/eth0/address | head -c 6 | awk '{print "worker-" $0}' > /etc/hostname - -... -``` - -The VM I tested with looks outputs `worker-e2fae8`. Pretty clean result, and if -you want to know the physical node that maps to each hostname, you can take note -of the MAC address beforehand and generate the same hash on another computer to -match them up. diff --git a/mod/web/blog/content/log/nohup.md b/mod/web/blog/content/log/nohup.md deleted file mode 100644 index 64f7983..0000000 --- a/mod/web/blog/content/log/nohup.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: "Spawning background processes" -date: "2024-11-17" ---- - -Working in a terminal, -I often pair my editor with a background process watching files. -Before reaching for terminal multiplexers, -see if you can get away with simple tty job control. -Spawn the background process, -still attached to the terminal instance: - -``` -program args & -``` - -Also redirect its output to a file, -for when the process writes to the tty from the background: - -``` -nohup program args & -``` - -Extra reading: - -- diff --git a/mod/web/blog/content/posts/_index.md b/mod/web/blog/content/posts/_index.md index 1f4024a..9efc62e 100644 --- a/mod/web/blog/content/posts/_index.md +++ b/mod/web/blog/content/posts/_index.md @@ -1,8 +1,6 @@ +++ title = "Posts" sort_by = "date" -template = "blog.html" page_template = "post.html" +redirect_to="/" +++ - -Longer form writing. diff --git a/mod/web/blog/content/posts/deterministic-hostnames.md b/mod/web/blog/content/posts/deterministic-hostnames.md new file mode 100644 index 0000000..43dc7d8 --- /dev/null +++ b/mod/web/blog/content/posts/deterministic-hostnames.md @@ -0,0 +1,80 @@ +--- +title: "Deterministic and unique network hostnames" +date: "2024-11-10" +--- + +As part of building out a Kubernetes cluster, I wanted to build and distribute a +single OS image to create stateless worker nodes. Using network booting, and +some clever tricks to differentiate nodes, we can create a scaleable and +efficient farm of workers for a cluster that don't even need disks. + +The idea came from a plan to build a cluster using the +[compute blade](https://computeblade.com/), and a few Raspberry Pi SBCs I +already own. Running the cluster from an SD card is not recommended due to the +not-so-great reliability of the flash used by most manufacturers, so I wanted to +try PXE booting each Pi to save money rather than purchasing an SSD for each +one. The compute blades do support an NVMe disk, but I plan to use those for a +storage cluster later, so they need to remain empty. + +## Base image + +Alpine Linux has been my preferred server OS for a long time. It provides a very +lightweight base system, and bundles an excellent bootstrapping system, +[apkovl](https://wiki.alpinelinux.org/wiki/Alpine_local_backup), that allows the +user to save a set of customisations to an system as an overlay to a stock +Alpine live image. In other words, we can create our image once, save the +changes as an `apkovl.tar.gz` file, and apply the same changes to a base system +on boot. This file can even be provided as a +[kernel parameter](https://wiki.alpinelinux.org/wiki/PXE_boot#Guide_to_options) +and will be fetched from a remote webserver automatically! + +Since the image and configuration will be shipped to the node via the network, +an added benefit of using Alpine is its tiny space consumption. I'm not using +enough nodes for this to really matter, but it's a cool optimization regardless. + +## Differentiating the nodes + +One of the main goals of this project is that there should be no persistent +storage required outside the boot image itself. Since every node will download +and generate the same root file-system on startup, the first problem that arises +is how the nodes will identify themselves both on the network and the cluster, +given that it's not possible to name them ahead of time. In other words, any +given node has to generate a unique hostname that won't collide with other +workers, and that will be the same each time that node boots. + +Since these nodes will not have a predefined name, we have to rely on +characteristics of the hardware to differentiate each one. The hardware MAC +address is perfect for this, since it's unique to to each node and will not be +wiped away after the node reboots. On a system like Linux that exposes its +hardware through a _sysfs_, we can find a file containing the address at +`/sys/class/net/eth0/address`. I don't really like the idea of attaching the +literal MAC address of the node to its network hostname, since it's a security +risk, and a bit too verbose. Instead, we can transform it into something safer +using a `sha1sum`, which is already present on our Alpine base system: + +```console +sha1sum /sys/class/net/eth0/address | head -c 6 | awk '{print "worker-" $0}' +``` + +### Applying the new name + +Ideally, the node should apply its generated hostname before reaching out for an +address over DHCP or joining the cluster. We can make sure it happens before any +traffic is sent out by adding a `pre-up` command to the right interface in +`/etc/network/interfaces`: + +``` + +... + +auto eth0 +iface eth0 inet dhcp + pre-up sha1sum /sys/class/net/eth0/address | head -c 6 | awk '{print "worker-" $0}' > /etc/hostname + +... +``` + +The VM I tested with looks outputs `worker-e2fae8`. Pretty clean result, and if +you want to know the physical node that maps to each hostname, you can take note +of the MAC address beforehand and generate the same hash on another computer to +match them up. diff --git a/mod/web/blog/content/posts/nohup.md b/mod/web/blog/content/posts/nohup.md new file mode 100644 index 0000000..64f7983 --- /dev/null +++ b/mod/web/blog/content/posts/nohup.md @@ -0,0 +1,26 @@ +--- +title: "Spawning background processes" +date: "2024-11-17" +--- + +Working in a terminal, +I often pair my editor with a background process watching files. +Before reaching for terminal multiplexers, +see if you can get away with simple tty job control. +Spawn the background process, +still attached to the terminal instance: + +``` +program args & +``` + +Also redirect its output to a file, +for when the process writes to the tty from the background: + +``` +nohup program args & +``` + +Extra reading: + +- diff --git a/mod/web/blog/static/avatar.png b/mod/web/blog/static/avatar.png deleted file mode 100644 index 91357dc..0000000 Binary files a/mod/web/blog/static/avatar.png and /dev/null differ diff --git a/mod/web/blog/static/style.css b/mod/web/blog/static/style.css deleted file mode 100644 index fdfd692..0000000 --- a/mod/web/blog/static/style.css +++ /dev/null @@ -1,85 +0,0 @@ -:root { - --bg: #181616; - --fg: #c5c9c5; - --link: #76946a; - --linkhover: #98bb6c; -} - -html { - scroll-behavior: smooth; - color-scheme: dark; -} - -body { - font-family: serif; - padding: 0 0.75em; - max-width: 42em; - margin: auto; - background: var(--bg); - color: var(--fg); -} - -p { - line-height: 1.3em; -} - -th, -td { - padding: 0.2em 0.4em; - border: thin solid; -} - -table { - border: thin solid; - border-collapse: collapse; -} - -article { - padding-top: 1em; -} - -footer { - margin: 3em auto; - text-align: center; -} - -a, -a:link { - color: var(--link); -} - -a:hover { - color: var(--linkhover); -} - -a[href^="http"]:where(:not([href*="4kb.net/"]))::after { - content: " \21e2"; -} - -article img { - display: block; - margin: 0 auto; - max-width: 80%; - height: auto; - object-fit: contain; -} - -nav { - margin-top: 1em; - display: flex; - justify-content: space-between; - align-items: center; - font-family: monospace; - font-weight: bold; - gap: 8px; -} - -nav div { - display: flex; - gap: 10px; -} - -pre { - padding: 1em; - overflow-x: scroll; -} diff --git a/mod/web/blog/templates/base.html b/mod/web/blog/templates/base.html index 6757d02..5a91df5 100644 --- a/mod/web/blog/templates/base.html +++ b/mod/web/blog/templates/base.html @@ -3,24 +3,86 @@ - - {% block title %}4kb.net{% endblock title %} + - -
+
+ Kleidi Bujari <mail@4kb.net> + +
{% block content %} {% endblock %}
-
-
- © 2024 Kleidi Bujari -
diff --git a/mod/web/blog/templates/blog.html b/mod/web/blog/templates/blog.html deleted file mode 100644 index 6c92a07..0000000 --- a/mod/web/blog/templates/blog.html +++ /dev/null @@ -1,22 +0,0 @@ -{% extends "base.html" %} - -{% block content %} - - - 4kb.net - - -

{{ section.title }}

- -{{ section.content | safe }} - -
    - {% for page in section.pages %} -
  • - [{{ page.date }}] - {{ page.title }} -
    • - {% endfor %} -
- -{% endblock content %} diff --git a/mod/web/blog/templates/index.html b/mod/web/blog/templates/index.html index 74e9a0f..6e8a2af 100644 --- a/mod/web/blog/templates/index.html +++ b/mod/web/blog/templates/index.html @@ -1,44 +1,16 @@ {% extends "base.html" %} {% block content %} -

welcome!

- -

- I'm a computer engineering undergrad studying in Toronto. This is my personal - site where I sometimes publish long form posts on topics I'm interested in, or - short "logs" for information I want to remember. -

- -Find me on: - - - - - - - - - - - - - - - -

Recent Logs

- -{% set section = get_section(path="log/_index.md") %} - -
    - {% for page in section.pages | slice(end=4) %} -
  • - [{{ page.date }}] - {{ page.title }} +
    + {% set section = get_section(path="posts/_index.md") %} + +
      + {% for page in section.pages %} +
    • + {{ page.date }} + {{ page.title }}
      • - {% endfor %} -
    - + {% endfor %} +
-{% endblock content %} + {% endblock content %} + diff --git a/mod/web/blog/templates/post.html b/mod/web/blog/templates/post.html index b1981e4..b819520 100644 --- a/mod/web/blog/templates/post.html +++ b/mod/web/blog/templates/post.html @@ -1,12 +1,14 @@ {% extends "base.html" %} + {% block title%} {{ page.title }} - {{ super() }} {% endblock title %} {% block content %} -

{{ page.title }}

+

{{ page.title }}

{{ page.content | safe }}
+ {% endblock content %} -- cgit v1.3.1