diff options
| -rw-r--r-- | about/podcasts.txt | 1 | ||||
| -rw-r--r-- | gemfeed/4 | 239 | ||||
| -rw-r--r-- | gemfeed/DRAFT-f3s-kubernetes-with-freebsd-bhyve.html | 242 | ||||
| -rw-r--r-- | gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-3.html | 421 | ||||
| -rw-r--r-- | gemfeed/f3s-kubernetes-with-freebsd-part-3/apc-back-ups.jpg | bin | 0 -> 295558 bytes | |||
| -rw-r--r-- | gemfeed/f3s-kubernetes-with-freebsd-part-3/f3s-changes.jpg | bin | 0 -> 54543 bytes |
6 files changed, 661 insertions, 242 deletions
diff --git a/about/podcasts.txt b/about/podcasts.txt index 19e38523..182d02cc 100644 --- a/about/podcasts.txt +++ b/about/podcasts.txt @@ -9,3 +9,4 @@ * Maintainable * Fork Around And Find Out * The ProdCast (Google SRE Podcast) +* BSD Now diff --git a/gemfeed/4 b/gemfeed/4 new file mode 100644 index 00000000..f4c59789 --- /dev/null +++ b/gemfeed/4 @@ -0,0 +1,239 @@ +# f3s: Kubernetes with FreeBSD - Rocky Linux Bhyve VMs - Part 3 + +This is the third blog post about my f3s series for my self-hosting demands in my home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution we will use on FreeBSD-based physical machines. + +<< template::inline::index f3s-kubernetes-with-freebsd-part + +=> ./f3s-kubernetes-with-frhyveeebsd-part-1/f3slogo.png f3s logo + +<< template::inline::toc + +## Introduction + +In this blog post, we are going to install the Bhyve hypervisor. + +The FreeBSD Bhyve hypervisor is a lightweight, modern hypervisor that enables virtualization on FreeBSD systems. Bhyve's strengths include its minimal overhead, which allows it to achieve near-native performance for virtual machines. It is designed to be efficient and lightweight, leveraging the capabilities of the FreeBSD operating system for performance and network management. + +Bhyve supports running a variety of guest operating systems, including FreeBSD, Linux, and Windows, on hardware platforms that support hardware virtualization extensions (such as Intel VT-x or AMD-V). In our case, we are going to virtualize Rocky Linux, which later on in this series will be used to run k3s. + +## Basic Bhyve setup + +For the management of the Bhyve VMs, we are using `vm-bhyve`, a tool not part of the FreeBSD operating system but available as a ready-to-use package. It eases VM management and reduces a lot of the overhead. We also install the required package to make Bhyve work with the UEFI firmware. + +=> https://github.com/churchers/vm-bhyve + +The following commands are executed on all three hosts `f0`, `f1`, and `f2`, where `re0` is the name of the Ethernet interface (which may need to be adjusted if your hardware is different): + +```sh +paul@f0:~ % doas pkg install vm-bhyve bhyve-firmware +paul@f0:~ % doas sysrc vm_enable=YES +vm_enable: -> YES +paul@f0:~ % doas sysrc vm_dir=zfs:zroot/bhyve +vm_dir: -> zfs:zroot/bhyve +paul@f0:~ % doas zfs create zroot/bhyve +paul@f0:~ % doas vm init +paul@f0:~ % doas vm switch create public +paul@f0:~ % doas vm switch add public re0 +``` + +Bhyve stores all it's data in the `/bhyve` of the `zroot` ZFS pool: + +```sh +paul@f0:~ % zfs list | grep bhyve +zroot/bhyve 1.74M 453G 1.74M /zroot/bhyve +``` + +For convenience, we also create this symlink: + +```sh +paul@f0:~ % doas ln -s /zroot/bhyve/ /bhyve + +``` + +Now, Bhyve is ready to rumble, but no VMs are there yet: + +```sh +paul@f0:~ % doas vm list +NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE +``` + +## Rocky Linux VMs + +### ISO download + +We're going to install the Rocky Linux from the latest minimal iso: + +```sh +paul@f0:~ % doas vm iso \ + https://download.rockylinux.org/pub/rocky/9/isos/x86_64/Rocky-9.5-x86_64-minimal.iso +/zroot/bhyve/.iso/Rocky-9.5-x86_64-minimal.iso 1808 MB 4780 kBps 06m28s +paul@f0:/bhyve % doas vm create rocky +``` +### VM configuration + +The default configuration looks like this now: + +```sh +paul@f0:/bhyve/rocky % cat rocky.conf +loader="bhyveload" +cpu=1 +memory=256M +network0_type="virtio-net" +network0_switch="public" +disk0_type="virtio-blk" +disk0_name="disk0.img" +uuid="1c4655ac-c828-11ef-a920-e8ff1ed71ca0" +network0_mac="58:9c:fc:0d:13:3f" +``` + +Whereas the `uuid` and the `network0_mac` differ on each of the 3 hosts. + +but in order to make Rocky Linux boot it (plus some other adjustments, e.g. as I am intending to run the majority of the workload in the k3s cluster running on those linux VMs, I give them beefy specs like 4 CPU cores and 14GB RAM), I run `doas vm configure rocky` and modified it to: + +``` +guest="linux" +loader="uefi" +uefi_vars="yes" +cpu=4 +memory=14G +network0_type="virtio-net" +network0_switch="public" +disk0_type="virtio-blk" +disk0_name="disk0.img" +graphics="yes" +graphics_vga=io +uuid="1c45400b-c828-11ef-8871-e8ff1ed71cac" +network0_mac="58:9c:fc:0d:13:3f" +``` + +### VM installation + +To start the installer from the downloaded ISO, I run: + +```sh +paul@f0:~ % doas vm install rocky Rocky-9.5-x86_64-minimal.iso +Starting rocky + * found guest in /zroot/bhyve/rocky + * booting... + +paul@f0:/bhyve/rocky % doas vm list +NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE +rocky default uefi 4 14G 0.0.0.0:5900 No Locked (f0.lan.buetow.org) + +paul@f0:/bhyve/rocky % doas sockstat -4 | grep 5900 +root bhyve 6079 8 tcp4 *:5900 *:* +``` + +Port 5900 now also opened for VNC connections, so I connected to it with a VNC client and run through the installation dialogs. I'm sure this could be done unattended or more automated, there are only 3 VMs to install, and the automation doesn't seem worth it as we are doing it only once in a year or less often. + +### Increase of the disk image + +By default the VMs disk image is only 20G, which is a bit small for my purposes, so I stopped the VMs again and run `truncate` on the image file to enlarge them to 100G, and re-started the installation: + +```sh +paul@f0:/bhyve/rocky % doas vm stop rocky +paul@f0:/bhyve/rocky % doas truncate -s 100G disk0.img +paul@f0:/bhyve/rocky % doas vm install rocky Rocky-9.5-x86_64-minimal.iso +``` + +### Connect to VPN + +For the installation, I opened the VPN client on my Fedora laptop (GNOME comes with a simple VPN client) and ran through the base installation for each of the VMs manually. Again, I am sure this could have been automated a bit more, but there were just 3 VMs, and it wasn't worth the effort. The three VNC addresses of the VMs were: `vnc://f0:5900`, `vnc://f1:5900`, and `vnc://f0:5900`. + +I mostly selected the default settings (auto partitioning on the 100GB drive and a root user password). After the installation, the VMs were rebooted. + +## After install + +I performed the following steps for all 3 VMs. In the following, the examples are all executed on `f0` (bzw the VM `r0` running on `f0`): + +### VM auto-start after host reboot + +To automatically start the VM on the servers I added the following to the `rc.conf` on the FreeBSD hosts: + +```sh + +paul@f0:/bhyve/rocky % cat <<END | doas tee -a /etc/rc.conf +vm_list="rocky" +vm_delay="5" +``` + +The `vm_delay` isn't really required. It is used to wait 5 seconds before starting each VM, but as of now, there is only one VM per host. Maybe later, when there are more, this will be useful to have. After adding, there's now a `Yes` indicator in the `AUTO` column. + +```sh +paul@f0:~ % doas vm list +NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE +rocky default uefi 4 14G 0.0.0.0:5900 Yes [1] Running (2063) +``` + +### Static IP configuration + +After that, I changed the network configuration of the VMs to be static (from DHCP) here. As per previous post of this series, the 3 FreeBSD hosts were already in my `/etc/hosts` file: + +``` +192.168.1.130 f0 f0.lan f0.lan.buetow.org +192.168.1.131 f1 f1.lan f1.lan.buetow.org +192.168.1.132 f2 f2.lan f2.lan.buetow.org +``` + +For the Rocky VMs I added those to the FreeBSD hosts systems as well: + +```sh +paul@f0:/bhyve/rocky % cat <<END | doas tee -a /etc/hosts +192.168.1.120 r0 r0.lan r0.lan.buetow.org +192.168.1.121 r1 r1.lan r1.lan.buetow.org +192.168.1.122 r2 r2.lan r2.lan.buetow.org +END +``` + +and configured the IPs accordingly on the VMs themselves by opening a root shell via RDP to the VMs and entering the following commands on each of the VMs: + +```sh +[root@r0 ~] % dnmcli connection modify enp0s5 ipv4.address 192.168.1.120/24 +[root@r0 ~] % dnmcli connection modify enp0s5 ipv4.gateway 192.168.1.1 +[root@r0 ~] % dnmcli connection modify enp0s5 ipv4.dns 192.168.1.1 +[root@r0 ~] % dnmcli connection modify enp0s5 ipv4.method manual +[root@r0 ~] % dnmcli connection down enp0s5 +[root@r0 ~] % dnmcli connection up enp0s5 +[root@r0 ~] % hostnamectl set-hostname r0.lan.buetow.org +[root@r0 ~] % cat <<END >>/etc/hosts +192.168.1.120 r0 r0.lan r0.lan.buetow.org +192.168.1.121 r1 r1.lan r1.lan.buetow.org +192.168.1.122 r2 r2.lan r2.lan.buetow.org +END +```` + +Whereas: + +* `192.168.1.120` is the IP of the VM itself (here: `r0.lan.buetow.org`) +* `192.168.1.1` is the address of my home router, which also does DNS. + +### Permitting root login + +As these VMs arent directly reachable via SSH from the internet, I enabled `root` login by adding a line with `PermitRootLogin yes` to `/etc/sshd/sshd_config`. + +Once done, I rebooted the VM by running `reboot` inside of the vm to test whether everything was configured and persisted correctly. + +After reboot, I copied my public key from my Laptop to the 3 VMs: + +```sh +% for i in 0 1 2; do ssh-copy-id root@r$i.lan.buetow.org; done +``` + +And then I edited the `/etc/ssh/sshd_config` file again on all 3 VMs and configured `PasswordAuthentication no`, to only allow SSH key authentication from now on. + +### Install latest updates + +```sh +[root@r0 ~] % dnf update +[root@r0 ~] % dreboot +``` + +CPU STRESS TESTER VM VS NOT VM + +Other *BSD-related posts: + +<< template::inline::index bsd + +E-Mail your comments to `paul@nospam.buetow.org` :-) + +=> ../ Back to the main site diff --git a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-bhyve.html b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-bhyve.html deleted file mode 100644 index 23c891cc..00000000 --- a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-bhyve.html +++ /dev/null @@ -1,242 +0,0 @@ -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> -<head> -<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> -<title>f3s: Kubernetes with FreeBSD - Rocky Linux Bhyve VMs - Part 3</title> -<link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> -<link rel="stylesheet" href="../style.css" /> -<link rel="stylesheet" href="style-override.css" /> -</head> -<body> -<p class="header"> -View this page as <a href="https://codeberg.org/snonux/foo.zone/src/branch/content-md/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-bhyve.md">Markdown</a> | <a href="gemini://foo.zone/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-bhyve.gmi">Gemini</a> -</p> -<h1 style='display: inline' id='f3s-kubernetes-with-freebsd---rocky-linux-bhyve-vms---part-3'>f3s: Kubernetes with FreeBSD - Rocky Linux Bhyve VMs - Part 3</h1><br /> -<br /> -<span>This is the third blog post about my f3s series for my self-hosting demands in my home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution we will use on FreeBSD-based physical machines.</span><br /> -<br /> -<a class='textlink' href='./2024-11-17-f3s-kubernetes-with-freebsd-part-1.html'>2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage</a><br /> -<a class='textlink' href='./2024-12-03-f3s-kubernetes-with-freebsd-part-2.html'>2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation</a><br /> -<br /> -<a href='./f3s-kubernetes-with-frhyveeebsd-part-1/f3slogo.png'><img alt='f3s logo' title='f3s logo' src='./f3s-kubernetes-with-frhyveeebsd-part-1/f3slogo.png' /></a><br /> -<br /> -<h2 style='display: inline' id='table-of-contents'>Table of Contents</h2><br /> -<br /> -<ul> -<li><a href='#f3s-kubernetes-with-freebsd---rocky-linux-bhyve-vms---part-3'>f3s: Kubernetes with FreeBSD - Rocky Linux Bhyve VMs - Part 3</a></li> -<li>⇢ <a href='#introduction'>Introduction</a></li> -<li>⇢ <a href='#basic-bhyve-setup'>Basic Bhyve setup</a></li> -<li>⇢ <a href='#rocky-linux-vms'>Rocky Linux VMs</a></li> -<li>⇢ ⇢ <a href='#iso-download'>ISO download</a></li> -<li>⇢ ⇢ <a href='#vm-configuration'>VM configuration</a></li> -<li>⇢ ⇢ <a href='#vm-installation'>VM installation</a></li> -<li>⇢ ⇢ <a href='#increase-of-the-disk-image'>Increase of the disk image</a></li> -<li>⇢ ⇢ <a href='#connect-to-vpn'>Connect to VPN</a></li> -<li>⇢ <a href='#after-install'>After install</a></li> -</ul><br /> -<h2 style='display: inline' id='introduction'>Introduction</h2><br /> -<br /> -<span>In this blog post, we are going to install the Bhyve hypervisor.</span><br /> -<br /> -<span>The FreeBSD Bhyve hypervisor is a lightweight, modern hypervisor that enables virtualization on FreeBSD systems. Bhyve's strengths include its minimal overhead, which allows it to achieve near-native performance for virtual machines. It is designed to be efficient and lightweight, leveraging the capabilities of the FreeBSD operating system for performance and network management.</span><br /> -<br /> -<span>Bhyve supports running a variety of guest operating systems, including FreeBSD, Linux, and Windows, on hardware platforms that support hardware virtualization extensions (such as Intel VT-x or AMD-V). In our case, we are going to virtualize Rocky Linux, which later on in this series will be used to run k3s.</span><br /> -<br /> -<h2 style='display: inline' id='basic-bhyve-setup'>Basic Bhyve setup</h2><br /> -<br /> -<span>For the management of the Bhyve VMs, we are using <span class='inlinecode'>vm-bhyve</span>, a tool not part of the FreeBSD operating system but available as a ready-to-use package. It eases VM management and reduces a lot of the overhead. We also install the required package to make Bhyve work with the UEFI firmware.</span><br /> -<br /> -<a class='textlink' href='https://github.com/churchers/vm-bhyve'>https://github.com/churchers/vm-bhyve</a><br /> -<br /> -<span>The following commands are executed on all three hosts <span class='inlinecode'>f0</span>, <span class='inlinecode'>f1</span>, and <span class='inlinecode'>f2</span>, where <span class='inlinecode'>re0</span> is the name of the Ethernet interface (which may need to be adjusted if your hardware is different):</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>paul@f0:~ % doas pkg install vm-bhyve bhyve-firmware -paul@f0:~ % doas sysrc vm_enable=YES -vm_enable: -> YES -paul@f0:~ % doas sysrc vm_dir=zfs:zroot/bhyve -vm_dir: -> zfs:zroot/bhyve -paul@f0:~ % doas zfs create zroot/bhyve -paul@f0:~ % doas vm init -paul@f0:~ % doas vm switch create public -paul@f0:~ % doas vm switch add public re0 -</pre> -<br /> -<span>Bhyve stores all it's data in the <span class='inlinecode'>/bhyve</span> of the <span class='inlinecode'>zroot</span> ZFS pool:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>paul@f0:~ % zfs list | grep bhyve -zroot/bhyve <font color="#000000">1</font>.74M 453G <font color="#000000">1</font>.74M /zroot/bhyve -</pre> -<br /> -<span>For convenience, we also create this symlink:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>paul@f0:~ % doas ln -s /zroot/bhyve/ /bhyve - -</pre> -<br /> -<span>Now, Bhyve is ready to rumble, but no VMs are there yet:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>paul@f0:~ % doas vm list -NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE -</pre> -<br /> -<h2 style='display: inline' id='rocky-linux-vms'>Rocky Linux VMs</h2><br /> -<br /> -<h3 style='display: inline' id='iso-download'>ISO download</h3><br /> -<br /> -<span>We're going to install the Rocky Linux from the latest minimal iso:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>paul@f0:~ % doas vm iso \ - https://download.rockylinux.org/pub/rocky/<font color="#000000">9</font>/isos/x86_64/Rocky-<font color="#000000">9.5</font>-x86_64-minimal.iso -/zroot/bhyve/.iso/Rocky-<font color="#000000">9.5</font>-x86_64-minimal.iso <font color="#000000">1808</font> MB <font color="#000000">4780</font> kBps 06m28s -paul@f0:/bhyve % doas vm create rocky -</pre> -<h3 style='display: inline' id='vm-configuration'>VM configuration</h3><br /> -<br /> -<span>The default configuration looks like this now:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>paul@f0:/bhyve/rocky % cat rocky.conf -loader=<font color="#808080">"bhyveload"</font> -cpu=<font color="#000000">1</font> -memory=256M -network0_type=<font color="#808080">"virtio-net"</font> -network0_switch=<font color="#808080">"public"</font> -disk0_type=<font color="#808080">"virtio-blk"</font> -disk0_name=<font color="#808080">"disk0.img"</font> -uuid=<font color="#808080">"1c4655ac-c828-11ef-a920-e8ff1ed71ca0"</font> -network0_mac=<font color="#808080">"58:9c:fc:0d:13:3f"</font> -</pre> -<br /> -<span>Whereas the <span class='inlinecode'>uuid</span> and the <span class='inlinecode'>network0_mac</span> differ on each of the 3 hosts.</span><br /> -<br /> -<span>but in order to make Rocky Linux boot it (plus some other adjustments, e.g. as I am intending to run the majority of the workload in the k3s cluster running on those linux VMs, I give them beefy specs like 4 CPU cores and 14GB RAM), I modified it to:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>guest=<font color="#808080">"linux"</font> -loader=<font color="#808080">"uefi"</font> -uefi_vars=<font color="#808080">"yes"</font> -cpu=<font color="#000000">4</font> -memory=14G -network0_type=<font color="#808080">"virtio-net"</font> -network0_switch=<font color="#808080">"public"</font> -disk0_type=<font color="#808080">"virtio-blk"</font> -disk0_name=<font color="#808080">"disk0.img"</font> -graphics=<font color="#808080">"yes"</font> -graphics_vga=io -uuid=<font color="#808080">"1c45400b-c828-11ef-8871-e8ff1ed71cac"</font> -network0_mac=<font color="#808080">"58:9c:fc:0d:13:3f"</font> -</pre> -<br /> -<h3 style='display: inline' id='vm-installation'>VM installation</h3><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>paul@f0:~ % doas vm install rocky Rocky-<font color="#000000">9.5</font>-x86_64-minimal.iso -Starting rocky - * found guest <b><u><font color="#000000">in</font></u></b> /zroot/bhyve/rocky - * booting... - -paul@f0:/bhyve/rocky % doas vm list -NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE -rocky default uefi <font color="#000000">4</font> 14G <font color="#000000">0.0</font>.<font color="#000000">0.0</font>:<font color="#000000">5900</font> No Locked (f0.lan.buetow.org) - -paul@f0:/bhyve/rocky % doas sockstat -<font color="#000000">4</font> | grep <font color="#000000">5900</font> -root bhyve <font color="#000000">6079</font> <font color="#000000">8</font> tcp4 *:<font color="#000000">5900</font> *:* -</pre> -<br /> -<span>Port 5900 is now also open for VNC connections, so we connect to it with a VNC client and run through the installation dialogs. I'm sure this could be done unattended or more automated, but we have only 3 VMs to install, and the automation doesn't seem worth it as we are doing it only once.</span><br /> -<br /> -<h3 style='display: inline' id='increase-of-the-disk-image'>Increase of the disk image</h3><br /> -<br /> -<span>By default the VMs disk image is only 20G, which is a bit small for my purposes, so I stopped the VMs again and run <span class='inlinecode'>truncate</span> on the image file to enlarge them to 100G, and re-started the installation:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>paul@f0:/bhyve/rocky % doas vm stop rocky -paul@f0:/bhyve/rocky % doas truncate -s 100G disk0.img -paul@f0:/bhyve/rocky % doas vm install rocky Rocky-<font color="#000000">9.5</font>-x86_64-minimal.iso -</pre> -<br /> -<h3 style='display: inline' id='connect-to-vpn'>Connect to VPN</h3><br /> -<br /> -<span>For the installation, I opened the VPN client on my Fedora laptop (GNOME comes with a simple VPN client) and ran through the base installation for each of the VMs manually. I am sure this could have been automated a bit more, but there were just 3 VMs, and it wasn't worth the effort. The three VNC addresses of the VMs were: <span class='inlinecode'>vnc://f0:5900</span>, <span class='inlinecode'>vnc://f1:5900</span>, and <span class='inlinecode'>vnc://f0:5900</span>.</span><br /> -<br /> -<span>I mostly selected the default settings (auto partitioning on the 100GB drive and a root user password). After the installation, the VMs were rebooted.</span><br /> -<br /> -<h2 style='display: inline' id='after-install'>After install</h2><br /> -<br /> -<span>After that, I changed the network configuration to be static here as well.</span><br /> -<br /> -<span>As per previous post of this series, the 3 FreeBSD hosts were already in my <span class='inlinecode'>/etc/hosts</span> file:</span><br /> -<br /> -<pre> -192.168.1.130 f0 f0.lan f0.lan.buetow.org -192.168.1.131 f1 f1.lan f1.lan.buetow.org -192.168.1.132 f2 f2.lan f2.lan.buetow.org -</pre> -<br /> -<span>For the Rocky VMs I added those:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>cat <<END >>/etc/hosts -<font color="#000000">192.168</font>.<font color="#000000">1.120</font> r0 r0.lan r0.lan.buetow.org -<font color="#000000">192.168</font>.<font color="#000000">1.121</font> r1 r1.lan r1.lan.buetow.org -<font color="#000000">192.168</font>.<font color="#000000">1.122</font> r2 r2.lan r2.lan.buetow.org -END -</pre> -<span>and configured the IPs accordingly on the VMs themselves.</span><br /> -<br /> -<br /> -<span>Other *BSD-related posts:</span><br /> -<br /> -<a class='textlink' href='./2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html'>2016-04-09 Jails and ZFS with Puppet on FreeBSD</a><br /> -<a class='textlink' href='./2022-07-30-lets-encrypt-with-openbsd-and-rex.html'>2022-07-30 Let's Encrypt with OpenBSD and Rex</a><br /> -<a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD</a><br /> -<a class='textlink' href='./2024-01-13-one-reason-why-i-love-openbsd.html'>2024-01-13 One reason why I love OpenBSD</a><br /> -<a class='textlink' href='./2024-04-01-KISS-high-availability-with-OpenBSD.html'>2024-04-01 KISS high-availability with OpenBSD</a><br /> -<a class='textlink' href='./2024-11-17-f3s-kubernetes-with-freebsd-part-1.html'>2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage</a><br /> -<a class='textlink' href='./2024-12-03-f3s-kubernetes-with-freebsd-part-2.html'>2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>paul@nospam.buetow.org</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> -<p class="footer"> -Generated with <a href="https://codeberg.org/snonux/gemtexter">Gemtexter 3.0.1-develop</a> | -served by <a href="https://www.OpenBSD.org">OpenBSD</a>/<a href="https://man.openbsd.org/relayd.8">relayd(8)</a>+<a href="https://man.openbsd.org/httpd.8">httpd(8)</a> | -<a href="https://foo.zone/site-mirrors.html">Site Mirrors</a> -</p> -</body> -</html> diff --git a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-3.html b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-3.html new file mode 100644 index 00000000..155a5542 --- /dev/null +++ b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-3.html @@ -0,0 +1,421 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> +<head> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<title>f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts</title> +<link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> +<link rel="stylesheet" href="../style.css" /> +<link rel="stylesheet" href="style-override.css" /> +</head> +<body> +<p class="header"> +View this page as <a href="https://codeberg.org/snonux/foo.zone/src/branch/content-md/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-3.md">Markdown</a> | <a href="gemini://foo.zone/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-3.gmi">Gemini</a> +</p> +<h1 style='display: inline' id='f3s-kubernetes-with-freebsd---part-3-protecting-from-power-cuts'>f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts</h1><br /> +<br /> +<span>This is the third blog post about my f3s series for my self-hosting demands in my home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution we will use on FreeBSD-based physical machines.</span><br /> +<br /> +<a class='textlink' href='./2024-11-17-f3s-kubernetes-with-freebsd-part-1.html'>2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage</a><br /> +<a class='textlink' href='./2024-12-03-f3s-kubernetes-with-freebsd-part-2.html'>2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation</a><br /> +<br /> +<a href='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png'><img alt='f3s logo' title='f3s logo' src='./f3s-kubernetes-with-freebsd-part-1/f3slogo.png' /></a><br /> +<br /> +<h2 style='display: inline' id='table-of-contents'>Table of Contents</h2><br /> +<br /> +<ul> +<li><a href='#f3s-kubernetes-with-freebsd---part-3-protecting-from-power-cuts'>f3s: Kubernetes with FreeBSD - Part 3: Protecting from power cuts</a></li> +<li>⇢ <a href='#introduction'>Introduction</a></li> +<li>⇢ <a href='#changes-since-last-time'>Changes since last time</a></li> +<li>⇢ ⇢ <a href='#freebsd-upgrade-from-141-to-142'>FreeBSD upgrade from 14.1 to 14.2</a></li> +<li>⇢ ⇢ <a href='#a-new-home-behind-the-tv'>A new home (behind the TV)</a></li> +<li>⇢ <a href='#the-ups-hardware'>The UPS hardware</a></li> +<li>⇢ <a href='#configuring-freebsd-to-work-with-the-ups'>Configuring FreeBSD to Work with the UPS</a></li> +<li>⇢ ⇢ <a href='#usb-device-detection'>USB Device Detection</a></li> +<li>⇢ ⇢ <a href='#apcupsd-installation'><span class='inlinecode'>apcupsd</span> Installation</a></li> +<li>⇢ ⇢ <a href='#ups-connectivity-test'>UPS Connectivity Test</a></li> +<li>⇢ <a href='#apc-info-on-partner-nodes'>APC Info on Partner Nodes:</a></li> +<li>⇢ ⇢ <a href='#installation-on-partners'>Installation on partners</a></li> +<li>⇢ <a href='#power-outage-simulation'>Power outage simulation</a></li> +<li>⇢ ⇢ <a href='#pulling-the-plug'>Pulling the plug</a></li> +<li>⇢ ⇢ <a href='#restoring-power'>Restoring power</a></li> +</ul><br /> +<h2 style='display: inline' id='introduction'>Introduction</h2><br /> +<br /> +<span>In this blog post, we are setting up the UPS for the cluster. A UPS, or Uninterruptible Power Supply, safeguards my cluster from unexpected power outages and surges. It acts as a backup battery that kicks in when the electricity cuts out—especially useful in my area, where power cuts are frequent—allowing for a graceful system shutdown and preventing data loss and corruption. This is especially important since I will also store some of my data on the f3s nodes.</span><br /> +<br /> +<h2 style='display: inline' id='changes-since-last-time'>Changes since last time</h2><br /> +<br /> +<h3 style='display: inline' id='freebsd-upgrade-from-141-to-142'>FreeBSD upgrade from 14.1 to 14.2</h3><br /> +<br /> +<span>There has been a new release since the last blog post in this series. The upgrade from 14.1 was as easy as:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f0: ~ % doas freebsd-update fetch +paul@f0: ~ % doas freebsd-update install +paul@f0: ~ % doas freebsd-update -r <font color="#000000">14.2</font>-RELEASE upgrade +paul@f0: ~ % doas freebsd-update install +paul@f0: ~ % doas shutdown -r now +</pre> +<br /> +<span>And after rebooting, I ran:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f0: ~ % doas freebsd-update install +paul@f0: ~ % doas pkg update +paul@f0: ~ % doas pkg upgrade +paul@f0: ~ % doas shutdown -r now +</pre> +<br /> +<span>And after another reboot, I was on 14.2:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f0:~ % uname -a +FreeBSD f0.lan.buetow.org <font color="#000000">14.2</font>-RELEASE FreeBSD <font color="#000000">14.2</font>-RELEASE + releng/<font color="#000000">14.2</font>-n<font color="#000000">269506</font>-c8918d6c7412 GENERIC amd64 +</pre> +<br /> +<span>And, of course, I ran this on all 3 nodes!</span><br /> +<br /> +<h3 style='display: inline' id='a-new-home-behind-the-tv'>A new home (behind the TV)</h3><br /> +<br /> +<span>I've put all the infrastructure behind my TV, as plenty of space is available. The TV hides most of the setup, which drastically improved the SAF (spouse acceptance factor).</span><br /> +<br /> +<a href='./f3s-kubernetes-with-freebsd-part-3/f3s-changes.jpg'><img alt='New hardware placement arrangement' title='New hardware placement arrangement' src='./f3s-kubernetes-with-freebsd-part-3/f3s-changes.jpg' /></a><br /> +<br /> +<span>I got rid of the mini-switch I mentioned in the previous blog post. I have the TP-Link EAP615-Wall mounted on the wall nearby, which is my OpenWrt-powered Wi-Fi hotspot. It also has 3 Ethernet ports, to which I connected the Beelink nodes. That's the device you see at the very top.</span><br /> +<br /> +<span>The Ethernet cables go downward through the cable boxes to the Beelink nodes. In addition to the Beelink f3s nodes, I connected the TP-Link to the UPS as well (not discussed further in this blog post, but the positive side effect is that my Wi-Fi will still work during a power loss for some time—and during a power cut, the Beelink nodes will still be able to communicate with each other).</span><br /> +<br /> +<span>On the very left (the black box) is the UPS, with four power outlets. Three go to the Beelink nodes, and one goes to the TP-Link. A USB output is also connected to the first Beelink node, <span class='inlinecode'>f0</span>. </span><br /> +<br /> +<span>On the very right (halfway hidden behind the TV) are the 3 Beelink nodes stacked on top of each other. The only downside (or upside?) is that my 14-month-old daughter is now chaos-testing the Beelink nodes, as the red power buttons (now reachable for her) are very attractive for her to press when passing by randomly. :-) Luckily, that will only cause graceful system shutdowns!</span><br /> +<br /> +<h2 style='display: inline' id='the-ups-hardware'>The UPS hardware</h2><br /> +<br /> +<span>I wanted a UPS that I could connect to via FreeBSD, and that would provide enough backup power to operate the cluster for a couple of minutes (it turned out to be around an hour, but this time will likely be shortened after future hardware upgrades, like additional drives and a backup enclosure) and to automatically initiate the shutdown of all the f3s nodes.</span><br /> +<br /> +<span>I decided on the APC Back-UPS BX750MI model because:</span><br /> +<br /> +<ul> +<li>Zero noise level when there is no power cut (some light noise when the battery is in operation during a power cut).</li> +<li>Cost: It is relatively affordable (not costing thousands).</li> +<li>USB connectivity: Can be connected via USB to one of the FreeBSD hosts to read the UPS status.</li> +<li>A power output of 750VA (or 410 watts), suitable for an hour of runtime for my f3s nodes (plus the Wi-Fi router).</li> +<li>Multiple power outlets: Can connect all 3 f3s nodes directly.</li> +<li>User-replaceable batteries: I can replace the batteries myself after two years or more (depending on usage).</li> +<li>Its compact design. Overall, I like how it looks.</li> +</ul><br /> +<a href='./f3s-kubernetes-with-freebsd-part-3/apc-back-ups.jpg'><img alt='The APC Back-UPS BX750MI in operation.' title='The APC Back-UPS BX750MI in operation.' src='./f3s-kubernetes-with-freebsd-part-3/apc-back-ups.jpg' /></a><br /> +<br /> +<h2 style='display: inline' id='configuring-freebsd-to-work-with-the-ups'>Configuring FreeBSD to Work with the UPS</h2><br /> +<br /> +<h3 style='display: inline' id='usb-device-detection'>USB Device Detection</h3><br /> +<br /> +<span>Once plugged in via USB on FreeBSD, I could see the following in the kernel messages:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f0: ~ % doas dmesg | grep UPS +ugen0.<font color="#000000">2</font>: <American Power Conversion Back-UPS BX750MI> at usbus0 +</pre> +<br /> +<h3 style='display: inline' id='apcupsd-installation'><span class='inlinecode'>apcupsd</span> Installation</h3><br /> +<br /> +<span>To make use of the USB connection, the <span class='inlinecode'>apcupsd</span> package had to be installed:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f0: ~ % doas install apcupsd +</pre> +<br /> +<span>I have made the following modifications to the configuration file so that the UPS can be used via the USB interface:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f0:/usr/local/etc/apcupsd % diff -u apcupsd.conf.sample apcupsd.conf +--- apcupsd.conf.sample <font color="#000000">2024</font>-<font color="#000000">11</font>-<font color="#000000">01</font> <font color="#000000">16</font>:<font color="#000000">40</font>:<font color="#000000">42.000000000</font> +<font color="#000000">0200</font> ++++ apcupsd.conf <font color="#000000">2024</font>-<font color="#000000">12</font>-<font color="#000000">03</font> <font color="#000000">10</font>:<font color="#000000">58</font>:<font color="#000000">24.009501000</font> +<font color="#000000">0200</font> +@@ -<font color="#000000">31</font>,<font color="#000000">7</font> +<font color="#000000">31</font>,<font color="#000000">7</font> @@ + <i><font color="silver"># 940-1524C, 940-0024G, 940-0095A, 940-0095B,</font></i> + <i><font color="silver"># 940-0095C, 940-0625A, M-04-02-2000</font></i> + <i><font color="silver">#</font></i> +-UPSCABLE smart ++UPSCABLE usb + + <i><font color="silver"># To get apcupsd to work, in addition to defining the cable</font></i> + <i><font color="silver"># above, you must also define a UPSTYPE, which corresponds to</font></i> +@@ -<font color="#000000">88</font>,<font color="#000000">8</font> +<font color="#000000">88</font>,<font color="#000000">10</font> @@ + <i><font color="silver"># that apcupsd binds to that particular unit</font></i> + <i><font color="silver"># (helpful if you have more than one USB UPS).</font></i> + <i><font color="silver">#</font></i> +-UPSTYPE apcsmart +-DEVICE /dev/usv ++UPSTYPE usb ++DEVICE + + <i><font color="silver"># POLLTIME <int></font></i> + <i><font color="silver"># Interval (in seconds) at which apcupsd polls the UPS for status. This</font></i> +</pre> +<br /> +<span>I left the remaining settings as the default ones; for example, the following are of main interest:</span><br /> +<br /> +<pre> +# If during a power failure, the remaining battery percentage +# (as reported by the UPS) is below or equal to BATTERYLEVEL, +# apcupsd will initiate a system shutdown. +BATTERYLEVEL 5 + +# If during a power failure, the remaining runtime in minutes +# (as calculated internally by the UPS) is below or equal to MINUTES, +# apcupsd, will initiate a system shutdown. +MINUTES 3 +</pre> +<br /> +<span>I then enabled and started the daemon:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f0:/usr/local/etc/apcupsd % doas sysrc apcupsd_enable=YES +apcupsd_enable: -> YES +paul@f0:/usr/local/etc/apcupsd % doas service apcupsd start +Starting apcupsd. +</pre> +<br /> +<h3 style='display: inline' id='ups-connectivity-test'>UPS Connectivity Test</h3><br /> +<br /> +<span>And voila, I could now access the UPS information via the <span class='inlinecode'>apcaccess</span> command; how convenient :-) (I also read through the manual page, which provides a good understanding of what else can be done with it!).</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f0:~ % apcaccess +APC : <font color="#000000">001</font>,<font color="#000000">035</font>,<font color="#000000">0857</font> +DATE : <font color="#000000">2025</font>-<font color="#000000">01</font>-<font color="#000000">26</font> <font color="#000000">14</font>:<font color="#000000">43</font>:<font color="#000000">27</font> +<font color="#000000">0200</font> +HOSTNAME : f0.lan.buetow.org +VERSION : <font color="#000000">3.14</font>.<font color="#000000">14</font> (<font color="#000000">31</font> May <font color="#000000">2016</font>) freebsd +UPSNAME : f0.lan.buetow.org +CABLE : USB Cable +DRIVER : USB UPS Driver +UPSMODE : Stand Alone +STARTTIME: <font color="#000000">2025</font>-<font color="#000000">01</font>-<font color="#000000">26</font> <font color="#000000">14</font>:<font color="#000000">43</font>:<font color="#000000">25</font> +<font color="#000000">0200</font> +MODEL : Back-UPS BX750MI +STATUS : ONLINE +LINEV : <font color="#000000">230.0</font> Volts +LOADPCT : <font color="#000000">4.0</font> Percent +BCHARGE : <font color="#000000">100.0</font> Percent +TIMELEFT : <font color="#000000">65.3</font> Minutes +MBATTCHG : <font color="#000000">5</font> Percent +MINTIMEL : <font color="#000000">3</font> Minutes +MAXTIME : <font color="#000000">0</font> Seconds +SENSE : Medium +LOTRANS : <font color="#000000">145.0</font> Volts +HITRANS : <font color="#000000">295.0</font> Volts +ALARMDEL : No alarm +BATTV : <font color="#000000">13.6</font> Volts +LASTXFER : Automatic or explicit self <b><u><font color="#000000">test</font></u></b> +NUMXFERS : <font color="#000000">0</font> +TONBATT : <font color="#000000">0</font> Seconds +CUMONBATT: <font color="#000000">0</font> Seconds +XOFFBATT : N/A +SELFTEST : NG +STATFLAG : <font color="#000000">0x05000008</font> +SERIALNO : 9B2414A03599 +BATTDATE : <font color="#000000">2001</font>-<font color="#000000">01</font>-<font color="#000000">01</font> +NOMINV : <font color="#000000">230</font> Volts +NOMBATTV : <font color="#000000">12.0</font> Volts +NOMPOWER : <font color="#000000">410</font> Watts +END APC : <font color="#000000">2025</font>-<font color="#000000">01</font>-<font color="#000000">26</font> <font color="#000000">14</font>:<font color="#000000">44</font>:<font color="#000000">06</font> +<font color="#000000">0200</font> +</pre> +<br /> +<h2 style='display: inline' id='apc-info-on-partner-nodes'>APC Info on Partner Nodes:</h2><br /> +<br /> +<span>So far, so good. Host <span class='inlinecode'>f0</span> would shut down itself when short on power. But what about the <span class='inlinecode'>f1</span> and <span class='inlinecode'>f2</span> nodes? They aren't connected directly to the UPS and, therefore, wouldn't know that their power is about to be cut off. For this, <span class='inlinecode'>apcupsd</span> running on the <span class='inlinecode'>f1</span> and <span class='inlinecode'>f2</span> nodes can be configured to retrieve UPS information via the network from the <span class='inlinecode'>apcupsd</span> server running on the <span class='inlinecode'>f0</span> node, which is connected directly to the APC via USB.</span><br /> +<br /> +<span>Of course, this won't work when <span class='inlinecode'>f0</span> is down. In this case, no operational node would be connected to the UPS via USB; therefore, the current power status would not be known. However, I consider this a rare circumstance. Furthermore, in case of an <span class='inlinecode'>f0</span> system crash, sudden power outages on the two other nodes would occur at different times, making real data loss (the main concern here) effectively impossible.</span><br /> +<br /> +<h3 style='display: inline' id='installation-on-partners'>Installation on partners</h3><br /> +<br /> +<span>To do this, I installed <span class='inlinecode'>apcupsd</span> via <span class='inlinecode'>doas pkg install apcupsd</span> on <span class='inlinecode'>f1</span> and <span class='inlinecode'>f2</span>, and then I could connect to it this way:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f1:~ % apcaccess -h f0.lan.buetow.org | grep Percent +LOADPCT : <font color="#000000">12.0</font> Percent +BCHARGE : <font color="#000000">94.0</font> Percent +MBATTCHG : <font color="#000000">5</font> Percent +</pre> +<br /> +<span>But I want the daemon to be configured and enabled in such a way that it connects to the master UPS node (the one with the UPS connected via USB) so that it can also initiate a system shutdown when the UPS battery reaches low levels. For that, <span class='inlinecode'>apcupsd</span> itself needs to be aware of the UPS status.</span><br /> +<br /> +<span>On <span class='inlinecode'>f1</span> and <span class='inlinecode'>f2</span>, I changed the configuration to use <span class='inlinecode'>f0</span> (where <span class='inlinecode'>apcupsd</span> is listening) as a remote device. I also changed the <span class='inlinecode'>MINUTES</span> setting from 3 to 6 and the <span class='inlinecode'>BATTERYLEVEL</span> setting from 5 to 10 to ensure that the <span class='inlinecode'>f1</span> and <span class='inlinecode'>f2</span> nodes could still connect to the <span class='inlinecode'>f0</span> node for UPS information before <span class='inlinecode'>f0</span> decides to shut down itself. So <span class='inlinecode'>f1</span> and <span class='inlinecode'>f2</span> must shut down earlier than <span class='inlinecode'>f0</span>:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f2:/usr/local/etc/apcupsd % diff -u apcupsd.conf.sample apcupsd.conf +--- apcupsd.conf.sample <font color="#000000">2024</font>-<font color="#000000">11</font>-<font color="#000000">01</font> <font color="#000000">16</font>:<font color="#000000">40</font>:<font color="#000000">42.000000000</font> +<font color="#000000">0200</font> ++++ apcupsd.conf <font color="#000000">2025</font>-<font color="#000000">01</font>-<font color="#000000">26</font> <font color="#000000">15</font>:<font color="#000000">52</font>:<font color="#000000">45.108469000</font> +<font color="#000000">0200</font> +@@ -<font color="#000000">31</font>,<font color="#000000">7</font> +<font color="#000000">31</font>,<font color="#000000">7</font> @@ + <i><font color="silver"># 940-1524C, 940-0024G, 940-0095A, 940-0095B,</font></i> + <i><font color="silver"># 940-0095C, 940-0625A, M-04-02-2000</font></i> + <i><font color="silver">#</font></i> +-UPSCABLE smart ++UPSCABLE ether + + <i><font color="silver"># To get apcupsd to work, in addition to defining the cable</font></i> + <i><font color="silver"># above, you must also define a UPSTYPE, which corresponds to</font></i> +@@ -<font color="#000000">52</font>,<font color="#000000">7</font> +<font color="#000000">52</font>,<font color="#000000">6</font> @@ + <i><font color="silver"># Network Information Server. This is used if the</font></i> + <i><font color="silver"># UPS powering your computer is connected to a</font></i> + <i><font color="silver"># different computer for monitoring.</font></i> +-<i><font color="silver">#</font></i> + <i><font color="silver"># snmp hostname:port:vendor:community</font></i> + <i><font color="silver"># SNMP network link to an SNMP-enabled UPS device.</font></i> + <i><font color="silver"># Hostname is the ip address or hostname of the UPS</font></i> +@@ -<font color="#000000">88</font>,<font color="#000000">8</font> +<font color="#000000">87</font>,<font color="#000000">8</font> @@ + <i><font color="silver"># that apcupsd binds to that particular unit</font></i> + <i><font color="silver"># (helpful if you have more than one USB UPS).</font></i> + <i><font color="silver">#</font></i> +-UPSTYPE apcsmart +-DEVICE /dev/usv ++UPSTYPE net ++DEVICE f0.lan.buetow.org:<font color="#000000">3551</font> + + <i><font color="silver"># POLLTIME <int></font></i> + <i><font color="silver"># Interval (in seconds) at which apcupsd polls the UPS for status. This</font></i> +@@ -<font color="#000000">147</font>,<font color="#000000">12</font> +<font color="#000000">146</font>,<font color="#000000">12</font> @@ + <i><font color="silver"># If during a power failure, the remaining battery percentage</font></i> + <i><font color="silver"># (as reported by the UPS) is below or equal to BATTERYLEVEL,</font></i> + <i><font color="silver"># apcupsd will initiate a system shutdown.</font></i> +-BATTERYLEVEL <font color="#000000">5</font> ++BATTERYLEVEL <font color="#000000">10</font> + + <i><font color="silver"># If during a power failure, the remaining runtime in minutes</font></i> + <i><font color="silver"># (as calculated internally by the UPS) is below or equal to MINUTES,</font></i> + <i><font color="silver"># apcupsd, will initiate a system shutdown.</font></i> +-MINUTES <font color="#000000">3</font> ++MINUTES <font color="#000000">6</font> + + <i><font color="silver"># If during a power failure, the UPS has run on batteries for TIMEOUT</font></i> + <i><font color="silver"># many seconds or longer, apcupsd will initiate a system shutdown.</font></i> + +</pre> +<span>So I also ran the following commands on <span class='inlinecode'>f1</span> and <span class='inlinecode'>f2</span>:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f1:/usr/local/etc/apcupsd % doas sysrc apcupsd_enable=YES +apcupsd_enable: -> YES +paul@f1:/usr/local/etc/apcupsd % doas service apcupsd start +Starting apcupsd. +</pre> +<br /> +<span>And then I was able to connect to localhost via the <span class='inlinecode'>apcaccess</span> command:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f1:~ % doas apcaccess | grep Percent +LOADPCT : <font color="#000000">5.0</font> Percent +BCHARGE : <font color="#000000">95.0</font> Percent +MBATTCHG : <font color="#000000">5</font> Percent +</pre> +<br /> +<h2 style='display: inline' id='power-outage-simulation'>Power outage simulation</h2><br /> +<br /> +<h3 style='display: inline' id='pulling-the-plug'>Pulling the plug</h3><br /> +<br /> +<span>I simulated a power outage by removing the power input from the APC. Immediately, the following message appeared on all the nodes:</span><br /> +<br /> +<pre> +Broadcast Message from root@f0.lan.buetow.org + (no tty) at 15:03 EET... + +Power failure. Running on UPS batteries. +</pre> +<br /> +<span>I ran the following command to confirm the available battery time:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>paul@f0:/usr/local/etc/apcupsd % apcaccess -p TIMELEFT +<font color="#000000">63.9</font> Minutes +</pre> +<br /> +<span>And after almost 60 minutes (<span class='inlinecode'>f1</span> and <span class='inlinecode'>f2</span> a bit earlier, <span class='inlinecode'>f0</span> a bit later due to the different <span class='inlinecode'>BATTERYLEVEL</span> and <span class='inlinecode'>MINUTES</span> settings outlined earlier), the following broadcast was sent out:</span><br /> +<br /> +<pre> +Broadcast Message from root@f0.lan.buetow.org + (no tty) at 15:08 EET... + + *** FINAL System shutdown message from paul@f1.lan.buetow.org *** + +System going down IMMEDIATELY + +apcupsd initiated shutdown +</pre> +<br /> +<span>And all the nodes shut down safely before the UPS ran out of battery!</span><br /> +<br /> +<h3 style='display: inline' id='restoring-power'>Restoring power</h3><br /> +<br /> +<span>After restoring power, I checked the logs in <span class='inlinecode'>/var/log/daemon.log</span> and found the following on all 3 nodes:</span><br /> +<br /> +<pre> +Jan 26 17:36:24 f2 apcupsd[2159]: Power failure. +Jan 26 17:36:30 f2 apcupsd[2159]: Running on UPS batteries. +Jan 26 17:36:30 f2 apcupsd[2159]: Battery charge below low limit. +Jan 26 17:36:30 f2 apcupsd[2159]: Initiating system shutdown! +Jan 26 17:36:30 f2 apcupsd[2159]: User logins prohibited +Jan 26 17:36:32 f2 apcupsd[2159]: apcupsd exiting, signal 15 +Jan 26 17:36:32 f2 apcupsd[2159]: apcupsd shutdown succeeded +</pre> +<br /> +<span>All good :-) </span><br /> +<br /> +<span>Other BSD related posts are:</span><br /> +<br /> +<a class='textlink' href='./2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html'>2016-04-09 Jails and ZFS with Puppet on FreeBSD</a><br /> +<a class='textlink' href='./2022-07-30-lets-encrypt-with-openbsd-and-rex.html'>2022-07-30 Let's Encrypt with OpenBSD and Rex</a><br /> +<a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD</a><br /> +<a class='textlink' href='./2024-01-13-one-reason-why-i-love-openbsd.html'>2024-01-13 One reason why I love OpenBSD</a><br /> +<a class='textlink' href='./2024-04-01-KISS-high-availability-with-OpenBSD.html'>2024-04-01 KISS high-availability with OpenBSD</a><br /> +<a class='textlink' href='./2024-11-17-f3s-kubernetes-with-freebsd-part-1.html'>2024-11-17 f3s: Kubernetes with FreeBSD - Part 1: Setting the stage</a><br /> +<a class='textlink' href='./2024-12-03-f3s-kubernetes-with-freebsd-part-2.html'>2024-12-03 f3s: Kubernetes with FreeBSD - Part 2: Hardware and base installation</a><br /> +<br /> +<span>E-Mail your comments to <span class='inlinecode'>paul@nospam.buetow.org</span> :-)</span><br /> +<br /> +<a class='textlink' href='../'>Back to the main site</a><br /> +<p class="footer"> +Generated with <a href="https://codeberg.org/snonux/gemtexter">Gemtexter 3.0.1-develop</a> | +served by <a href="https://www.OpenBSD.org">OpenBSD</a>/<a href="https://man.openbsd.org/relayd.8">relayd(8)</a>+<a href="https://man.openbsd.org/httpd.8">httpd(8)</a> | +<a href="https://foo.zone/site-mirrors.html">Site Mirrors</a> +</p> +</body> +</html> diff --git a/gemfeed/f3s-kubernetes-with-freebsd-part-3/apc-back-ups.jpg b/gemfeed/f3s-kubernetes-with-freebsd-part-3/apc-back-ups.jpg Binary files differnew file mode 100644 index 00000000..15b1ca78 --- /dev/null +++ b/gemfeed/f3s-kubernetes-with-freebsd-part-3/apc-back-ups.jpg diff --git a/gemfeed/f3s-kubernetes-with-freebsd-part-3/f3s-changes.jpg b/gemfeed/f3s-kubernetes-with-freebsd-part-3/f3s-changes.jpg Binary files differnew file mode 100644 index 00000000..5752f2fc --- /dev/null +++ b/gemfeed/f3s-kubernetes-with-freebsd-part-3/f3s-changes.jpg |
