From 1f3253cfc352e31ebbc736496ab3a34d80a7a2a7 Mon Sep 17 00:00:00 2001
From: Paul Buetow
Date: Tue, 27 Jan 2026 10:10:18 +0200
Subject: Update content for html
---
about/resources.html | 206 ++++++++++-----------
...5-07-14-f3s-kubernetes-with-freebsd-part-6.html | 140 +++++++-------
gemfeed/atom.xml | 142 +++++++-------
index.html | 2 +-
uptime-stats.html | 2 +-
5 files changed, 246 insertions(+), 246 deletions(-)
diff --git a/about/resources.html b/about/resources.html
index 10bf01d5..210a2fff 100644
--- a/about/resources.html
+++ b/about/resources.html
@@ -50,68 +50,68 @@
In random order:
-
Raku Fundamentals; Moritz Lenz; Apress
-
Polished Ruby Programming; Jeremy Evans; Packt Publishing
-
Kubernetes Cookbook; Sameer Naik, Sébastien Goasguen, Jonathan Michaux; O'Reilly
-
Pro Puppet; James Turnbull, Jeffrey McCune; Apress
-
Think Raku (aka Think Perl 6); Laurent Rosenfeld, Allen B. Downey; O'Reilly
-
Perl New Features; Joshua McAdams, brian d foy; Perl School
-
Learn You Some Erlang for Great Good; Fred Herbert; No Starch Press
-
Terraform Cookbook; Mikael Krief; Packt Publishing
-
21st Century C: C Tips from the New School; Ben Klemens; O'Reilly
Data Science at the Command Line; Jeroen Janssens; O'Reilly
-
Object-Oriented Programming with ANSI-C; Axel-Tobias Schreiner
-
Tmux 2: Productive Mouse-free Development; Brain P. Hogan; The Pragmatic Programmers
-
Java ist auch eine Insel; Christian Ullenboom;
-
Distributed Systems: Principles and Paradigms; Andrew S. Tanenbaum; Pearson
+
Effective Java; Joshua Bloch; Addison-Wesley Professional
+
Leanring eBPF; Liz Rice; O'Reilly
+
The Practise of System and Network Administration; Thomas A. Limoncelli, Christina J. Hogan, Strata R. Chalup; Addison-Wesley Professional Pro Git; Scott Chacon, Ben Straub; Apress
The DevOps Handbook; Gene Kim, Jez Humble, Patrick Debois, John Willis; Audible
-
Ultimate Go Notebook; Bill Kennedy
-
Programming Ruby 3.3 (5th Edition); Noel Rappin, with Dave Thomas; The Pragmatic Bookshelf
-
The Docker Book; James Turnbull; Kindle
-
Programming Perl aka "The Camel Book"; Tom Christiansen, brian d foy, Larry Wall & Jon Orwant; O'Reilly
-
DNS and BIND; Cricket Liu; O'Reilly
-
Higher Order Perl; Mark Dominus; Morgan Kaufmann
+
Effective awk programming; Arnold Robbins; O'Reilly
+
Funktionale Programmierung; Peter Pepper; Springer
+
Kubernetes Cookbook; Sameer Naik, Sébastien Goasguen, Jonathan Michaux; O'Reilly
100 Go Mistakes and How to Avoid Them; Teiva Harsanyi; Manning Publications
-
Leanring eBPF; Liz Rice; O'Reilly
-
Learn You a Haskell for Great Good!; Miran Lipovaca; No Starch Press
-
Developing Games in Java; David Brackeen and others...; New Riders
-
The Kubernetes Book; Nigel Poulton; Unabridged Audiobook
Concurrency in Go; Katherine Cox-Buday; O'Reilly
-
The Go Programming Language; Alan A. A. Donovan; Addison-Wesley Professional
-
Funktionale Programmierung; Peter Pepper; Springer
-
The Pragmatic Programmer; David Thomas; Addison-Wesley
-
Raku Recipes; J.J. Merelo; Apress
-
The KCNA (Kubernetes and Cloud Native Associate) Book; Nigel Poulton
+
Hands-on Infrastructure Monitoring with Prometheus; Joel Bastos, Pedro Araujo; Packt
+
Go Brain Teasers - Exercise Your Mind; Miki Tebeka; The Pragmatic Programmers
Chaos Engineering - System Resiliency in Practice; Casey Rosenthal and Nora Jones; eBook
-
97 things every SRE should know; Emil Stolarsky, Jaime Woo; O'Reilly
-
Seeking SRE: Conversations About Running Production Systems at Scale; David N. Blank-Edelman; eBook
-
Site Reliability Engineering; How Google runs production systems; O'Reilly
-
Clusterbau mit Linux-HA; Michael Schwartzkopff; O'Reilly
+
Ultimate Go Notebook; Bill Kennedy
+
Learn You Some Erlang for Great Good; Fred Herbert; No Starch Press
+
Developing Games in Java; David Brackeen and others...; New Riders
+
Pro Puppet; James Turnbull, Jeffrey McCune; Apress
+
The KCNA (Kubernetes and Cloud Native Associate) Book; Nigel Poulton
+
Tmux 2: Productive Mouse-free Development; Brain P. Hogan; The Pragmatic Programmers
+
The Docker Book; James Turnbull; Kindle
+
Amazon Web Services in Action; Michael Wittig and Andreas Wittig; Manning Publications
C++ Programming Language; Bjarne Stroustrup;
-
Systemprogrammierung in Go; Frank Müller; dpunkt
+
Raku Fundamentals; Moritz Lenz; Apress
+
The Go Programming Language; Alan A. A. Donovan; Addison-Wesley Professional
+
Think Raku (aka Think Perl 6); Laurent Rosenfeld, Allen B. Downey; O'Reilly
DevOps And Site Reliability Engineering Handbook; Stephen Fleming; Audible
-
Go Brain Teasers - Exercise Your Mind; Miki Tebeka; The Pragmatic Programmers
-
Hands-on Infrastructure Monitoring with Prometheus; Joel Bastos, Pedro Araujo; Packt
+
Programming Perl aka "The Camel Book"; Tom Christiansen, brian d foy, Larry Wall & Jon Orwant; O'Reilly
+
Raku Recipes; J.J. Merelo; Apress
+
Clusterbau mit Linux-HA; Michael Schwartzkopff; O'Reilly
+
21st Century C: C Tips from the New School; Ben Klemens; O'Reilly
+
Java ist auch eine Insel; Christian Ullenboom;
+
Terraform Cookbook; Mikael Krief; Packt Publishing
Modern Perl; Chromatic ; Onyx Neon Press
+
Perl New Features; Joshua McAdams, brian d foy; Perl School
+
Higher Order Perl; Mark Dominus; Morgan Kaufmann
+
Learn You a Haskell for Great Good!; Miran Lipovaca; No Starch Press
+
Object-Oriented Programming with ANSI-C; Axel-Tobias Schreiner
Systems Performance Tuning; Gian-Paolo D. Musumeci and others...; O'Reilly
-
Amazon Web Services in Action; Michael Wittig and Andreas Wittig; Manning Publications
-
Effective Java; Joshua Bloch; Addison-Wesley Professional
-
Effective awk programming; Arnold Robbins; O'Reilly
-
The Practise of System and Network Administration; Thomas A. Limoncelli, Christina J. Hogan, Strata R. Chalup; Addison-Wesley Professional Pro Git; Scott Chacon, Ben Straub; Apress
+
Site Reliability Engineering; How Google runs production systems; O'Reilly
+
97 things every SRE should know; Emil Stolarsky, Jaime Woo; O'Reilly
+
DNS and BIND; Cricket Liu; O'Reilly
+
Systemprogrammierung in Go; Frank Müller; dpunkt
+
Polished Ruby Programming; Jeremy Evans; Packt Publishing
+
The Pragmatic Programmer; David Thomas; Addison-Wesley
+
Distributed Systems: Principles and Paradigms; Andrew S. Tanenbaum; Pearson
+
Seeking SRE: Conversations About Running Production Systems at Scale; David N. Blank-Edelman; eBook
+
Programming Ruby 3.3 (5th Edition); Noel Rappin, with Dave Thomas; The Pragmatic Bookshelf
+
The Kubernetes Book; Nigel Poulton; Unabridged Audiobook
Technical references
I didn't read them from the beginning to the end, but I am using them to look up things. The books are in random order:
-
BPF Performance Tools - Linux System and Application Observability, Brendan Gregg; Addison Wesley
-
Understanding the Linux Kernel; Daniel P. Bovet, Marco Cesati; O'Reilly
Groovy Kurz & Gut; Joerg Staudemeier; O'Reilly
+
BPF Performance Tools - Linux System and Application Observability, Brendan Gregg; Addison Wesley
Go: Design Patterns for Real-World Projects; Mat Ryer; Packt
The Linux Programming Interface; Michael Kerrisk; No Starch Press
-
Algorithms; Robert Sedgewick, Kevin Wayne; Addison Wesley
-
Relayd and Httpd Mastery; Michael W Lucas
+
Understanding the Linux Kernel; Daniel P. Bovet, Marco Cesati; O'Reilly
Implementing Service Level Objectives; Alex Hidalgo; O'Reilly
+
Relayd and Httpd Mastery; Michael W Lucas
+
Algorithms; Robert Sedgewick, Kevin Wayne; Addison Wesley
Self-development and soft-skills books
@@ -119,43 +119,43 @@
Digital Minimalism; Cal Newport; Portofolio Penguin
-
Slow Productivity; Cal Newport; Penguin Random House
-
Influence without Authority; A. Cohen, D. Bradford; Wiley
-
Getting Things Done; David Allen
+
Consciousness: A Very Short Introduction; Susan Blackmore; Oxford Uiversity Press
+
The Courage to Be Disliked; Ichiro Kishimi and Fumitake Koga; Audiobook
+
The Phoenix Project - A Novel About IT, DevOps, and Helping your Business Win; Gene Kim and Kevin Behr; Trade Select
+
97 Things Every Engineering Manager Should Know; Camille Fournier; Audiobook
+
Atomic Habits; James Clear; Random House Business
Soft Skills; John Sommez; Manning Publications
-
Stop starting, start finishing; Arne Roock; Lean-Kanban University
-
The Bullet Journal Method; Ryder Carroll; Fourth Estate
+
Ultralearning; Scott Young; Thorsons
Coders at Work - Reflections on the craft of programming, Peter Seibel and Mitchell Dorian et al., Audiobook
-
The Software Engineer's Guidebook: Navigating senior, tech lead, and staff engineer positions at tech companies and startups; Gergely Orosz; Audiobook
+
The Good Enough Job; Simone Stolzoff; Ebury Edge
+
Psycho-Cybernetics; Maxwell Maltz; Perigee Books
+
So Good They Can't Ignore You; Cal Newport; Business Plus
+
Time Management for System Administrators; Thomas A. Limoncelli; O'Reilly
+
The Power of Now; Eckhard Tolle; Yellow Kite
+
The Daily Stoic; Ryan Holiday, Stephen Hanselman; Profile Books
Eat That Frog!; Brian Tracy; Hodder Paperbacks
+
Eat That Frog; Brian Tracy
+
Staff Engineer: Leadership beyond the management track; Will Larson; Audiobook
Never Split the Difference; Chris Voss, Tahl Raz; Random House Business
-
The Power of Now; Eckhard Tolle; Yellow Kite
-
Ultralearning; Anna Laurent; Self-published via Amazon
+
The Bullet Journal Method; Ryder Carroll; Fourth Estate
Deep Work; Cal Newport; Piatkus
-
Meditation for Mortals, Oliver Burkeman, Audiobook
+
The Obstacle Is The Way; Ryan Holiday; Profile Books Ltd
The Joy of Missing Out; Christina Crook; New Society Publishers
+
Getting Things Done; David Allen
+
Stop starting, start finishing; Arne Roock; Lean-Kanban University
+
Meditation for Mortals, Oliver Burkeman, Audiobook
+
Who Moved My Cheese?; Dr. Spencer Johnson; Vermilion
+
101 Essays that change the way you think; Brianna Wiest; Audiobook
The Complete Software Developer's Career Guide; John Sonmez; Unabridged Audiobook
-
Search Inside Yourself - The Unexpected path to Achieving Success, Happiness (and World Peace); Chade-Meng Tan, Daniel Goleman, Jon Kabat-Zinn; HarperOne
+
Influence without Authority; A. Cohen, D. Bradford; Wiley
+
The Software Engineer's Guidebook: Navigating senior, tech lead, and staff engineer positions at tech companies and startups; Gergely Orosz; Audiobook
The 7 Habits Of Highly Effective People; Stephen R. Covey; Simon & Schuster UK
-
Consciousness: A Very Short Introduction; Susan Blackmore; Oxford Uiversity Press
-
The Obstacle Is The Way; Ryan Holiday; Profile Books Ltd
-
The Phoenix Project - A Novel About IT, DevOps, and Helping your Business Win; Gene Kim and Kevin Behr; Trade Select
-
Time Management for System Administrators; Thomas A. Limoncelli; O'Reilly
-
So Good They Can't Ignore You; Cal Newport; Business Plus
-
Solve for Happy; Mo Gawdat (RE-READ 1ST TIME)
-
The Daily Stoic; Ryan Holiday, Stephen Hanselman; Profile Books
-
Who Moved My Cheese?; Dr. Spencer Johnson; Vermilion
Buddah and Einstein walk into a Bar; Guy Joseph Ale, Claire Bloom; Blackstone Publishing
-
The Good Enough Job; Simone Stolzoff; Ebury Edge
-
The Courage to Be Disliked; Ichiro Kishimi and Fumitake Koga; Audiobook
-
97 Things Every Engineering Manager Should Know; Camille Fournier; Audiobook
-
101 Essays that change the way you think; Brianna Wiest; Audiobook
-
Staff Engineer: Leadership beyond the management track; Will Larson; Audiobook
-
Eat That Frog; Brian Tracy
+
Slow Productivity; Cal Newport; Penguin Random House
+
Ultralearning; Anna Laurent; Self-published via Amazon
+
Search Inside Yourself - The Unexpected path to Achieving Success, Happiness (and World Peace); Chade-Meng Tan, Daniel Goleman, Jon Kabat-Zinn; HarperOne
+
Solve for Happy; Mo Gawdat (RE-READ 1ST TIME)
The Off Switch; Mark Cropley; Virgin Books (RE-READ 1ST TIME)
@@ -164,31 +164,31 @@
Some of these were in-person with exams; others were online learning lectures only. In random order:
-
Cloud Operations on AWS - Learn how to configure, deploy, maintain, and troubleshoot your AWS environments; 3-day online live training with labs; Amazon
-
Apache Tomcat Best Practises; 3-day on-site training
+
Red Hat Certified System Administrator; Course + certification (Although I had the option, I decided not to take the next course as it is more effective to self learn what I need)
Linux Security and Isolation APIs Training; Michael Kerrisk; 3-day on-site training
-
Protocol buffers; O'Reilly Online
-
F5 Loadbalancers Training; 2-day on-site training; F5, Inc.
+
AWS Immersion Day; Amazon; 1-day interactive online training
+
Functional programming lecture; Remote University of Hagen
Ultimate Go Programming; Bill Kennedy; O'Reilly Online
Structure and Interpretation of Computer Programs; Harold Abelson and more...;
-
The Well-Grounded Rubyist Video Edition; David. A. Black; O'Reilly Online
-
Scripting Vim; Damian Conway; O'Reilly Online
-
Functional programming lecture; Remote University of Hagen
+
Algorithms Video Lectures; Robert Sedgewick; O'Reilly Online
MySQL Deep Dive Workshop; 2-day on-site training
+
Apache Tomcat Best Practises; 3-day on-site training
Developing IaC with Terraform (with Live Lessons); O'Reilly Online
-
Red Hat Certified System Administrator; Course + certification (Although I had the option, I decided not to take the next course as it is more effective to self learn what I need)
The Ultimate Kubernetes Bootcamp; School of Devops; O'Reilly Online
-
Algorithms Video Lectures; Robert Sedgewick; O'Reilly Online
-
AWS Immersion Day; Amazon; 1-day interactive online training
+
F5 Loadbalancers Training; 2-day on-site training; F5, Inc.
+
Protocol buffers; O'Reilly Online
+
The Well-Grounded Rubyist Video Edition; David. A. Black; O'Reilly Online
+
Scripting Vim; Damian Conway; O'Reilly Online
+
Cloud Operations on AWS - Learn how to configure, deploy, maintain, and troubleshoot your AWS environments; 3-day online live training with labs; Amazon
Technical guides
These are not whole books, but guides (smaller or larger) which I found very useful. in random order:
-
How CPUs work at https://cpu.land
-
Raku Guide at https://raku.guide
Advanced Bash-Scripting Guide
+
Raku Guide at https://raku.guide
+
How CPUs work at https://cpu.land
Podcasts
@@ -197,51 +197,51 @@
In random order:
-
Pratical AI
-
Modern Mentor
-
Hidden Brain
-
Deep Questions with Cal Newport
The Changelog Podcast(s)
+
Cup o' Go [Golang]
Fallthrough [Golang]
-
The Pragmatic Engineer Podcast
+
Deep Questions with Cal Newport
Fork Around And Find Out
-
Backend Banter
-
BSD Now [BSD]
-
Dev Interrupted
-
Cup o' Go [Golang]
-
Maintainable
Wednesday Wisdom
+
Dev Interrupted
+
Hidden Brain
+
Backend Banter
The ProdCast (Google SRE Podcast)
+
Maintainable
+
The Pragmatic Engineer Podcast
+
Modern Mentor
+
Pratical AI
+
BSD Now [BSD]
Podcasts I liked
I liked them but am not listening to them anymore. The podcasts have either "finished" (no more episodes) or I stopped listening to them due to time constraints or a shift in my interests.
-
Ship It (predecessor of Fork Around And Find Out)
-
Go Time (predecessor of fallthrough)
-
Modern Mentor
CRE: Chaosradio Express [german]
FLOSS weekly
Java Pub House
+
Ship It (predecessor of Fork Around And Find Out)
+
Modern Mentor
+
Go Time (predecessor of fallthrough)
Newsletters I like
This is a mix of tech and non-tech newsletters I am subscribed to. In random order:
-
-Update: 27.01.2026 I have since replaced the 1TB drives with 4TB drives for more storage capacity. The upgrade procedure was different for each node:
-
-
Upgrading f1 (simpler approach)
-
-Since f1 is the replication sink, the upgrade was straightforward:
-
-
-
1. Physically replaced the 1TB drive with the 4TB drive
-
2. Re-setup the drive as described earlier in this blog post
-
3. Re-replicated all data from f0 to f1 via zrepl
-
4. Reloaded the encryption keys as described in this blog post
-
5. Set the mount point again for the encrypted dataset, explicitly as read-only (since f1 is the replication sink)
-
-
Upgrading f0 (using ZFS resilvering)
-
-For f0, which is the primary storage node, I used ZFS resilvering to avoid data loss:
-
-
-
1. Plugged the new 4TB drive into an external USB SSD drive reader
-
2. Attached the 4TB drive to the zdata pool for resilvering
-
3. Once resilvering completed, detached the 1TB drive from the zdata pool
-
4. Shutdown f0 and physically replaced the internal drive
-
5. Booted with the new drive in place
-
6. Expanded the pool to use the full 4TB capacity:
-
-
-
paul@f0:~ % doas zpool online -e /dev/ada1
-
-
-
-
7. Reloaded the encryption keys as described in this blog post
-
8. Set the mount point again for the encrypted dataset
-
-This was a one-time effort on both nodes - after a reboot, everything was remembered and came up normally. Here are the updated outputs:
-
-
-
paul@f0:~ % doas zpool list
-NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
-zdata 3.63T 677G 2.97T - - 3% 18% 1.00x ONLINE -
-zroot 472G 68.4G 404G - - 13% 14% 1.00x ONLINE -
-
-paul@f0:~ % doas camcontrol devlist
-<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0)
-<SD Ultra 3D 4TB 530500WD> at scbus1 target 0 lun 0 (pass1,ada1)
-<Generic Flash Disk 8.07> at scbus2 target 0 lun 0 (da0,pass2)
-
-
-We're still using different SSD models on f1 (WD Blue SA510 4TB) to avoid simultaneous failures:
-
-
-
paul@f1:~ % doas camcontrol devlist
-<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0)
-<WD Blue SA510 2.5 4TB 530500WD> at scbus1 target 0 lun 0 (pass1,ada1)
-<Generic Flash Disk 8.07> at scbus2 target 0 lun 0 (da0,pass2)
-
-
ZFS encryption keys
ZFS native encryption requires encryption keys to unlock datasets. We need a secure method to store these keys that balances security with operational needs:
@@ -2097,6 +2030,73 @@ Jul 06 10:Applications should handle brief NFS errors gracefully
For zero-downtime requirements, consider synchronous replication or distributed storage (see "Future storage explorations" section later in this blog post)
+
Update: Upgrade to 4TB drives
+
+Update: 27.01.2026 I have since replaced the 1TB drives with 4TB drives for more storage capacity. The upgrade procedure was different for each node!
+
+
Upgrading f1 (simpler approach)
+
+Since f1 is the replication sink, the upgrade was straightforward:
+
+
+
1. Physically replaced the 1TB drive with the 4TB drive
+
2. Re-setup the drive as described earlier in this blog post
+
3. Re-replicated all data from f0 to f1 via zrepl
+
4. Reloaded the encryption keys as described in this blog post
+
5. Set the mount point again for the encrypted dataset, explicitly as read-only (since f1 is the replication sink)
+
+
Upgrading f0 (using ZFS resilvering)
+
+For f0, which is the primary storage node, I used ZFS resilvering to avoid data loss:
+
+
+
1. Plugged the new 4TB drive into an external USB SSD drive reader
+
2. Attached the 4TB drive to the zdata pool for resilvering
+
3. Once resilvering completed, detached the 1TB drive from the zdata pool
+
4. Shutdown f0 and physically replaced the internal drive
+
5. Booted with the new drive in place
+
6. Expanded the pool to use the full 4TB capacity:
+
+
+
paul@f0:~ % doas zpool online -e /dev/ada1
+
+
+
+
7. Reloaded the encryption keys as described in this blog post
+
8. Set the mount point again for the encrypted dataset
+
+This was a one-time effort on both nodes - after a reboot, everything was remembered and came up normally. Here are the updated outputs:
+
+
+
paul@f0:~ % doas zpool list
+NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
+zdata 3.63T 677G 2.97T - - 3% 18% 1.00x ONLINE -
+zroot 472G 68.4G 404G - - 13% 14% 1.00x ONLINE -
+
+paul@f0:~ % doas camcontrol devlist
+<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0)
+<SD Ultra 3D 4TB 530500WD> at scbus1 target 0 lun 0 (pass1,ada1)
+<Generic Flash Disk 8.07> at scbus2 target 0 lun 0 (da0,pass2)
+
+
+We're still using different SSD models on f1 (WD Blue SA510 4TB) to avoid simultaneous failures:
+
+
+
paul@f1:~ % doas camcontrol devlist
+<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0)
+<WD Blue SA510 2.5 4TB 530500WD> at scbus1 target 0 lun 0 (pass1,ada1)
+<Generic Flash Disk 8.07> at scbus2 target 0 lun 0 (da0,pass2)
+
+
Conclusion
We've built a robust, encrypted storage system for our FreeBSD-based Kubernetes cluster that provides:
diff --git a/gemfeed/atom.xml b/gemfeed/atom.xml
index eedbf0d9..5a7541e1 100644
--- a/gemfeed/atom.xml
+++ b/gemfeed/atom.xml
@@ -1,6 +1,6 @@
- 2026-01-27T09:57:01+02:00
+ 2026-01-27T10:09:14+02:00foo.zone feedTo be in the .zone!
@@ -6480,9 +6480,6 @@ content = "{CODE}"
-
-Update: 27.01.2026 I have since replaced the 1TB drives with 4TB drives for more storage capacity. The upgrade procedure was different for each node:
-
-
Upgrading f1 (simpler approach)
-
-Since f1 is the replication sink, the upgrade was straightforward:
-
-
-
1. Physically replaced the 1TB drive with the 4TB drive
-
2. Re-setup the drive as described earlier in this blog post
-
3. Re-replicated all data from f0 to f1 via zrepl
-
4. Reloaded the encryption keys as described in this blog post
-
5. Set the mount point again for the encrypted dataset, explicitly as read-only (since f1 is the replication sink)
-
-
Upgrading f0 (using ZFS resilvering)
-
-For f0, which is the primary storage node, I used ZFS resilvering to avoid data loss:
-
-
-
1. Plugged the new 4TB drive into an external USB SSD drive reader
-
2. Attached the 4TB drive to the zdata pool for resilvering
-
3. Once resilvering completed, detached the 1TB drive from the zdata pool
-
4. Shutdown f0 and physically replaced the internal drive
-
5. Booted with the new drive in place
-
6. Expanded the pool to use the full 4TB capacity:
-
-
-
paul@f0:~ % doas zpool online -e /dev/ada1
-
-
-
-
7. Reloaded the encryption keys as described in this blog post
-
8. Set the mount point again for the encrypted dataset
-
-This was a one-time effort on both nodes - after a reboot, everything was remembered and came up normally. Here are the updated outputs:
-
-
-
paul@f0:~ % doas zpool list
-NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
-zdata 3.63T 677G 2.97T - - 3% 18% 1.00x ONLINE -
-zroot 472G 68.4G 404G - - 13% 14% 1.00x ONLINE -
-
-paul@f0:~ % doas camcontrol devlist
-<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0)
-<SD Ultra 3D 4TB 530500WD> at scbus1 target 0 lun 0 (pass1,ada1)
-<Generic Flash Disk 8.07> at scbus2 target 0 lun 0 (da0,pass2)
-
-
-We're still using different SSD models on f1 (WD Blue SA510 4TB) to avoid simultaneous failures:
-
-
-
paul@f1:~ % doas camcontrol devlist
-<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0)
-<WD Blue SA510 2.5 4TB 530500WD> at scbus1 target 0 lun 0 (pass1,ada1)
-<Generic Flash Disk 8.07> at scbus2 target 0 lun 0 (da0,pass2)
-
-
ZFS encryption keys
ZFS native encryption requires encryption keys to unlock datasets. We need a secure method to store these keys that balances security with operational needs:
@@ -8543,6 +8476,73 @@ Jul 06 10:Applications should handle brief NFS errors gracefully
For zero-downtime requirements, consider synchronous replication or distributed storage (see "Future storage explorations" section later in this blog post)
+
Update: Upgrade to 4TB drives
+
+Update: 27.01.2026 I have since replaced the 1TB drives with 4TB drives for more storage capacity. The upgrade procedure was different for each node!
+
+
Upgrading f1 (simpler approach)
+
+Since f1 is the replication sink, the upgrade was straightforward:
+
+
+
1. Physically replaced the 1TB drive with the 4TB drive
+
2. Re-setup the drive as described earlier in this blog post
+
3. Re-replicated all data from f0 to f1 via zrepl
+
4. Reloaded the encryption keys as described in this blog post
+
5. Set the mount point again for the encrypted dataset, explicitly as read-only (since f1 is the replication sink)
+
+
Upgrading f0 (using ZFS resilvering)
+
+For f0, which is the primary storage node, I used ZFS resilvering to avoid data loss:
+
+
+
1. Plugged the new 4TB drive into an external USB SSD drive reader
+
2. Attached the 4TB drive to the zdata pool for resilvering
+
3. Once resilvering completed, detached the 1TB drive from the zdata pool
+
4. Shutdown f0 and physically replaced the internal drive
+
5. Booted with the new drive in place
+
6. Expanded the pool to use the full 4TB capacity:
+
+
+
paul@f0:~ % doas zpool online -e /dev/ada1
+
+
+
+
7. Reloaded the encryption keys as described in this blog post
+
8. Set the mount point again for the encrypted dataset
+
+This was a one-time effort on both nodes - after a reboot, everything was remembered and came up normally. Here are the updated outputs:
+
+
+
paul@f0:~ % doas zpool list
+NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
+zdata 3.63T 677G 2.97T - - 3% 18% 1.00x ONLINE -
+zroot 472G 68.4G 404G - - 13% 14% 1.00x ONLINE -
+
+paul@f0:~ % doas camcontrol devlist
+<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0)
+<SD Ultra 3D 4TB 530500WD> at scbus1 target 0 lun 0 (pass1,ada1)
+<Generic Flash Disk 8.07> at scbus2 target 0 lun 0 (da0,pass2)
+
+
+We're still using different SSD models on f1 (WD Blue SA510 4TB) to avoid simultaneous failures:
+
+
+
paul@f1:~ % doas camcontrol devlist
+<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0)
+<WD Blue SA510 2.5 4TB 530500WD> at scbus1 target 0 lun 0 (pass1,ada1)
+<Generic Flash Disk 8.07> at scbus2 target 0 lun 0 (da0,pass2)
+
+
Conclusion
We've built a robust, encrypted storage system for our FreeBSD-based Kubernetes cluster that provides:
diff --git a/index.html b/index.html
index d6985922..92246748 100644
--- a/index.html
+++ b/index.html
@@ -13,7 +13,7 @@
Hello!
-This site was generated at 2026-01-27T09:57:01+02:00 by Gemtexter
+This site was generated at 2026-01-27T10:09:14+02:00 by Gemtexter
-This site was last updated at 2026-01-27T09:57:00+02:00
+This site was last updated at 2026-01-27T10:09:14+02:00
The following stats were collected via uptimed on all of my personal computers over many years and the output was generated by guprecords, the global uptime records stats analyser of mine.