From 697343e53c9ea3dfdb8f37237b0189a9dd6c5324 Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Tue, 27 Jan 2026 10:10:19 +0200 Subject: Update content for gemtext --- ...25-07-14-f3s-kubernetes-with-freebsd-part-6.gmi | 116 ++++++++--------- gemfeed/atom.xml | 142 ++++++++++----------- 2 files changed, 129 insertions(+), 129 deletions(-) (limited to 'gemfeed') diff --git a/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi b/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi index 3a96cee9..0463ea36 100644 --- a/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi +++ b/gemfeed/2025-07-14-f3s-kubernetes-with-freebsd-part-6.gmi @@ -20,9 +20,6 @@ This is the sixth blog post about the f3s series for self-hosting demands in a h * ⇢ f3s: Kubernetes with FreeBSD - Part 6: Storage * ⇢ ⇢ Introduction * ⇢ ⇢ Additional storage capacity -* ⇢ ⇢ Update: Upgrade to 4TB drives -* ⇢ ⇢ ⇢ Upgrading f1 (simpler approach) -* ⇢ ⇢ ⇢ Upgrading f0 (using ZFS resilvering) * ⇢ ⇢ ZFS encryption keys * ⇢ ⇢ ⇢ UFS on USB keys * ⇢ ⇢ ⇢ Generating encryption keys @@ -65,6 +62,9 @@ This is the sixth blog post about the f3s series for self-hosting demands in a h * ⇢ ⇢ ⇢ Testing NFS Mount with Stunnel * ⇢ ⇢ ⇢ Testing CARP Failover with mounted clients and stale file handles: * ⇢ ⇢ ⇢ Complete Failover Test +* ⇢ ⇢ Update: Upgrade to 4TB drives +* ⇢ ⇢ ⇢ Upgrading f1 (simpler approach) +* ⇢ ⇢ ⇢ Upgrading f0 (using ZFS resilvering) * ⇢ ⇢ Conclusion * ⇢ ⇢ Future Storage Explorations * ⇢ ⇢ ⇢ MinIO for S3-Compatible Object Storage @@ -120,61 +120,6 @@ paul@f1:/ % doas camcontrol devlist at scbus1 target 0 lun 0 (pass1,ada1) ``` -## Update: Upgrade to 4TB drives - -> Update: 27.01.2026 I have since replaced the 1TB drives with 4TB drives for more storage capacity. The upgrade procedure was different for each node: - -### Upgrading f1 (simpler approach) - -Since f1 is the replication sink, the upgrade was straightforward: - -* 1. Physically replaced the 1TB drive with the 4TB drive -* 2. Re-setup the drive as described earlier in this blog post -* 3. Re-replicated all data from f0 to f1 via zrepl -* 4. Reloaded the encryption keys as described in this blog post -* 5. Set the mount point again for the encrypted dataset, explicitly as read-only (since f1 is the replication sink) - -### Upgrading f0 (using ZFS resilvering) - -For f0, which is the primary storage node, I used ZFS resilvering to avoid data loss: - -* 1. Plugged the new 4TB drive into an external USB SSD drive reader -* 2. Attached the 4TB drive to the zdata pool for resilvering -* 3. Once resilvering completed, detached the 1TB drive from the zdata pool -* 4. Shutdown f0 and physically replaced the internal drive -* 5. Booted with the new drive in place -* 6. Expanded the pool to use the full 4TB capacity: - -```sh -paul@f0:~ % doas zpool online -e /dev/ada1 -``` - -* 7. Reloaded the encryption keys as described in this blog post -* 8. Set the mount point again for the encrypted dataset - -This was a one-time effort on both nodes - after a reboot, everything was remembered and came up normally. Here are the updated outputs: - -```sh -paul@f0:~ % doas zpool list -NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT -zdata 3.63T 677G 2.97T - - 3% 18% 1.00x ONLINE - -zroot 472G 68.4G 404G - - 13% 14% 1.00x ONLINE - - -paul@f0:~ % doas camcontrol devlist -<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0) - at scbus1 target 0 lun 0 (pass1,ada1) - at scbus2 target 0 lun 0 (da0,pass2) -``` - -We're still using different SSD models on f1 (WD Blue SA510 4TB) to avoid simultaneous failures: - -```sh -paul@f1:~ % doas camcontrol devlist -<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0) - at scbus1 target 0 lun 0 (pass1,ada1) - at scbus2 target 0 lun 0 (da0,pass2) -``` - ## ZFS encryption keys ZFS native encryption requires encryption keys to unlock datasets. We need a secure method to store these keys that balances security with operational needs: @@ -1854,6 +1799,61 @@ Important Considerations: * Applications should handle brief NFS errors gracefully * For zero-downtime requirements, consider synchronous replication or distributed storage (see "Future storage explorations" section later in this blog post) +## Update: Upgrade to 4TB drives + +> Update: 27.01.2026 I have since replaced the 1TB drives with 4TB drives for more storage capacity. The upgrade procedure was different for each node! + +### Upgrading f1 (simpler approach) + +Since f1 is the replication sink, the upgrade was straightforward: + +* 1. Physically replaced the 1TB drive with the 4TB drive +* 2. Re-setup the drive as described earlier in this blog post +* 3. Re-replicated all data from f0 to f1 via zrepl +* 4. Reloaded the encryption keys as described in this blog post +* 5. Set the mount point again for the encrypted dataset, explicitly as read-only (since f1 is the replication sink) + +### Upgrading f0 (using ZFS resilvering) + +For f0, which is the primary storage node, I used ZFS resilvering to avoid data loss: + +* 1. Plugged the new 4TB drive into an external USB SSD drive reader +* 2. Attached the 4TB drive to the zdata pool for resilvering +* 3. Once resilvering completed, detached the 1TB drive from the zdata pool +* 4. Shutdown f0 and physically replaced the internal drive +* 5. Booted with the new drive in place +* 6. Expanded the pool to use the full 4TB capacity: + +```sh +paul@f0:~ % doas zpool online -e /dev/ada1 +``` + +* 7. Reloaded the encryption keys as described in this blog post +* 8. Set the mount point again for the encrypted dataset + +This was a one-time effort on both nodes - after a reboot, everything was remembered and came up normally. Here are the updated outputs: + +```sh +paul@f0:~ % doas zpool list +NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT +zdata 3.63T 677G 2.97T - - 3% 18% 1.00x ONLINE - +zroot 472G 68.4G 404G - - 13% 14% 1.00x ONLINE - + +paul@f0:~ % doas camcontrol devlist +<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0) + at scbus1 target 0 lun 0 (pass1,ada1) + at scbus2 target 0 lun 0 (da0,pass2) +``` + +We're still using different SSD models on f1 (WD Blue SA510 4TB) to avoid simultaneous failures: + +```sh +paul@f1:~ % doas camcontrol devlist +<512GB SSD D910R170> at scbus0 target 0 lun 0 (pass0,ada0) + at scbus1 target 0 lun 0 (pass1,ada1) + at scbus2 target 0 lun 0 (da0,pass2) +``` + ## Conclusion We've built a robust, encrypted storage system for our FreeBSD-based Kubernetes cluster that provides: diff --git a/gemfeed/atom.xml b/gemfeed/atom.xml index 932a5798..cc13b174 100644 --- a/gemfeed/atom.xml +++ b/gemfeed/atom.xml @@ -1,6 +1,6 @@ - 2026-01-27T09:57:01+02:00 + 2026-01-27T10:09:14+02:00 foo.zone feed To be in the .zone! @@ -6480,9 +6480,6 @@ content = "{CODE}"
  • f3s: Kubernetes with FreeBSD - Part 6: Storage
  • ⇢ Introduction
  • ⇢ Additional storage capacity
    • -
  • ⇢ Update: Upgrade to 4TB drives
    • -
  • ⇢ ⇢ Upgrading f1 (simpler approach)
    • -
  • ⇢ ⇢ Upgrading f0 (using ZFS resilvering)
  • ⇢ ZFS encryption keys
  • ⇢ ⇢ UFS on USB keys
  • ⇢ ⇢ Generating encryption keys
    • @@ -6525,6 +6522,9 @@ content = "{CODE}"
  • ⇢ ⇢ Testing NFS Mount with Stunnel
  • ⇢ ⇢ Testing CARP Failover with mounted clients and stale file handles:
  • ⇢ ⇢ Complete Failover Test
    • +
  • ⇢ Update: Upgrade to 4TB drives
    • +
  • ⇢ ⇢ Upgrading f1 (simpler approach)
    • +
  • ⇢ ⇢ Upgrading f0 (using ZFS resilvering)
  • ⇢ Conclusion
  • ⇢ Future Storage Explorations
  • ⇢ ⇢ MinIO for S3-Compatible Object Storage
    • @@ -6588,73 +6588,6 @@ http://www.gnu.org/software/src-highlite --> <CT1000BX500SSD1 M6CR072> at scbus1 target 0 lun 0 (pass1,ada1)
    -

    Update: Upgrade to 4TB drives


    -
    -Update: 27.01.2026 I have since replaced the 1TB drives with 4TB drives for more storage capacity. The upgrade procedure was different for each node:
    -
    -

    Upgrading f1 (simpler approach)


    -
    -Since f1 is the replication sink, the upgrade was straightforward:
    -
    -
      -
    • 1. Physically replaced the 1TB drive with the 4TB drive
      • -
    • 2. Re-setup the drive as described earlier in this blog post
      • -
    • 3. Re-replicated all data from f0 to f1 via zrepl
      • -
    • 4. Reloaded the encryption keys as described in this blog post
      • -
    • 5. Set the mount point again for the encrypted dataset, explicitly as read-only (since f1 is the replication sink)
      • -

    -

    Upgrading f0 (using ZFS resilvering)


    -
    -For f0, which is the primary storage node, I used ZFS resilvering to avoid data loss:
    -
    -
      -
    • 1. Plugged the new 4TB drive into an external USB SSD drive reader
      • -
    • 2. Attached the 4TB drive to the zdata pool for resilvering
      • -
    • 3. Once resilvering completed, detached the 1TB drive from the zdata pool
      • -
    • 4. Shutdown f0 and physically replaced the internal drive
      • -
    • 5. Booted with the new drive in place
      • -
    • 6. Expanded the pool to use the full 4TB capacity:
      • -

    - -
    paul@f0:~ % doas zpool online -e /dev/ada1
-
    -
    -
      -
    • 7. Reloaded the encryption keys as described in this blog post
      • -
    • 8. Set the mount point again for the encrypted dataset
      • -

    -This was a one-time effort on both nodes - after a reboot, everything was remembered and came up normally. Here are the updated outputs:
    -
    - -
    paul@f0:~ % doas zpool list
-NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
-zdata  3.63T   677G  2.97T        -         -     3%    18%  1.00x    ONLINE  -
-zroot   472G  68.4G   404G        -         -    13%    14%  1.00x    ONLINE  -
-
-paul@f0:~ % doas camcontrol devlist
-<512GB SSD D910R170>               at scbus0 target 0 lun 0 (pass0,ada0)
-<SD Ultra 3D 4TB 530500WD>         at scbus1 target 0 lun 0 (pass1,ada1)
-<Generic Flash Disk 8.07>          at scbus2 target 0 lun 0 (da0,pass2)
-
    -
    -We're still using different SSD models on f1 (WD Blue SA510 4TB) to avoid simultaneous failures:
    -
    - -
    paul@f1:~ % doas camcontrol devlist
-<512GB SSD D910R170>               at scbus0 target 0 lun 0 (pass0,ada0)
-<WD Blue SA510 2.5 4TB 530500WD>   at scbus1 target 0 lun 0 (pass1,ada1)
-<Generic Flash Disk 8.07>          at scbus2 target 0 lun 0 (da0,pass2)
-
    -

    ZFS encryption keys



    ZFS native encryption requires encryption keys to unlock datasets. We need a secure method to store these keys that balances security with operational needs:
    @@ -8543,6 +8476,73 @@ Jul 06 10:Applications should handle brief NFS errors gracefully
  • For zero-downtime requirements, consider synchronous replication or distributed storage (see "Future storage explorations" section later in this blog post)

    • +

    Update: Upgrade to 4TB drives


    +
    +Update: 27.01.2026 I have since replaced the 1TB drives with 4TB drives for more storage capacity. The upgrade procedure was different for each node!
    +
    +

    Upgrading f1 (simpler approach)


    +
    +Since f1 is the replication sink, the upgrade was straightforward:
    +
    +
      +
    • 1. Physically replaced the 1TB drive with the 4TB drive
      • +
    • 2. Re-setup the drive as described earlier in this blog post
      • +
    • 3. Re-replicated all data from f0 to f1 via zrepl
      • +
    • 4. Reloaded the encryption keys as described in this blog post
      • +
    • 5. Set the mount point again for the encrypted dataset, explicitly as read-only (since f1 is the replication sink)
      • +

    +

    Upgrading f0 (using ZFS resilvering)


    +
    +For f0, which is the primary storage node, I used ZFS resilvering to avoid data loss:
    +
    +
      +
    • 1. Plugged the new 4TB drive into an external USB SSD drive reader
      • +
    • 2. Attached the 4TB drive to the zdata pool for resilvering
      • +
    • 3. Once resilvering completed, detached the 1TB drive from the zdata pool
      • +
    • 4. Shutdown f0 and physically replaced the internal drive
      • +
    • 5. Booted with the new drive in place
      • +
    • 6. Expanded the pool to use the full 4TB capacity:
      • +

    + +
    paul@f0:~ % doas zpool online -e /dev/ada1
+
    +
    +
      +
    • 7. Reloaded the encryption keys as described in this blog post
      • +
    • 8. Set the mount point again for the encrypted dataset
      • +

    +This was a one-time effort on both nodes - after a reboot, everything was remembered and came up normally. Here are the updated outputs:
    +
    + +
    paul@f0:~ % doas zpool list
+NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
+zdata  3.63T   677G  2.97T        -         -     3%    18%  1.00x    ONLINE  -
+zroot   472G  68.4G   404G        -         -    13%    14%  1.00x    ONLINE  -
+
+paul@f0:~ % doas camcontrol devlist
+<512GB SSD D910R170>               at scbus0 target 0 lun 0 (pass0,ada0)
+<SD Ultra 3D 4TB 530500WD>         at scbus1 target 0 lun 0 (pass1,ada1)
+<Generic Flash Disk 8.07>          at scbus2 target 0 lun 0 (da0,pass2)
+
    +
    +We're still using different SSD models on f1 (WD Blue SA510 4TB) to avoid simultaneous failures:
    +
    + +
    paul@f1:~ % doas camcontrol devlist
+<512GB SSD D910R170>               at scbus0 target 0 lun 0 (pass0,ada0)
+<WD Blue SA510 2.5 4TB 530500WD>   at scbus1 target 0 lun 0 (pass1,ada1)
+<Generic Flash Disk 8.07>          at scbus2 target 0 lun 0 (da0,pass2)
+
    +

    Conclusion



    We've built a robust, encrypted storage system for our FreeBSD-based Kubernetes cluster that provides:
    -- cgit v1.2.3