summaryrefslogtreecommitdiff
path: root/gemfeed
diff options
context:
space:
mode:
authorPaul Buetow <paul@buetow.org>2025-04-04 22:00:36 +0300
committerPaul Buetow <paul@buetow.org>2025-04-04 22:00:36 +0300
commitf495b636fd6e866a6eed2da06dcdcda8d56dcf6a (patch)
tree4fc2acde06ac50a34ea3ac4fd0dcfc1b95a15c4e /gemfeed
parent4a6a88c3a376dadbdfb51c75daaf806d3f509276 (diff)
another draft
Diffstat (limited to 'gemfeed')
-rw-r--r--gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-4.gmi.tpl68
1 files changed, 47 insertions, 21 deletions
diff --git a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-4.gmi.tpl b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-4.gmi.tpl
index 903a503b..104b797a 100644
--- a/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-4.gmi.tpl
+++ b/gemfeed/DRAFT-f3s-kubernetes-with-freebsd-part-4.gmi.tpl
@@ -14,13 +14,13 @@ In this blog post, we are going to install the Bhyve hypervisor.
The FreeBSD Bhyve hypervisor is a lightweight, modern hypervisor that enables virtualization on FreeBSD systems. Bhyve's strengths include its minimal overhead, which allows it to achieve near-native performance for virtual machines. It's efficient and lightweight, leveraging the capabilities of the FreeBSD operating system for performance and network management.
-Bhyve supports running a variety of guest operating systems, including FreeBSD, Linux, and Windows, on hardware platforms that support hardware virtualization extensions (such as Intel VT-x or AMD-V). In our case, we are going to virtualize Rocky Linux, which later on in this series will be used to run k3s.
+Bhyve supports running various guest operating systems, including FreeBSD, Linux, and Windows, on hardware platforms that support hardware virtualization extensions (such as Intel VT-x or AMD-V). In our case, we are going to virtualize Rocky Linux, which will later in this series be used to run k3s.
## Check for `POPCNT` CPU support
-POPCNT is a CPU instruction that counts the number of set bits (ones) in a binary number. In terms of CPU virtualization and Bhyve support for the POPCNT instruction is important because guest operating systems utilize this instruction to perform various tasks more efficiently. If the host CPU supports POPCNT, Bhyve can pass this capability to virtual machines to for better performance. Without POPCNT support, some applications might not run, or they might perform suboptimally in virtualized environments.
+POPCNT is a CPU instruction that counts the number of set bits (ones) in a binary number. CPU virtualization and Bhyve support for the POPCNT instruction are important because guest operating systems utilize this instruction to perform various tasks more efficiently. If the host CPU supports POPCNT, Bhyve can pass this capability to virtual machines for better performance. Without POPCNT support, some applications might not run or perform sub-optimally in virtualized environments.
-To check for `POPCNT` support, I run:
+To check for `POPCNT` support, run:
```sh
paul@f0:~ % dmesg | grep 'Features2=.*POPCNT'
@@ -33,7 +33,7 @@ So it's there! All good.
## Basic Bhyve setup
-For the management of the Bhyve VMs, we are using `vm-bhyve`, a tool not part of the FreeBSD operating system but available as a ready-to-use package. It eases VM management and reduces a lot of the overhead. We also install the required package to make Bhyve work with the UEFI firmware.
+For managing the Bhyve VMs, we are using `vm-bhyve`, a tool not part of the FreeBSD operating system but available as a ready-to-use package. It eases VM management and reduces a lot of overhead. We also install the required package to make Bhyve work with the UEFI firmware.
=> https://github.com/churchers/vm-bhyve
@@ -103,9 +103,9 @@ uuid="1c4655ac-c828-11ef-a920-e8ff1ed71ca0"
network0_mac="58:9c:fc:0d:13:3f"
```
-Whereas the `uuid` and the `network0_mac` differ on each of the 3 hosts.
+The `uuid` and the `network0_mac` differ on each host.
-but in order to make Rocky Linux boot it (plus some other adjustments, e.g. as I am intending to run the majority of the workload in the k3s cluster running on those linux VMs, I give them beefy specs like 4 CPU cores and 14GB RAM), I run `doas vm configure rocky` and modified it to:
+But to make Rocky Linux boot it (plus some other adjustments, e.g. as we intend to run the majority of the workload in the k3s cluster running on those Linux VMs, we give them beefy specs like 4 CPU cores and 14GB RAM). So we run `doas vm configure rocky` and modified it to:
```
guest="linux"
@@ -141,11 +141,11 @@ paul@f0:/bhyve/rocky % doas sockstat -4 | grep 5900
root bhyve 6079 8 tcp4 *:5900 *:*
```
-Port 5900 now also opened for VNC connections, so I connected to it with a VNC client and run through the installation dialogs. I'm sure this could be done unattended or more automated, there are only 3 VMs to install, and the automation doesn't seem worth it as we are doing it only once in a year or less often.
+Port 5900 now also opens for VNC connections, so I connected it with a VNC client and ran through the installation dialogues. This could be done unattended or more automated, but there are only 3 VMs to install, and the automation doesn't seem worth it as we do it only once a year or less often.
### Increase of the disk image
-By default the VMs disk image is only 20G, which is a bit small for my purposes, so I stopped the VMs again and run `truncate` on the image file to enlarge them to 100G, and re-started the installation:
+By default, the VM disk image is only 20G, which is a bit small for my purposes, so I stopped the VMs again, ran `truncate` on the image file to enlarge them to 100G, and re-started the installation:
```sh
paul@f0:/bhyve/rocky % doas vm stop rocky
@@ -155,17 +155,17 @@ paul@f0:/bhyve/rocky % doas vm install rocky Rocky-9.5-x86_64-minimal.iso
### Connect to VNC
-For the installation, I opened the VNC client on my Fedora laptop (GNOME comes with a simple VNC client) and ran through the base installation for each of the VMs manually. Again, I am sure this could have been automated a bit more, but there were just 3 VMs, and it wasn't worth the effort. The three VNC addresses of the VMs were: `vnc://f0:5900`, `vnc://f1:5900`, and `vnc://f0:5900`.
+For the installation, I opened the VNC client on my Fedora laptop (GNOME comes with a simple VNC client) and manually ran through the base installation for each of the VMs. Again, I am sure this could have been automated a bit more, but there were just three VMs, and it wasn't worth the effort. The three VNC addresses of the VMs were `vnc://f0:5900`, `vnc://f1:5900`, and `vnc://f0:5900`.
-I mostly selected the default settings (auto partitioning on the 100GB drive and a root user password). After the installation, the VMs were rebooted.
+I primarily selected the default settings (auto partitioning on the 100GB drive and a root user password). After the installation, the VMs were rebooted.
## After install
-I performed the following steps for all 3 VMs. In the following, the examples are all executed on `f0` (bzw the VM `r0` running on `f0`):
+I performed the following steps for all 3 VMs. In the following, the examples are all executed on `f0` (the VM `r0` running on `f0`):
### VM auto-start after host reboot
-To automatically start the VM on the servers I added the following to the `rc.conf` on the FreeBSD hosts:
+To automatically start the VM on the servers, I added the following to the `rc.conf` on the FreeBSD hosts:
```sh
@@ -174,7 +174,7 @@ vm_list="rocky"
vm_delay="5"
```
-The `vm_delay` isn't really required. It is used to wait 5 seconds before starting each VM, but as of now, there is only one VM per host. Maybe later, when there are more, this will be useful to have. After adding, there's now a `Yes` indicator in the `AUTO` column.
+The `vm_delay` isn't really required. It is used to wait 5 seconds before starting each VM, but there is currently only one VM per host. Maybe later, when there are more, this will be useful. After adding, there's now a `Yes` indicator in the `AUTO` column.
```sh
paul@f0:~ % doas vm list
@@ -184,7 +184,7 @@ rocky default uefi 4 14G 0.0.0.0:5900 Yes [1] Running (2063)
### Static IP configuration
-After that, I changed the network configuration of the VMs to be static (from DHCP) here. As per previous post of this series, the 3 FreeBSD hosts were already in my `/etc/hosts` file:
+After that, I changed the network configuration of the VMs to be static (from DHCP) here. As per the previous post of this series, the 3 FreeBSD hosts were already in my `/etc/hosts` file:
```
192.168.1.130 f0 f0.lan f0.lan.buetow.org
@@ -192,7 +192,7 @@ After that, I changed the network configuration of the VMs to be static (from DH
192.168.1.132 f2 f2.lan f2.lan.buetow.org
```
-For the Rocky VMs I added those to the FreeBSD hosts systems as well:
+For the Rocky VMs, I added those to the FreeBSD host systems as well:
```sh
paul@f0:/bhyve/rocky % cat <<END | doas tee -a /etc/hosts
@@ -202,12 +202,12 @@ paul@f0:/bhyve/rocky % cat <<END | doas tee -a /etc/hosts
END
```
-and configured the IPs accordingly on the VMs themselves by opening a root shell via RDP to the VMs and entering the following commands on each of the VMs:
+And configured the IPs accordingly on the VMs themselves by opening a root shell via RDP to the VMs and entering the following commands on each of the VMs:
```sh
[root@r0 ~] % dnmcli connection modify enp0s5 ipv4.address 192.168.1.120/24
[root@r0 ~] % dnmcli connection modify enp0s5 ipv4.gateway 192.168.1.1
-[root@r0 ~] % dnmcli connection modify enp0s5 ipv4.dns 192.168.1.1
+[root@r0 ~] % dnmcli connection modify enp0s5 ipv4.DNS 192.168.1.1
[root@r0 ~] % dnmcli connection modify enp0s5 ipv4.method manual
[root@r0 ~] % dnmcli connection down enp0s5
[root@r0 ~] % dnmcli connection up enp0s5
@@ -226,9 +226,9 @@ Whereas:
### Permitting root login
-As these VMs arent directly reachable via SSH from the internet, I enabled `root` login by adding a line with `PermitRootLogin yes` to `/etc/sshd/sshd_config`.
+As these VMs aren't directly reachable via SSH from the internet, I enabled `root` login by adding a line with `PermitRootLogin yes` to `/etc/sshd/sshd_config`.
-Once done, I rebooted the VM by running `reboot` inside of the vm to test whether everything was configured and persisted correctly.
+Once done, I rebooted the VM by running `reboot` inside the VM to test whether everything was configured and persisted correctly.
After reboot, I copied my public key from my Laptop to the 3 VMs:
@@ -236,7 +236,7 @@ After reboot, I copied my public key from my Laptop to the 3 VMs:
% for i in 0 1 2; do ssh-copy-id root@r$i.lan.buetow.org; done
```
-And then I edited the `/etc/ssh/sshd_config` file again on all 3 VMs and configured `PasswordAuthentication no`, to only allow SSH key authentication from now on.
+Then, I edited the `/etc/ssh/sshd_config` file again on all 3 VMs and configured `PasswordAuthentication no` to only allow SSH key authentication from now on.
### Install latest updates
@@ -331,7 +331,33 @@ cpu: Intel(R) N100
BenchmarkCPUSilly1-4 1000000000 0.4347 ns/op
BenchmarkCPUSilly2-4 1000000000 0.4345 ns/op
```
-The Linux benchmark seems to be slightly slower than the FreeBSD one. The Go version is also a bit older. I tried the same with the up-to-date version of Go (1.24.x) with similar results. It could be that there is a slight Bhyve overhead, or FreeBSD is just slightly more efficient in this benchmark. Overall, this shows that there is great performance in Bhyve nevertheless.
+The Linux benchmark is slightly slower than the FreeBSD one. The Go version is also a bit older. I tried the same with the up-to-date version of Go (1.24.x) with similar results. There could be a slight Bhyve overhead, or FreeBSD is just slightly more efficient in this benchmark. Overall, this shows that Bhyve performs excellently.
+
+But as I am curious and don't want to compare apples with bananas, I decided to install a FreeBSD Bhyve VM to run the same silly benchmark in it. I am not going through the details of how to install a FreeBSD Bhyve VM here; you can easily look it up in the documentation.
+
+But here are the results running the same silly benchmark in a FreeBSD Bhyve VM with the same FreeBSD and Go versions as the host system (I have the VM 4 vCPUs and 12GB of RAM; the benchmark won't use as many CPUs anyway):
+
+```sh
+root@freebsd:~/git/sillybench # go test -bench=.
+goos: freebsd
+goarch: amd64
+pkg: codeberg.org/snonux/sillybench
+cpu: Intel(R) N100
+BenchmarkCPUSilly1 1000000000 0.4273 ns/op
+BenchmarkCPUSilly2 1000000000 0.4286 ns/op
+PASS
+ok codeberg.org/snonux/sillybench 0.949s
+```
+
+It's a bit better than Linux! I am sure that this is not really a scientific benchmark, so take the results with a grain of salt!
+## Conclusion
+
+Having Linux VMs running inside FreeBSD's Bhyve is a solid move for future F3s hosting in my home lab. Bhyve provides a reliable way to manage VMs without much hassle. With Linux VMs, I can tap into all the cool stuff (e.g., Kubernetes) in the Linux world while keeping the steady reliability of FreeBSD.
+
+Future uses (out of scope for this blog series) would be additional VMs for different workloads. For example, how about a Windows VM or a NetBSD VM to tinker with?
+
+This flexibility is great for keeping options open and managing different kinds of workloads without overcomplicating things. Overall, it's a nice setup for getting the most out of my hardware and keeping things running smoothly.
+
Other *BSD-related posts: