From e9fc52626b6f10197a8e1431e2271e6dd3f634ff Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Sun, 26 Jan 2025 23:22:24 +0200 Subject: Update content for html --- gemfeed/4 | 239 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 239 insertions(+) create mode 100644 gemfeed/4 (limited to 'gemfeed/4') diff --git a/gemfeed/4 b/gemfeed/4 new file mode 100644 index 00000000..f4c59789 --- /dev/null +++ b/gemfeed/4 @@ -0,0 +1,239 @@ +# f3s: Kubernetes with FreeBSD - Rocky Linux Bhyve VMs - Part 3 + +This is the third blog post about my f3s series for my self-hosting demands in my home lab. f3s? The "f" stands for FreeBSD, and the "3s" stands for k3s, the Kubernetes distribution we will use on FreeBSD-based physical machines. + +<< template::inline::index f3s-kubernetes-with-freebsd-part + +=> ./f3s-kubernetes-with-frhyveeebsd-part-1/f3slogo.png f3s logo + +<< template::inline::toc + +## Introduction + +In this blog post, we are going to install the Bhyve hypervisor. + +The FreeBSD Bhyve hypervisor is a lightweight, modern hypervisor that enables virtualization on FreeBSD systems. Bhyve's strengths include its minimal overhead, which allows it to achieve near-native performance for virtual machines. It is designed to be efficient and lightweight, leveraging the capabilities of the FreeBSD operating system for performance and network management. + +Bhyve supports running a variety of guest operating systems, including FreeBSD, Linux, and Windows, on hardware platforms that support hardware virtualization extensions (such as Intel VT-x or AMD-V). In our case, we are going to virtualize Rocky Linux, which later on in this series will be used to run k3s. + +## Basic Bhyve setup + +For the management of the Bhyve VMs, we are using `vm-bhyve`, a tool not part of the FreeBSD operating system but available as a ready-to-use package. It eases VM management and reduces a lot of the overhead. We also install the required package to make Bhyve work with the UEFI firmware. + +=> https://github.com/churchers/vm-bhyve + +The following commands are executed on all three hosts `f0`, `f1`, and `f2`, where `re0` is the name of the Ethernet interface (which may need to be adjusted if your hardware is different): + +```sh +paul@f0:~ % doas pkg install vm-bhyve bhyve-firmware +paul@f0:~ % doas sysrc vm_enable=YES +vm_enable: -> YES +paul@f0:~ % doas sysrc vm_dir=zfs:zroot/bhyve +vm_dir: -> zfs:zroot/bhyve +paul@f0:~ % doas zfs create zroot/bhyve +paul@f0:~ % doas vm init +paul@f0:~ % doas vm switch create public +paul@f0:~ % doas vm switch add public re0 +``` + +Bhyve stores all it's data in the `/bhyve` of the `zroot` ZFS pool: + +```sh +paul@f0:~ % zfs list | grep bhyve +zroot/bhyve 1.74M 453G 1.74M /zroot/bhyve +``` + +For convenience, we also create this symlink: + +```sh +paul@f0:~ % doas ln -s /zroot/bhyve/ /bhyve + +``` + +Now, Bhyve is ready to rumble, but no VMs are there yet: + +```sh +paul@f0:~ % doas vm list +NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE +``` + +## Rocky Linux VMs + +### ISO download + +We're going to install the Rocky Linux from the latest minimal iso: + +```sh +paul@f0:~ % doas vm iso \ + https://download.rockylinux.org/pub/rocky/9/isos/x86_64/Rocky-9.5-x86_64-minimal.iso +/zroot/bhyve/.iso/Rocky-9.5-x86_64-minimal.iso 1808 MB 4780 kBps 06m28s +paul@f0:/bhyve % doas vm create rocky +``` +### VM configuration + +The default configuration looks like this now: + +```sh +paul@f0:/bhyve/rocky % cat rocky.conf +loader="bhyveload" +cpu=1 +memory=256M +network0_type="virtio-net" +network0_switch="public" +disk0_type="virtio-blk" +disk0_name="disk0.img" +uuid="1c4655ac-c828-11ef-a920-e8ff1ed71ca0" +network0_mac="58:9c:fc:0d:13:3f" +``` + +Whereas the `uuid` and the `network0_mac` differ on each of the 3 hosts. + +but in order to make Rocky Linux boot it (plus some other adjustments, e.g. as I am intending to run the majority of the workload in the k3s cluster running on those linux VMs, I give them beefy specs like 4 CPU cores and 14GB RAM), I run `doas vm configure rocky` and modified it to: + +``` +guest="linux" +loader="uefi" +uefi_vars="yes" +cpu=4 +memory=14G +network0_type="virtio-net" +network0_switch="public" +disk0_type="virtio-blk" +disk0_name="disk0.img" +graphics="yes" +graphics_vga=io +uuid="1c45400b-c828-11ef-8871-e8ff1ed71cac" +network0_mac="58:9c:fc:0d:13:3f" +``` + +### VM installation + +To start the installer from the downloaded ISO, I run: + +```sh +paul@f0:~ % doas vm install rocky Rocky-9.5-x86_64-minimal.iso +Starting rocky + * found guest in /zroot/bhyve/rocky + * booting... + +paul@f0:/bhyve/rocky % doas vm list +NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE +rocky default uefi 4 14G 0.0.0.0:5900 No Locked (f0.lan.buetow.org) + +paul@f0:/bhyve/rocky % doas sockstat -4 | grep 5900 +root bhyve 6079 8 tcp4 *:5900 *:* +``` + +Port 5900 now also opened for VNC connections, so I connected to it with a VNC client and run through the installation dialogs. I'm sure this could be done unattended or more automated, there are only 3 VMs to install, and the automation doesn't seem worth it as we are doing it only once in a year or less often. + +### Increase of the disk image + +By default the VMs disk image is only 20G, which is a bit small for my purposes, so I stopped the VMs again and run `truncate` on the image file to enlarge them to 100G, and re-started the installation: + +```sh +paul@f0:/bhyve/rocky % doas vm stop rocky +paul@f0:/bhyve/rocky % doas truncate -s 100G disk0.img +paul@f0:/bhyve/rocky % doas vm install rocky Rocky-9.5-x86_64-minimal.iso +``` + +### Connect to VPN + +For the installation, I opened the VPN client on my Fedora laptop (GNOME comes with a simple VPN client) and ran through the base installation for each of the VMs manually. Again, I am sure this could have been automated a bit more, but there were just 3 VMs, and it wasn't worth the effort. The three VNC addresses of the VMs were: `vnc://f0:5900`, `vnc://f1:5900`, and `vnc://f0:5900`. + +I mostly selected the default settings (auto partitioning on the 100GB drive and a root user password). After the installation, the VMs were rebooted. + +## After install + +I performed the following steps for all 3 VMs. In the following, the examples are all executed on `f0` (bzw the VM `r0` running on `f0`): + +### VM auto-start after host reboot + +To automatically start the VM on the servers I added the following to the `rc.conf` on the FreeBSD hosts: + +```sh + +paul@f0:/bhyve/rocky % cat <>/etc/hosts +192.168.1.120 r0 r0.lan r0.lan.buetow.org +192.168.1.121 r1 r1.lan r1.lan.buetow.org +192.168.1.122 r2 r2.lan r2.lan.buetow.org +END +```` + +Whereas: + +* `192.168.1.120` is the IP of the VM itself (here: `r0.lan.buetow.org`) +* `192.168.1.1` is the address of my home router, which also does DNS. + +### Permitting root login + +As these VMs arent directly reachable via SSH from the internet, I enabled `root` login by adding a line with `PermitRootLogin yes` to `/etc/sshd/sshd_config`. + +Once done, I rebooted the VM by running `reboot` inside of the vm to test whether everything was configured and persisted correctly. + +After reboot, I copied my public key from my Laptop to the 3 VMs: + +```sh +% for i in 0 1 2; do ssh-copy-id root@r$i.lan.buetow.org; done +``` + +And then I edited the `/etc/ssh/sshd_config` file again on all 3 VMs and configured `PasswordAuthentication no`, to only allow SSH key authentication from now on. + +### Install latest updates + +```sh +[root@r0 ~] % dnf update +[root@r0 ~] % dreboot +``` + +CPU STRESS TESTER VM VS NOT VM + +Other *BSD-related posts: + +<< template::inline::index bsd + +E-Mail your comments to `paul@nospam.buetow.org` :-) + +=> ../ Back to the main site -- cgit v1.2.3