diff options
| author | Paul Buetow <paul@buetow.org> | 2024-04-30 13:14:09 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2024-04-30 13:14:09 +0300 |
| commit | 07c56086aa0c4e015c9044e333ae4001debcb28d (patch) | |
| tree | 2b2c0a3dae32313d55c6f223095612dcf0b7d779 /gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html | |
| parent | 5ad16713db1f011c08d2db602ed5b1d3294f0939 (diff) | |
Update content for html
Diffstat (limited to 'gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html')
| -rw-r--r-- | gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html | 48 |
1 files changed, 25 insertions, 23 deletions
diff --git a/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html b/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html index 36c88618..ffd80d59 100644 --- a/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html +++ b/gemfeed/2024-04-01-KISS-high-availability-with-OpenBSD.html @@ -8,7 +8,7 @@ <link rel="stylesheet" href="style-override.css" /> </head> <body> -<h1 style='display: inline'>KISS high-availability with OpenBSD</h1><br /> +<h1 style='display: inline' id='KISShighavailabilitywithOpenBSD'>KISS high-availability with OpenBSD</h1><br /> <br /> <span class='quote'>Published at 2024-03-30T22:12:56+02:00</span><br /> <br /> @@ -38,18 +38,20 @@ _____|_:_:_| (o)-(o) |_:_:_|--'`-. ,--. ksh under-water (((\'/ </pre> <br /> <pre> -Table of contents: - KISS high-availability with OpenBSD - My auto-failover requirements - My HA solution - Only OpenBSD base installation required - Fairly cheap and geo-redundant - Failover time and split-brain - Failover support for multiple protocols - Let's encrypt TLS certificates - Monitoring - Rex automation - More HA +Table of contents +================= + +KISS high-availability with OpenBSD + My auto-failover requirements + My HA solution + Only OpenBSD base installation required + Fairly cheap and geo-redundant + Failover time and split-brain + Failover support for multiple protocols + Let's encrypt TLS certificates + Monitoring + Rex automation + More HA </pre> <br /> <span>I have always wanted a highly available setup for my personal websites. I could have used off-the-shelf hosting solutions or hosted my sites in an AWS S3 bucket. I have used technologies like (in unsorted and slightly unrelated order) BGP, LVS/IPVS, ldirectord, Pacemaker, STONITH, scripted VIP failover via ARP, heartbeat, heartbeat2, Corosync, keepalived, DRBD, and commercial F5 Load Balancers for high availability at work. </span><br /> @@ -60,7 +62,7 @@ Table of contents: <br /> <span class='quote'>PS: ASCII-art reflects an OpenBSD under-water world with all the tools available in the base system.</span><br /> <br /> -<h2 style='display: inline'>My auto-failover requirements</h2><br /> +<h2 style='display: inline' id='Myautofailoverrequirements'>My auto-failover requirements</h2><br /> <br /> <ul> <li>Be OpenBSD-based (I prefer OpenBSD because of the cleanliness and good documentation) and rely on as few external packages as possible. </li> @@ -73,9 +75,9 @@ Table of contents: <li>Have good monitoring in place so I know when a failover was performed and when something went wrong with the failover.</li> <li>Don't configure everything manually. The configuration should be automated and reproducible.</li> </ul><br /> -<h2 style='display: inline'>My HA solution</h2><br /> +<h2 style='display: inline' id='MyHAsolution'>My HA solution</h2><br /> <br /> -<h3 style='display: inline'>Only OpenBSD base installation required</h3><br /> +<h3 style='display: inline' id='OnlyOpenBSDbaseinstallationrequired'>Only OpenBSD base installation required</h3><br /> <br /> <span>My HA solution for Web and Gemini is based on DNS (OpenBSD's <span class='inlinecode'>nsd</span>) and a simple shell script (OpenBSD's <span class='inlinecode'>ksh</span> and some little <span class='inlinecode'>sed</span> and <span class='inlinecode'>awk</span> and <span class='inlinecode'>grep</span>). All software used here is part of the OpenBSD base system and no external package needs to be installed - OpenBSD is a complete operating system.</span><br /> <br /> @@ -231,7 +233,7 @@ echo <font color="#FF0000">"Failover of zone $zone to $MASTER completed"</font> <br /> <a class='textlink' href='https://codeberg.org/snonux/rexfiles/src/branch/master/frontends/scripts/dns-failover.ksh'>dns-failover.ksh</a><br /> <br /> -<h3 style='display: inline'>Fairly cheap and geo-redundant</h3><br /> +<h3 style='display: inline' id='Fairlycheapandgeoredundant'>Fairly cheap and geo-redundant</h3><br /> <br /> <span>I am renting two small OpenBSD VMs: One at OpenBSD Amsterdam and the other at Hetzner Cloud. So, both VMs are hosted at another provider, in different IP subnets, and in different countries (the Netherlands and Germany).</span><br /> <br /> @@ -242,13 +244,13 @@ echo <font color="#FF0000">"Failover of zone $zone to $MASTER completed"</font> <br /> <span>A DNS-based failover is cheap, as there isn't any BGP or fancy load balancer to pay for. Small VMs also cost less than millions.</span><br /> <br /> -<h3 style='display: inline'>Failover time and split-brain</h3><br /> +<h3 style='display: inline' id='Failovertimeandsplitbrain'>Failover time and split-brain</h3><br /> <br /> <span>A DNS failover doesn't happen immediately. I've configured a DNS TTL of <span class='inlinecode'>300</span> seconds, and the failover script checks once per minute whether to perform a failover or not. So, in total, a failover can take six minutes (not including other DNS caching servers somewhere in the interweb, but that's fine - eventually, all requests will resolve to the new master after a failover).</span><br /> <br /> <span>A split-brain scenario between the old master and the new master might happen. That's OK, as my sites are static, and there's no database to synchronise other than HTML, CSS, and images when the site is updated.</span><br /> <br /> -<h3 style='display: inline'>Failover support for multiple protocols</h3><br /> +<h3 style='display: inline' id='Failoversupportformultipleprotocols'>Failover support for multiple protocols</h3><br /> <br /> <span>With the DNS failover, HTTP, HTTPS, and Gemini protocols are failovered. This works because all domain virtual hosts are configured on either VM's <span class='inlinecode'>httpd</span> (OpenBSD's HTTP server) and <span class='inlinecode'>relayd</span> (it's also part of OpenBSD and I use it to TLS offload the Gemini protocol). So, both VMs accept requests for all the hosts. It's just a matter of the DNS entries, which VM receives the requests.</span><br /> <br /> @@ -259,7 +261,7 @@ echo <font color="#FF0000">"Failover of zone $zone to $MASTER completed"</font> <br /> <span>On DNS failover, master and standby swap roles without config changes other than the DNS entries. That's KISS (keep it simple and stupid)!</span><br /> <br /> -<h3 style='display: inline'>Let's encrypt TLS certificates</h3><br /> +<h3 style='display: inline' id='LetsencryptTLScertificates'>Let's encrypt TLS certificates</h3><br /> <br /> <span>All my hosts use TLS certificates from Let's Encrypt. The ACME automation for requesting and keeping the certificates valid (up to date) requires that the host requesting a certificate from Let's Encrypt is also the host using that certificate.</span><br /> <br /> @@ -296,7 +298,7 @@ http://www.gnu.org/software/src-highlite --> <a class='textlink' href='https://man.OpenBSD.org/acme-client.1'>https://man.OpenBSD.org/acme-client.1</a><br /> <a class='textlink' href='./2022-07-30-lets-encrypt-with-openbsd-and-rex.html'>Let's Encrypt with OpenBSD and Rex</a><br /> <br /> -<h3 style='display: inline'>Monitoring</h3><br /> +<h3 style='display: inline' id='Monitoring'>Monitoring</h3><br /> <br /> <span>CRON is sending me an E-Mail whenever a failover is performed (or whenever a failover failed). Furthermore, I am monitoring my DNS servers and hosts through Gogios, the monitoring system I have developed. </span><br /> <br /> @@ -305,7 +307,7 @@ http://www.gnu.org/software/src-highlite --> <br /> <span>Gogios, as I developed it by myself, isn't part of the OpenBSD base system. </span><br /> <br /> -<h3 style='display: inline'>Rex automation</h3><br /> +<h3 style='display: inline' id='Rexautomation'>Rex automation</h3><br /> <br /> <span>I use Rexify, a friendly configuration management system that allows automatic deployment and configuration.</span><br /> <br /> @@ -314,7 +316,7 @@ http://www.gnu.org/software/src-highlite --> <br /> <span>Rex isn't part of the OpenBSD base system, but I didn't need to install any external software on OpenBSD either as Rex is invoked from my Laptop!</span><br /> <br /> -<h2 style='display: inline'>More HA</h2><br /> +<h2 style='display: inline' id='MoreHA'>More HA</h2><br /> <br /> <span>Other high-available services running on my OpenBSD VMs are my MTAs for mail forwarding (OpenSMTPD - also part of the OpenBSD base system) and the authoritative DNS servers (<span class='inlinecode'>nsd</span>) for all my domains. No particular HA setup is required, though, as the protocols (SMTP and DNS) already take care of the failover to the next available host! </span><br /> <br /> |