diff options
Diffstat (limited to 'gemfeed/DRAFT-ipv6test-deployment.html')
| -rw-r--r-- | gemfeed/DRAFT-ipv6test-deployment.html | 322 |
1 files changed, 322 insertions, 0 deletions
diff --git a/gemfeed/DRAFT-ipv6test-deployment.html b/gemfeed/DRAFT-ipv6test-deployment.html new file mode 100644 index 00000000..9bc33ffb --- /dev/null +++ b/gemfeed/DRAFT-ipv6test-deployment.html @@ -0,0 +1,322 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> +<head> +<meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> +<title>Deploying an IPv6 Test Service on Kubernetes</title> +<link rel="shortcut icon" type="image/gif" href="/favicon.ico" /> +<link rel="stylesheet" href="../style.css" /> +<link rel="stylesheet" href="style-override.css" /> +</head> +<body> +<p class="header"> +<a href="https://foo.zone">Home</a> | <a href="https://codeberg.org/snonux/foo.zone/src/branch/content-md/gemfeed/DRAFT-ipv6test-deployment.md">Markdown</a> | <a href="gemini://foo.zone/gemfeed/DRAFT-ipv6test-deployment.gmi">Gemini</a> +</p> +<h1 style='display: inline' id='deploying-an-ipv6-test-service-on-kubernetes'>Deploying an IPv6 Test Service on Kubernetes</h1><br /> +<br /> +<h2 style='display: inline' id='introduction'>Introduction</h2><br /> +<br /> +<span>This post covers deploying a simple IPv6/IPv4 connectivity test application to the f3s Kubernetes cluster. The application displays visitors' IP addresses and determines whether they're connecting via IPv6 or IPv4—useful for testing dual-stack connectivity.</span><br /> +<br /> +<span>The interesting technical challenge was preserving the original client IP address through multiple reverse proxies: from the OpenBSD relayd frontends, through Traefik ingress, to the Apache CGI backend.</span><br /> +<br /> +<a class='textlink' href='./2024-11-17-f3s-kubernetes-with-freebsd-part-1.html'>f3s series</a><br /> +<br /> +<h2 style='display: inline' id='architecture-overview'>Architecture Overview</h2><br /> +<br /> +<span>The request flow looks like this:</span><br /> +<br /> +<pre> +Client → relayd (OpenBSD) → Traefik (k3s) → Apache + CGI (Pod) +</pre> +<br /> +<span>Each hop needs to preserve the client's real IP address via the <span class='inlinecode'>X-Forwarded-For</span> header.</span><br /> +<br /> +<h2 style='display: inline' id='the-application'>The Application</h2><br /> +<br /> +<span>The application is a simple Perl CGI script that:</span><br /> +<br /> +<span>1. Detects whether the client is using IPv4 or IPv6</span><br /> +<span>2. Performs DNS lookups on client and server addresses</span><br /> +<span>3. Displays diagnostic information</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><i><font color="silver">#!/usr/bin/perl</font></i> +<b><u><font color="#000000">use</font></u></b> strict; +<b><u><font color="#000000">use</font></u></b> warnings; + +<b><u><font color="#000000">print</font></u></b> <font color="#808080">"Content-type: text/html\n\n"</font>; + +<b><u><font color="#000000">my</font></u></b> $is_ipv4 = ($ENV{REMOTE_ADDR} =~ <font color="#808080">/(?:\d+\.){3}\d/</font>); +<b><u><font color="#000000">print</font></u></b> <font color="#808080">"You are using: "</font> . ($is_ipv4 ? <font color="#808080">"IPv4"</font> : <font color="#808080">"IPv6"</font>) . <font color="#808080">"\n"</font>; +<b><u><font color="#000000">print</font></u></b> <font color="#808080">"Client address: $ENV{REMOTE_ADDR}\n"</font>; +</pre> +<br /> +<h2 style='display: inline' id='docker-image'>Docker Image</h2><br /> +<br /> +<span>The Docker image uses Apache httpd with CGI and <span class='inlinecode'>mod_remoteip</span> enabled:</span><br /> +<br /> +<pre> +FROM httpd:2.4-alpine + +RUN apk add --no-cache perl bind-tools + +# Enable CGI and remoteip modules +RUN sed -i 's/#LoadModule cgid_module/LoadModule cgid_module/' \ + /usr/local/apache2/conf/httpd.conf && \ + sed -i 's/#LoadModule remoteip_module/LoadModule remoteip_module/' \ + /usr/local/apache2/conf/httpd.conf && \ + echo 'RemoteIPHeader X-Forwarded-For' >> /usr/local/apache2/conf/httpd.conf && \ + echo 'RemoteIPInternalProxy 10.0.0.0/8' >> /usr/local/apache2/conf/httpd.conf && \ + echo 'RemoteIPInternalProxy 192.168.0.0/16' >> /usr/local/apache2/conf/httpd.conf + +COPY index.pl /usr/local/apache2/cgi-bin/index.pl +</pre> +<br /> +<span>The key is <span class='inlinecode'>mod_remoteip</span>: it reads the <span class='inlinecode'>X-Forwarded-For</span> header and sets <span class='inlinecode'>REMOTE_ADDR</span> to the original client IP. The <span class='inlinecode'>RemoteIPInternalProxy</span> directives tell Apache which upstream proxies to trust.</span><br /> +<br /> +<h2 style='display: inline' id='kubernetes-deployment'>Kubernetes Deployment</h2><br /> +<br /> +<span>The Helm chart is straightforward:</span><br /> +<br /> +<pre> +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ipv6test + namespace: services +spec: + replicas: 1 + selector: + matchLabels: + app: ipv6test + template: + spec: + containers: + - name: ipv6test + image: registry.lan.buetow.org:30001/ipv6test:1.1.0 + ports: + - containerPort: 80 +</pre> +<br /> +<h2 style='display: inline' id='configuring-traefik-to-trust-forwarded-headers'>Configuring Traefik to Trust Forwarded Headers</h2><br /> +<br /> +<span>By default, Traefik overwrites <span class='inlinecode'>X-Forwarded-For</span> with its own view of the client IP (which is the upstream proxy, not the real client). To preserve the original header, Traefik needs to trust the upstream proxies.</span><br /> +<br /> +<span>In k3s, this is configured via a HelmChartConfig:</span><br /> +<br /> +<pre> +apiVersion: helm.cattle.io/v1 +kind: HelmChartConfig +metadata: + name: traefik + namespace: kube-system +spec: + valuesContent: |- + additionalArguments: + - "--entryPoints.web.forwardedHeaders.trustedIPs=192.168.0.0/16,10.0.0.0/8" + - "--entryPoints.websecure.forwardedHeaders.trustedIPs=192.168.0.0/16,10.0.0.0/8" +</pre> +<br /> +<span>This tells Traefik to trust <span class='inlinecode'>X-Forwarded-For</span> headers from the WireGuard tunnel IPs (where relayd connects from) and internal pod networks.</span><br /> +<br /> +<h2 style='display: inline' id='relayd-configuration'>Relayd Configuration</h2><br /> +<br /> +<span>The OpenBSD relayd proxy already sets the <span class='inlinecode'>X-Forwarded-For</span> header:</span><br /> +<br /> +<pre> +http protocol "https" { + match request header set "X-Forwarded-For" value "$REMOTE_ADDR" + match request header set "X-Forwarded-Proto" value "https" +} +</pre> +<br /> +<h2 style='display: inline' id='ipv4-only-and-ipv6-only-subdomains'>IPv4-Only and IPv6-Only Subdomains</h2><br /> +<br /> +<span>To properly test IPv4 and IPv6 connectivity separately, three hostnames are configured:</span><br /> +<br /> +<ul> +<li>ipv6test.f3s.buetow.org - Dual stack (A + AAAA records)</li> +<li>ipv4.ipv6test.f3s.buetow.org - IPv4 only (A record only)</li> +<li>ipv6.ipv6test.f3s.buetow.org - IPv6 only (AAAA record only)</li> +</ul><br /> +<span>The NSD zone template dynamically generates the correct record types:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><% <b><u><font color="#000000">for</font></u></b> <b><u><font color="#000000">my</font></u></b> $host (@$f3s_hosts) { + <b><u><font color="#000000">my</font></u></b> $is_ipv6_only = $host =~ <font color="#808080">/^ipv6\./</font>; + <b><u><font color="#000000">my</font></u></b> $is_ipv4_only = $host =~ <font color="#808080">/^ipv4\./</font>; +-%> +<font color="#808080"><% unless ($is_ipv6_only) { -%></font> +<font color="#808080"><%= $host %></font>. <font color="#000000">300</font> IN A <font color="#808080"><%= $ips-></font>{current_master}{ipv4} %> +<font color="#808080"><% } -%></font> +<font color="#808080"><% unless ($is_ipv4_only) { -%></font> +<font color="#808080"><%= $host %></font>. <font color="#000000">300</font> IN AAAA <font color="#808080"><%= $ips-></font>{current_master}{ipv6} %> +<font color="#808080"><% } -%></font> +<font color="#808080"><% } -%></font> +</pre> +<br /> +<span>This ensures:</span><br /> +<ul> +<li>Hosts starting with <span class='inlinecode'>ipv6.</span> get only AAAA records</li> +<li>Hosts starting with <span class='inlinecode'>ipv4.</span> get only A records</li> +<li>All other hosts get both A and AAAA records</li> +</ul><br /> +<span>The Kubernetes ingress handles all three hostnames, routing to the same backend service.</span><br /> +<br /> +<h2 style='display: inline' id='tls-certificates-with-subject-alternative-names'>TLS Certificates with Subject Alternative Names</h2><br /> +<br /> +<span>Since Let's Encrypt validates domains via HTTP, the IPv6-only subdomain (<span class='inlinecode'>ipv6.ipv6test.f3s.buetow.org</span>) cannot be validated directly—Let's Encrypt's validation servers use IPv4. The solution is to include all subdomains as Subject Alternative Names (SANs) in the parent certificate.</span><br /> +<br /> +<span>The ACME client configuration template dynamically builds the SAN list:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><% <b><u><font color="#000000">for</font></u></b> <b><u><font color="#000000">my</font></u></b> $host (@$acme_hosts) { + <i><font color="silver"># Skip ipv4/ipv6 subdomains - they're included as SANs in parent cert</font></i> + <b><u><font color="#000000">next</font></u></b> <b><u><font color="#000000">if</font></u></b> $host =~ <font color="#808080">/^(ipv4|ipv6)\./</font>; +-%> +<% <b><u><font color="#000000">my</font></u></b> @alt_names = (<font color="#808080">"www.$host"</font>); + <b><u><font color="#000000">for</font></u></b> <b><u><font color="#000000">my</font></u></b> $sub_host (@$acme_hosts) { + <b><u><font color="#000000">if</font></u></b> ($sub_host =~ <font color="#808080">/^(ipv4|ipv6)\.\Q$host\E$/</font>) { + <b><u><font color="#000000">push</font></u></b> @alt_names, $sub_host; + } + } +-%> +domain <font color="#808080"><%= $host %></font> { + alternative names { <font color="#808080"><%= join(' ', @alt_names) %></font> } + ... +} +<font color="#808080"><% } -%></font> +</pre> +<br /> +<span>This generates a single certificate for <span class='inlinecode'>ipv6test.f3s.buetow.org</span> that includes:</span><br /> +<ul> +<li>www.ipv6test.f3s.buetow.org</li> +<li>ipv4.ipv6test.f3s.buetow.org</li> +<li>ipv6.ipv6test.f3s.buetow.org</li> +</ul><br /> +<h2 style='display: inline' id='dns-and-tls-deployment'>DNS and TLS Deployment</h2><br /> +<br /> +<span>The DNS records and ACME certificates are managed via Rex automation:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><b><u><font color="#000000">our</font></u></b> @f3s_hosts = <b><u><font color="#000000">qw</font></u></b>/ + ... + ipv6test.f3s.buetow.org + ipv4.ipv6test.f3s.buetow.org + ipv6.ipv6test.f3s.buetow.org +/; + +<b><u><font color="#000000">our</font></u></b> @acme_hosts = <b><u><font color="#000000">qw</font></u></b>/ + ... + ipv6test.f3s.buetow.org + ipv4.ipv6test.f3s.buetow.org + ipv6.ipv6test.f3s.buetow.org +/; +</pre> +<br /> +<span>Running <span class='inlinecode'>rex nsd httpd acme acme_invoke relayd</span> deploys the DNS zone, configures httpd for ACME challenges, obtains the certificates, and reloads relayd.</span><br /> +<br /> +<h2 style='display: inline' id='testing'>Testing</h2><br /> +<br /> +<span>Verify DNS records are correct:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>$ dig ipv4.ipv6test.f3s.buetow.org A +short +<font color="#000000">46.23</font>.<font color="#000000">94.99</font> + +$ dig ipv4.ipv6test.f3s.buetow.org AAAA +short +(no output - IPv4 only) + +$ dig ipv6.ipv6test.f3s.buetow.org AAAA +short +2a03:<font color="#000000">6000</font>:6f67:<font color="#000000">624</font>::<font color="#000000">99</font> + +$ dig ipv6.ipv6test.f3s.buetow.org A +short +(no output - IPv6 only) +</pre> +<br /> +<span>Verify the application shows the correct test type:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre>$ curl -s https://ipv<font color="#000000">4</font>.ipv6test.f3s.buetow.org/cgi-bin/index.pl | grep <font color="#808080">"Test Results"</font> +<h3>IPv4 Only Test Results:</h<font color="#000000">3</font>> +</pre> +<br /> +<span>The displayed IP should be the real client IP, not an internal cluster address.</span><br /> +<br /> +<h2 style='display: inline' id='w3c-compliant-html'>W3C Compliant HTML</h2><br /> +<br /> +<span>The CGI script generates valid HTML5 that passes W3C validation. Key considerations:</span><br /> +<br /> +<ul> +<li>Proper DOCTYPE, charset, and lang attributes</li> +<li>HTML-escaping command outputs (dig output contains <span class='inlinecode'><<>></span> characters)</li> +</ul><br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><b><u><font color="#000000">sub</font></u></b> html_escape { + <b><u><font color="#000000">my</font></u></b> $str = <b><u><font color="#000000">shift</font></u></b>; + $str =~ <b><u><font color="#000000">s</font></u></b>/&/&amp;/<b><u><font color="#000000">g</font></u></b>; + $str =~ <b><u><font color="#000000">s</font></u></b>/</&lt;/<b><u><font color="#000000">g</font></u></b>; + $str =~ <b><u><font color="#000000">s</font></u></b>/>/&gt;/<b><u><font color="#000000">g</font></u></b>; + <b><u><font color="#000000">return</font></u></b> $str; +} + +<b><u><font color="#000000">my</font></u></b> $digremote = html_escape(`dig -<b><u><font color="#000000">x</font></u></b> $ENV{REMOTE_ADDR}`); +</pre> +<br /> +<span>You can verify the output passes validation:</span><br /> +<br /> +<a class='textlink' href='https://validator.w3.org/nu/?doc=https%3A%2F%2Fipv6test.f3s.buetow.org%2Fcgi-bin%2Findex.pl'>W3C Validator</a><br /> +<br /> +<h2 style='display: inline' id='summary'>Summary</h2><br /> +<br /> +<span>Preserving client IP addresses through multiple reverse proxies requires configuration at each layer:</span><br /> +<br /> +<span>1. **relayd**: Sets <span class='inlinecode'>X-Forwarded-For</span> header</span><br /> +<span>2. **Traefik**: Trusts headers from known proxy IPs via <span class='inlinecode'>forwardedHeaders.trustedIPs</span></span><br /> +<span>3. **Apache**: Uses <span class='inlinecode'>mod_remoteip</span> to set <span class='inlinecode'>REMOTE_ADDR</span> from the header</span><br /> +<br /> +<span>Additional challenges solved:</span><br /> +<br /> +<ul> +<li>**TLS for IPv6-only hosts**: Use SANs to include all subdomains in a single certificate validated via the dual-stack parent domain</li> +<li>**W3C compliance**: HTML-escape all command outputs to handle special characters</li> +</ul><br /> +<span>The configuration is managed via GitOps with ArgoCD, including the Traefik HelmChartConfig.</span><br /> +<br /> +<a class='textlink' href='https://codeberg.org/snonux/ipv6test'>Source code</a><br /> +<a class='textlink' href='https://codeberg.org/snonux/conf/src/branch/master/f3s/ipv6test'>Kubernetes manifests</a><br /> +<a class='textlink' href='https://codeberg.org/snonux/conf/src/branch/master/f3s/traefik-config'>Traefik configuration</a><br /> +<br /> +<span>E-Mail your comments to paul@paulbias.net :-)</span><br /> +<br /> +<a class='textlink' href='./index.html'>← Back to the index</a><br /> +<p class="footer"> + Generated with <a href="https://codeberg.org/snonux/gemtexter">Gemtexter 3.0.1-develop</a> | + served by <a href="https://www.OpenBSD.org">OpenBSD</a>/<a href="https://man.openbsd.org/relayd.8">relayd(8)</a>+<a href="https://man.openbsd.org/httpd.8">httpd(8)</a> | + <a href="https://foo.zone/site-mirrors.html">Site Mirrors</a> + <br /> + Webring: <a href="https://shring.sh/foo.zone/previous">previous</a> | <a href="https://shring.sh">shring</a> | <a href="https://shring.sh/foo.zone/next">next</a> +</p> +</body> +</html> |