diff options
Diffstat (limited to 'gemfeed/2022-10-30-installing-dtail-on-openbsd.html')
| -rw-r--r-- | gemfeed/2022-10-30-installing-dtail-on-openbsd.html | 167 |
1 files changed, 116 insertions, 51 deletions
diff --git a/gemfeed/2022-10-30-installing-dtail-on-openbsd.html b/gemfeed/2022-10-30-installing-dtail-on-openbsd.html index feebd9fd..53bd9b1c 100644 --- a/gemfeed/2022-10-30-installing-dtail-on-openbsd.html +++ b/gemfeed/2022-10-30-installing-dtail-on-openbsd.html @@ -8,8 +8,10 @@ <link rel="stylesheet" href="style-override.css" /> </head> <body> -<h1>Installing DTail on OpenBSD</h1> -<p class="quote"><i>Published at 2022-10-30T11:03:19+02:00</i></p> +<h1 style='display: inline'>Installing DTail on OpenBSD</h1><br /> +<br /> +<span class=quote>Published at 2022-10-30T11:03:19+02:00</span><br /> +<br /> <pre> ,_---~~~~~----._ _,,_,*^____ _____``*g*\"*, @@ -46,18 +48,29 @@ / `._____V_____V' ' ' </pre> -<p>This will be a quick blog post, as I am busy with my personal life now. I have relocated to a different country and am still busy arranging things. So bear with me :-)</p> -<p> In this post, I want to give a quick overview (or how-to) about installing DTail on OpenBSD, as the official documentation only covers Red Hat and Fedora Linux! And this blog post will also be used as my reference!</p> -<a class="textlink" href="https://dtail.dev">https://dtail.dev</a><br /> -<p>I am using Rexify for my OpenBSD automation. Check out the following article covering my Rex setup in a little bit more detail:</p> -<a class="textlink" href="./2022-07-30-lets-encrypt-with-openbsd-and-rex.html">Let's Encrypt with OpenBSD and Rex</a><br /> -<p>I will also mention some relevant <span class="inlinecode">Rexfile</span> snippets in this post!</p> -<h2>Compile it</h2> -<p>First of all, DTail needs to be downloaded and compiled. For that, <span class="inlinecode">git</span>, <span class="inlinecode">go</span>, and <span class="inlinecode">gmake</span> are required:</p> +<br /> +<span>This will be a quick blog post, as I am busy with my personal life now. I have relocated to a different country and am still busy arranging things. So bear with me :-)</span><br /> +<br /> +<span> In this post, I want to give a quick overview (or how-to) about installing DTail on OpenBSD, as the official documentation only covers Red Hat and Fedora Linux! And this blog post will also be used as my reference!</span><br /> +<br /> +<a class=textlink href='https://dtail.dev'>https://dtail.dev</a><br /> +<br /> +<span>I am using Rexify for my OpenBSD automation. Check out the following article covering my Rex setup in a little bit more detail:</span><br /> +<br /> +<a class=textlink href='./2022-07-30-lets-encrypt-with-openbsd-and-rex.html'>Let's Encrypt with OpenBSD and Rex</a><br /> +<br /> +<span>I will also mention some relevant <span class=inlinecode>Rexfile</span> snippets in this post!</span><br /> +<br /> +<h2 style='display: inline'>Compile it</h2><br /> +<br /> +<span>First of all, DTail needs to be downloaded and compiled. For that, <span class=inlinecode>git</span>, <span class=inlinecode>go</span>, and <span class=inlinecode>gmake</span> are required:</span><br /> +<br /> <pre> $ doas pkg_add git go gmake </pre> -<p>I am happy that the Go Programming Language is readily available in the OpenBSD packaging system. Once the dependencies got installed, clone DTail and compile it:</p> +<br /> +<span>I am happy that the Go Programming Language is readily available in the OpenBSD packaging system. Once the dependencies got installed, clone DTail and compile it:</span><br /> +<br /> <pre> $ mkdir git $ cd git @@ -65,32 +78,44 @@ $ git clone https://github.com/mimecast/dtail $ cd dtail $ gmake </pre> -<p>You can verify the version by running the following command:</p> +<br /> +<span>You can verify the version by running the following command:</span><br /> +<br /> <pre> $ ./dtail --version DTail 4.1.0 Protocol 4.1 Have a lot of fun! $ file dtail dtail: ELF 64-bit LSB executable, x86-64, version 1 </pre> -<p>Now, there isn't any need anymore to keep <span class="inlinecode">git</span>, <span class="inlinecode">go</span> and <span class="inlinecode">gmake</span>, so they can be deinstalled now:</p> +<br /> +<span>Now, there isn't any need anymore to keep <span class=inlinecode>git</span>, <span class=inlinecode>go</span> and <span class=inlinecode>gmake</span>, so they can be deinstalled now:</span><br /> +<br /> <pre> $ doas pkg_delete git go gmake </pre> -<p>One day I shall create an official OpenBSD port for DTail.</p> -<h2>Install it</h2> -<p>Installing the binaries is now just a matter of copying them to <span class="inlinecode">/usr/local/bin</span> as follows:</p> +<br /> +<span>One day I shall create an official OpenBSD port for DTail.</span><br /> +<br /> +<h2 style='display: inline'>Install it</h2><br /> +<br /> +<span>Installing the binaries is now just a matter of copying them to <span class=inlinecode>/usr/local/bin</span> as follows:</span><br /> +<br /> <pre> $ for bin in dserver dcat dgrep dmap dtail dtailhealth; do doas cp -p $bin /usr/local/bin/$bin doas chown root:wheel /usr/local/bin/$bin done </pre> -<p>Also, we will be creating the <span class="inlinecode">_dserver</span> service user:</p> +<br /> +<span>Also, we will be creating the <span class=inlinecode>_dserver</span> service user:</span><br /> +<br /> <pre> $ doas adduser -class nologin -group _dserver -batch _dserver $ doas usermod -d /var/run/dserver/ _dserver </pre> -<p>The OpenBSD init script is created from scratch (not part of the official DTail project). Run the following to install the bespoke script:</p> +<br /> +<span>The OpenBSD init script is created from scratch (not part of the official DTail project). Run the following to install the bespoke script:</span><br /> +<br /> <pre> $ cat <<'END' | doas tee /etc/rc.d/dserver #!/bin/ksh @@ -112,8 +137,11 @@ rc_cmd $1 & END $ doas chmod 755 /etc/rc.d/dserver </pre> -<h3>Rexification</h3> -<p>This is the task for setting it up via Rex. Note the <span class="inlinecode">. . . .</span>, that's a placeholder which we will fill up more and more during this blog post:</p> +<br /> +<h3 style='display: inline'>Rexification</h3><br /> +<br /> +<span>This is the task for setting it up via Rex. Note the <span class=inlinecode>. . . .</span>, that's a placeholder which we will fill up more and more during this blog post:</span><br /> +<br /> <pre> desc 'Setup DTail'; task 'dtail', group => 'frontends', @@ -136,14 +164,19 @@ task 'dtail', group => 'frontends', service 'dserver', ensure => 'started'; }; </pre> -<h2>Configure it</h2> -<p>Now, DTail is fully installed but still needs to be configured. Grab the default config file from GitHub ...</p> +<br /> +<h2 style='display: inline'>Configure it</h2><br /> +<br /> +<span>Now, DTail is fully installed but still needs to be configured. Grab the default config file from GitHub ...</span><br /> +<br /> <pre> $ doas mkdir /etc/dserver $ curl https://raw.githubusercontent.com/mimecast/dtail/master/samples/dtail.json.sample | doas tee /etc/dserver/dtail.json </pre> -<p>... and then edit it and adjust <span class="inlinecode">LogDir</span> in the <span class="inlinecode">Common</span> section to <span class="inlinecode">/var/log/dserver</span>. The result will look like this:</p> +<br /> +<span>... and then edit it and adjust <span class=inlinecode>LogDir</span> in the <span class=inlinecode>Common</span> section to <span class=inlinecode>/var/log/dserver</span>. The result will look like this:</span><br /> +<br /> <pre> "Common": { "LogDir": "/var/log/dserver", @@ -154,8 +187,11 @@ $ curl https://raw.githubusercontent.com/mimecast/dtail/master/samples/dtail.jso "LogLevel": "Info" } </pre> -<h3>Rexification</h3> -<p>That's as simple as adding the following to the Rex task:</p> +<br /> +<h3 style='display: inline'>Rexification</h3><br /> +<br /> +<span>That's as simple as adding the following to the Rex task:</span><br /> +<br /> <pre> file '/etc/dserver', ensure => 'directory'; @@ -167,9 +203,13 @@ file '/etc/dserver/dtail.json', mode => '755', on_change => sub { $restart = TRUE }; </pre> -<h2>Update the key cache for it</h2> -<p>DTail relies on SSH for secure authentication and communication. However, the system user <span class="inlinecode">_dserver</span> has no permission to read the SSH public keys from the user's home directories, so the DTail server also checks for available public keys in an alternative path <span class="inlinecode">/var/run/dserver/cache</span>. </p> -<p>The following script, populating the DTail server key cache, can be run periodically via <span class="inlinecode">CRON</span>:</p> +<br /> +<h2 style='display: inline'>Update the key cache for it</h2><br /> +<br /> +<span>DTail relies on SSH for secure authentication and communication. However, the system user <span class=inlinecode>_dserver</span> has no permission to read the SSH public keys from the user's home directories, so the DTail server also checks for available public keys in an alternative path <span class=inlinecode>/var/run/dserver/cache</span>. </span><br /> +<br /> +<span>The following script, populating the DTail server key cache, can be run periodically via <span class=inlinecode>CRON</span>:</span><br /> +<br /> <pre> $ cat <<'END' | doas tee /usr/local/bin/dserver-update-key-cache.sh #!/bin/ksh @@ -209,13 +249,18 @@ echo 'All set...' END $ doas chmod 500 /usr/local/bin/dserver-update-key-cache.sh </pre> -<p>Note that the script above is a slight variation of the official DTail script. The official DTail one is a <span class="inlinecode">bash</span> script, but on OpenBSD, there's <span class="inlinecode">ksh</span>. I run it once daily by adding it to the <span class="inlinecode">daily.local</span>:</p> +<br /> +<span>Note that the script above is a slight variation of the official DTail script. The official DTail one is a <span class=inlinecode>bash</span> script, but on OpenBSD, there's <span class=inlinecode>ksh</span>. I run it once daily by adding it to the <span class=inlinecode>daily.local</span>:</span><br /> +<br /> <pre> $ echo /usr/local/bin/dserver-update-key-cache.sh | doas tee -a /etc/daily.local /usr/local/bin/dserver-update-key-cache.sh </pre> -<h3>Rexification</h3> -<p>That's done by adding ...</p> +<br /> +<h3 style='display: inline'>Rexification</h3><br /> +<br /> +<span>That's done by adding ...</span><br /> +<br /> <pre> file '/usr/local/bin/dserver-update-key-cache.sh', content => template('./scripts/dserver-update-key-cache.sh.tpl'), @@ -225,9 +270,13 @@ file '/usr/local/bin/dserver-update-key-cache.sh', append_if_no_such_line '/etc/daily.local', '/usr/local/bin/dserver-update-key-cache.sh'; </pre> -<p>... to the Rex task!</p> -<h2>Start it</h2> -<p>Now, it's time to enable and start the DTail server:</p> +<br /> +<span>... to the Rex task!</span><br /> +<br /> +<h2 style='display: inline'>Start it</h2><br /> +<br /> +<span>Now, it's time to enable and start the DTail server:</span><br /> +<br /> <pre> $ sudo rcctl enable dserver $ sudo rcctl start dserver @@ -249,7 +298,9 @@ INFO|1022-090739|86050|stats.go:53|2|11|7|||MAPREDUCE:STATS|currentConnections=0 . Ctr+C </pre> -<p>As we don't want to wait until tomorrow, let's populate the key cache manually:</p> +<br /> +<span>As we don't want to wait until tomorrow, let's populate the key cache manually:</span><br /> +<br /> <pre> $ doas /usr/local/bin/dserver-update-key-cache.sh Updating SSH key cache @@ -261,9 +312,13 @@ Caching /home/paul/.ssh/authorized_keys -> /var/cache/dserver/paul.authorized Caching /home/rex/.ssh/authorized_keys -> /var/cache/dserver/rex.authorized_keys All set... </pre> -<h2>Use it</h2> -<p>The DTail server is now ready to serve connections. You can use any DTail commands, such as <span class="inlinecode">dtail</span>, <span class="inlinecode">dgrep</span>, <span class="inlinecode">dmap</span>, <span class="inlinecode">dcat</span>, <span class="inlinecode">dtailhealth</span>, to do so. Checkout out all the usage examples on the official DTail page.</p> -<p>I have installed DTail server this way on my personal OpenBSD frontends <span class="inlinecode">blowfish</span>, and <span class="inlinecode">fishfinger</span>, and the following command connects as user <span class="inlinecode">rex</span> to both machines and greps the file <span class="inlinecode">/etc/fstab</span> for the string <span class="inlinecode">local</span>:</p> +<br /> +<h2 style='display: inline'>Use it</h2><br /> +<br /> +<span>The DTail server is now ready to serve connections. You can use any DTail commands, such as <span class=inlinecode>dtail</span>, <span class=inlinecode>dgrep</span>, <span class=inlinecode>dmap</span>, <span class=inlinecode>dcat</span>, <span class=inlinecode>dtailhealth</span>, to do so. Checkout out all the usage examples on the official DTail page.</span><br /> +<br /> +<span>I have installed DTail server this way on my personal OpenBSD frontends <span class=inlinecode>blowfish</span>, and <span class=inlinecode>fishfinger</span>, and the following command connects as user <span class=inlinecode>rex</span> to both machines and greps the file <span class=inlinecode>/etc/fstab</span> for the string <span class=inlinecode>local</span>:</span><br /> +<br /> <pre> ❯ ./dgrep -user rex -servers blowfish.buetow.org,fishfinger.buetow.org --regex local /etc/fstab CLIENT|earth|WARN|Encountered unknown host|{blowfish.buetow.org:2222 0xc0000a00f0 0xc0000a61e0 [blowfish.buetow.org]:2222 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9ZnF/LAk14SgqCzk38yENVTNfqibcluMTuKx1u53cKSp2xwHWzy0Ni5smFPpJDIQQljQEJl14ZdXvhhjp1kKHxJ79ubqRtIXBlC0PhlnP8Kd+mVLLHYpH9VO4rnaSfHE1kBjWkI7U6lLc6ks4flgAgGTS5Bb7pLAjwdWg794GWcnRh6kSUEQd3SftANqQLgCunDcP2Vc4KR9R78zBmEzXH/OPzl/ANgNA6wWO2OoKKy2VrjwVAab6FW15h3Lr6rYIw3KztpG+UMmEj5ReexIjXi/jUptdnUFWspvAmzIl6kwzzF8ExVyT9D75JRuHvmxXKKjyJRxqb8UnSh2JD4JN [23.88.35.144]:2222 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9ZnF/LAk14SgqCzk38yENVTNfqibcluMTuKx1u53cKSp2xwHWzy0Ni5smFPpJDIQQljQEJl14ZdXvhhjp1kKHxJ79ubqRtIXBlC0PhlnP8Kd+mVLLHYpH9VO4rnaSfHE1kBjWkI7U6lLc6ks4flgAgGTS5Bb7pLAjwdWg794GWcnRh6kSUEQd3SftANqQLgCunDcP2Vc4KR9R78zBmEzXH/OPzl/ANgNA6wWO2OoKKy2VrjwVAab6FW15h3Lr6rYIw3KztpG+UMmEj5ReexIjXi/jUptdnUFWspvAmzIl6kwzzF8ExVyT9D75JRuHvmxXKKjyJRxqb8UnSh2JD4JN 0xc0000a2180} @@ -275,24 +330,34 @@ CLIENT|earth|INFO|Added hosts to known hosts file|/home/paul/.ssh/known_hosts REMOTE|blowfish|100|7|fstab|31bfd9d9a6788844.h /usr/local ffs rw,wxallowed,nodev 1 2 REMOTE|fishfinger|100|7|fstab|093f510ec5c0f512.h /usr/local ffs rw,wxallowed,nodev 1 2 </pre> -<p>Running it the second time, and given that you trusted the keys the first time, it won't prompt you for the host keys anymore:</p> +<br /> +<span>Running it the second time, and given that you trusted the keys the first time, it won't prompt you for the host keys anymore:</span><br /> +<br /> <pre> ❯ ./dgrep -user rex -servers blowfish.buetow.org,fishfinger.buetow.org --regex local /etc/fstab REMOTE|blowfish|100|7|fstab|31bfd9d9a6788844.h /usr/local ffs rw,wxallowed,nodev 1 2 REMOTE|fishfinger|100|7|fstab|093f510ec5c0f512.h /usr/local ffs rw,wxallowed,nodev 1 2 </pre> -<h2>Conclusions</h2> -<p>It's a bit of manual work, but it's ok on this small scale! I shall invest time in creating an official OpenBSD port, though. That would render most of the manual steps obsolete, as outlined in this post!</p> -<p>Check out the following for more information:</p> -<a class="textlink" href="https://dtail.dev">https://dtail.dev</a><br /> -<a class="textlink" href="https://github.com/mimecast/dtail">https://github.com/mimecast/dtail</a><br /> -<a class="textlink" href="https://www.rexify.org">https://www.rexify.org</a><br /> -<p>Other related posts are:</p> -<a class="textlink" href="./2022-10-30-installing-dtail-on-openbsd.html">2022-10-30 Installing DTail on OpenBSD (You are currently reading this)</a><br /> -<a class="textlink" href="./2022-03-06-the-release-of-dtail-4.0.0.html">2022-03-06 The release of DTail 4.0.0</a><br /> -<a class="textlink" href="./2021-04-22-dtail-the-distributed-log-tail-program.html">2021-04-22 DTail - The distributed log tail program</a><br /> -<p>E-Mail your comments to hi@paul.cyou :-)</p> -<a class="textlink" href="../">Back to the main site</a><br /> +<br /> +<h2 style='display: inline'>Conclusions</h2><br /> +<br /> +<span>It's a bit of manual work, but it's ok on this small scale! I shall invest time in creating an official OpenBSD port, though. That would render most of the manual steps obsolete, as outlined in this post!</span><br /> +<br /> +<span>Check out the following for more information:</span><br /> +<br /> +<a class=textlink href='https://dtail.dev'>https://dtail.dev</a><br /> +<a class=textlink href='https://github.com/mimecast/dtail'>https://github.com/mimecast/dtail</a><br /> +<a class=textlink href='https://www.rexify.org'>https://www.rexify.org</a><br /> +<br /> +<span>Other related posts are:</span><br /> +<br /> +<a class=textlink href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD (You are currently reading this)</a><br /> +<a class=textlink href='./2022-03-06-the-release-of-dtail-4.0.0.html'>2022-03-06 The release of DTail 4.0.0</a><br /> +<a class=textlink href='./2021-04-22-dtail-the-distributed-log-tail-program.html'>2021-04-22 DTail - The distributed log tail program</a><br /> +<br /> +<span>E-Mail your comments to hi@paul.cyou :-)</span><br /> +<br /> +<a class=textlink href='../'>Back to the main site</a><br /> <p class="footer"> Generated with <a href="https://codeberg.org/snonux/gemtexter">Gemtexter</a> | served by <a href="https://www.OpenBSD.org">OpenBSD</a>/<a href="https://man.openbsd.org/httpd.8">httpd(8)</a> | |