diff options
| author | Paul Buetow <paul@buetow.org> | 2023-09-25 15:14:36 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2023-09-25 15:14:36 +0300 |
| commit | 121c3664914da4d3ece75f062f32fac868558144 (patch) | |
| tree | f9e3bc3fc9fe466004cf1ae1ab706a80f7489a0c /gemfeed/atom.xml | |
| parent | 3e46bf16106fb87aec7b6d8b4d4277bf3242af24 (diff) | |
Update content for html
Diffstat (limited to 'gemfeed/atom.xml')
| -rw-r--r-- | gemfeed/atom.xml | 814 |
1 files changed, 365 insertions, 449 deletions
diff --git a/gemfeed/atom.xml b/gemfeed/atom.xml index c19fbe87..b5314521 100644 --- a/gemfeed/atom.xml +++ b/gemfeed/atom.xml @@ -1,12 +1,332 @@ <?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom"> - <updated>2023-08-22T00:06:26+03:00</updated> + <updated>2023-09-25T15:14:22+03:00</updated> <title>foo.zone feed</title> <subtitle>To be in the .zone!</subtitle> <link href="https://foo.zone/gemfeed/atom.xml" rel="self" /> <link href="https://foo.zone/" /> <id>https://foo.zone/</id> <entry> + <title>DTail usage examples</title> + <link href="https://foo.zone/gemfeed/2023-09-25-dtail-usage-examples.html" /> + <id>https://foo.zone/gemfeed/2023-09-25-dtail-usage-examples.html</id> + <updated>2023-09-25T14:57:42+03:00</updated> + <author> + <name>Paul Buetow aka snonux</name> + <email>paul@dev.buetow.org</email> + </author> + <summary>Hey there. As I am pretty busy this month personally (I am now on Paternity Leave) and as I still want to post once monthly, the blog post of this month will only be some DTail usage examples. They're from the DTail documentation, but not all readers of my blog may be aware of those!</summary> + <content type="xhtml"> + <div xmlns="http://www.w3.org/1999/xhtml"> + <h1 style='display: inline'>DTail usage examples</h1><br /> +<br /> +<span class='quote'>Published at 2023-09-25T14:57:42+03:00</span><br /> +<br /> +<span>Hey there. As I am pretty busy this month personally (I am now on Paternity Leave) and as I still want to post once monthly, the blog post of this month will only be some DTail usage examples. They're from the DTail documentation, but not all readers of my blog may be aware of those!</span><br /> +<br /> +<span>DTail is a distributed DevOps tool for tailing, grepping, catting logs and other text files on many remote machines at once which I programmed in Go.</span><br /> +<br /> +<a class='textlink' href='https://dtail.dev'>https://dtail.dev</a><br /> +<br /> +<pre> + ,_---~~~~~----._ + _,,_,*^____ _____``*g*\"*, + ____ _____ _ _ / __/ /' ^. / \ ^@q f + | _ \_ _|_ _(_) | @f | ((@| |@)) l 0 _/ + | | | || |/ _` | | | \`/ \~____ / __ \_____/ \ + | |_| || | (_| | | | | _l__l_ I + |____/ |_|\__,_|_|_| } [______] I + ] | | | | + ] ~ ~ | + | Let's tail those logs! | + | | +</pre> +<br /> +<span>DTail consists out of a server and several client binaries. In this post, I am showcasing their use!</span><br /> +<br /> +<ul> +<li>Use <span class='inlinecode'>dtail</span> to follow logs</li> +<li>Use <span class='inlinecode'>dtail</span> to aggregate logs while they are followed</li> +<li>Use <span class='inlinecode'>dcat</span> to display logs and other text files already written</li> +<li>Use <span class='inlinecode'>dgrep</span> to grep (search) logs and other text files already written</li> +<li>Use <span class='inlinecode'>dmap</span> to aggregate logs and other text files already written</li> +<li><span class='inlinecode'>dserver</span> is the DTail server, where all the clients can connect to</li> +</ul><br /> +<h2 style='display: inline'>Following logs</h2><br /> +<br /> +<span>The following example demonstrates how to follow logs of several servers at once. The server list is provided as a flat text file. The example filters all records containing the string <span class='inlinecode'>INFO</span>. Any other Go compatible regular expression can also be used instead of <span class='inlinecode'>INFO</span>.</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt --grep INFO --files <font color="#FF0000">"/var/log/dserver/*.log"</font> +</pre> +<br /> +<span>Hint: you can also provide a comma separated server list, e.g.: <span class='inlinecode'>servers server1.example.org,server2.example.org:PORT,...</span></span><br /> +<br /> +<a href='./dtail-usage-examples/dtail.gif'><img alt='Tail example' title='Tail example' src='./dtail-usage-examples/dtail.gif' /></a><br /> +<br /> +<span class='quote'>Hint: You can also use the shorthand version (omitting the <span class='inlinecode'>--files</span>)</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt --grep INFO <font color="#FF0000">"/var/log/dserver/*.log"</font> +</pre> +<br /> +<h2 style='display: inline'>Aggregating logs</h2><br /> +<br /> +<span>To run ad-hoc map-reduce aggregations on newly written log lines you must add a query. The following example follows all remote log lines and prints out every few seconds the result to standard output.</span><br /> +<br /> +<span class='quote'>Hint: To run a map-reduce query across log lines written in the past, please use the <span class='inlinecode'>dmap</span> command instead.</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt <font color="#990000">\</font> + --files <font color="#FF0000">'/var/log/dserver/*.log'</font> <font color="#990000">\</font> + --query <font color="#FF0000">'from STATS select sum($goroutines),sum($cgocalls),</font> +<font color="#FF0000"> last($time),max(lifetimeConnections)'</font> +</pre> +<br /> +<span>Beware: For map-reduce queries to work, you have to ensure that DTail supports your log format. Check out the documentaiton of the DTail query language and the DTail log formats on the DTail homepage for more information.</span><br /> +<br /> +<a href='./dtail-usage-examples/dtail-map.gif'><img alt='Tail map-reduce example' title='Tail map-reduce example' src='./dtail-usage-examples/dtail-map.gif' /></a><br /> +<br /> +<span class='quote'>Hint: You can also use the shorthand version:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt <font color="#990000">\</font> + --files <font color="#FF0000">'/var/log/dserver/*.log'</font> <font color="#990000">\</font> + <font color="#FF0000">'from STATS select sum($goroutines),sum($cgocalls),</font> +<font color="#FF0000"> last($time),max(lifetimeConnections)'</font> +</pre> +<br /> +<span>Here is another example:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt <font color="#990000">\</font> + --files <font color="#FF0000">'/var/log/dserver/*.log'</font> <font color="#990000">\</font> + --query <font color="#FF0000">'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg,</font> +<font color="#FF0000"> lifetimeConnections group by $hostname order by max($cgocalls)'</font> +</pre> +<br /> +<a href='./dtail-usage-examples/dtail-map2.gif'><img alt='Tail map-reduce example 2' title='Tail map-reduce example 2' src='./dtail-usage-examples/dtail-map2.gif' /></a><br /> +<br /> +<span>You can also continuously append the results to a CSV file by adding <span class='inlinecode'>outfile append filename.csv</span> to the query:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt <font color="#990000">\</font> + --files <font color="#FF0000">'/var/log/dserver/*.log'</font> <font color="#990000">\</font> + --query <font color="#FF0000">'from STATS select ... outfile append result.csv'</font> +</pre> +<br /> +<h2 style='display: inline'>How to use <span class='inlinecode'>dcat</span></h2><br /> +<br /> +<span>The following example demonstrates how to cat files (display the full content of the files) on several servers at once.</span><br /> +<br /> +<span>As you can see in this example, a DTail client also creates a local log file of all received data in <span class='inlinecode'>~/log</span>. You can also use the <span class='inlinecode'>noColor</span> and <span class='inlinecode'>-plain</span> flags (this all also work with other DTail commands than <span class='inlinecode'>dcat</span>).</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dcat --servers serverlist<font color="#990000">.</font>txt --files /etc/hostname +</pre> +<br /> +<a href='./dtail-usage-examples/dcat.gif'><img alt='Cat example' title='Cat example' src='./dtail-usage-examples/dcat.gif' /></a><br /> +<br /> +<span class='quote'>Hint: You can also use the shorthand version:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dcat --servers serverlist<font color="#990000">.</font>txt /etc/hostname +</pre> +<br /> +<h2 style='display: inline'>How to use <span class='inlinecode'>dgrep</span></h2><br /> +<br /> +<span>The following example demonstrates how to grep files (display only the lines which match a given regular expression) of multiple servers at once. In this example, we look after some entries in <span class='inlinecode'>/etc/passwd</span>. This time, we don't provide the server list via an file but rather via a comma separated list directly on the command line. We also explore the <span class='inlinecode'>-before</span>, <span class='inlinecode'>-after</span> and <span class='inlinecode'>-max</span> flags (see animation).</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dgrep --servers server1<font color="#990000">.</font>example<font color="#990000">.</font>org<font color="#990000">:</font><font color="#993399">2223</font> <font color="#990000">\</font> + --files /etc/passwd <font color="#990000">\</font> + --regex nologin +</pre> +<br /> +<span>Generally, <span class='inlinecode'>dgrep</span> is also a very useful way to search historic application logs for certain content.</span><br /> +<br /> +<a href='./dtail-usage-examples/dgrep.gif'><img alt='Grep example' title='Grep example' src='./dtail-usage-examples/dgrep.gif' /></a><br /> +<br /> +<span class='quote'>Hint: <span class='inlinecode'>-regex</span> is an alias for <span class='inlinecode'>-grep</span>.</span><br /> +<br /> +<h2 style='display: inline'>How to use <span class='inlinecode'>dmap</span></h2><br /> +<br /> +<span>To run a map-reduce aggregation over logs written in the past, the <span class='inlinecode'>dmap</span> command can be used. The following example aggregates all map-reduce fields <span class='inlinecode'>dmap</span> will print interim results every few seconds. You can also write the result to an CSV file by adding <span class='inlinecode'>outfile result.csv</span> to the query.</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dmap --servers serverlist<font color="#990000">.</font>txt <font color="#990000">\</font> + --files <font color="#FF0000">'/var/log/dserver/*.log'</font> <font color="#990000">\</font> + --query <font color="#FF0000">'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg,</font> +<font color="#FF0000"> lifetimeConnections group by $hostname order by max($cgocalls)'</font> +</pre> +<br /> +<span>Remember: For that to work, you have to make sure that DTail supports your log format. You can either use the ones already defined in <span class='inlinecode'>internal/mapr/logformat</span> or add an extension to support a custom log format. The example here works out of the box though, as DTail understands its own log format already. </span><br /> +<br /> +<a href='./dtail-usage-examples/dmap.gif'><img alt='DMap example' title='DMap example' src='./dtail-usage-examples/dmap.gif' /></a><br /> +<br /> +<h2 style='display: inline'>How to use the DTail serverless mode</h2><br /> +<br /> +<span>Until now, all examples so far required to have remote server(s) to connect to. That makes sense, as after all DTail is a *distributed* tool. However, there are circumstances where you don't really need to connect to a server remotely. For example, you already have a login shell open to the server an all what you want is to run some queries directly on local log files.</span><br /> +<br /> +<span>The serverless mode does not require any <span class='inlinecode'>dserver</span> up and running and therefore there is no networking/SSH involved. </span><br /> +<br /> +<span>All commands shown so far also work in a serverless mode. All what needs to be done is to omit a server list. The DTail client then starts in serverless mode.</span><br /> +<br /> +<h3 style='display: inline'>Serverless map-reduce query</h3><br /> +<br /> +<span>The following <span class='inlinecode'>dmap</span> example is the same as the previously shown one, but the difference is that it operates on a local log file directly:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dmap --files /var/log/dserver/dserver<font color="#990000">.</font>log + --query <font color="#FF0000">'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg,</font> +<font color="#FF0000"> lifetimeConnections group by $hostname order by max($cgocalls)'</font> +</pre> +<br /> +<span>As a shorthand version the following command can be used:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dmap <font color="#FF0000">'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg,</font> +<font color="#FF0000"> lifetimeConnections group by $hostname order by max($cgocalls)'</font> <font color="#990000">\</font> + /var/log/dsever/dserver<font color="#990000">.</font>log +</pre> +<br /> +<span>You can also use a file input pipe as follows:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> cat /var/log/dserver/dserver<font color="#990000">.</font>log <font color="#990000">|</font> <font color="#990000">\</font> + dmap <font color="#FF0000">'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg,</font> +<font color="#FF0000"> lifetimeConnections group by $hostname order by max($cgocalls)'</font> +</pre> +<br /> +<h3 style='display: inline'>Aggregating CSV files</h3><br /> +<br /> +<span>In essence, this works exactly like aggregating logs. All files operated on must be valid CSV files and the first line of the CSV must be the header. E.g.:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> cat example<font color="#990000">.</font>csv +name<font color="#990000">,</font>lastname<font color="#990000">,</font>age<font color="#990000">,</font>profession +Michael<font color="#990000">,</font>Jordan<font color="#990000">,</font><font color="#993399">40</font><font color="#990000">,</font>Basketball player +Michael<font color="#990000">,</font>Jackson<font color="#990000">,</font><font color="#993399">100</font><font color="#990000">,</font>Singer +Albert<font color="#990000">,</font>Einstein<font color="#990000">,</font><font color="#993399">200</font><font color="#990000">,</font>Physician +<font color="#990000">%</font> dmap --query <font color="#FF0000">'select lastname,name where age > 40 logformat csv outfile result.csv'</font> example<font color="#990000">.</font>csv +<font color="#990000">%</font> cat result<font color="#990000">.</font>csv +lastname<font color="#990000">,</font>name +Jackson<font color="#990000">,</font>Michael +Einstein<font color="#990000">,</font>Albert +</pre> +<br /> +<span>DMap can also be used to query and aggregate CSV files from remote servers.</span><br /> +<br /> +<h3 style='display: inline'>Other serverless commands</h3><br /> +<br /> +<span>The serverless mode works transparently with all other DTail commands. Here are some examples:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail /var/log/dserver/dserver<font color="#990000">.</font>log +</pre> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --logLevel trace /var/log/dserver/dserver<font color="#990000">.</font>log +</pre> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dcat /etc/passwd +</pre> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dcat --plain /etc/passwd <font color="#990000">></font> /etc/test +<i><font color="#9A1900"># Should show no differences.</font></i> +diff /etc/test /etc/passwd +</pre> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dgrep --regex ERROR --files /var/log/dserver/dsever<font color="#990000">.</font>log +</pre> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dgrep --before <font color="#993399">10</font> --after <font color="#993399">10</font> --max <font color="#993399">10</font> --grep ERROR /var/log/dserver/dsever<font color="#990000">.</font>log +</pre> +<br /> +<span>Use <span class='inlinecode'>--help</span> for more available options. Or go to the DTail page for more information! Hope you find DTail useful!</span><br /> +<br /> +<span>Other related posts are:</span><br /> +<br /> +<a class='textlink' href='./2021-04-22-dtail-the-distributed-log-tail-program.html'>2021-04-22 DTail - The distributed log tail program</a><br /> +<a class='textlink' href='./2022-03-06-the-release-of-dtail-4.0.0.html'>2022-03-06 The release of DTail 4.0.0</a><br /> +<a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD</a><br /> +<a class='textlink' href='./2023-09-25-dtail-usage-examples.html'>2023-09-25 DTail usage examples (You are currently reading this)</a><br /> +<br /> +<span>I hope you find the tools presented in this post useful!</span><br /> +<br /> +<span>Paul</span><br /> +<br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> +<br /> +<a class='textlink' href='../'>Back to the main site</a><br /> + </div> + </content> + </entry> + <entry> <title>Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect</title> <link href="https://foo.zone/gemfeed/2023-08-20-site-reliability-engineering-part-3.html" /> <id>https://foo.zone/gemfeed/2023-08-20-site-reliability-engineering-part-3.html</id> @@ -74,7 +394,7 @@ <br /> <span>The fourth part of this blog series will be published soon :-)</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -139,7 +459,7 @@ <br /> <a class='textlink' href='./2023-08-20-site-reliability-engineering-part-3.html'>2023-08-20 Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -213,7 +533,7 @@ DC on fire: <br /> <a class='textlink' href='./2023-08-19-site-reliability-engineering-part-2.html'>2023-08-19 Site Reliability Engineering - Part 2: Operational Balance in SRE</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -341,7 +661,7 @@ http://www.gnu.org/software/src-highlite --> <a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again²</a><br /> <a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³ (You are currently reading this)</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -664,7 +984,7 @@ http://www.gnu.org/software/src-highlite --> <a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes</a><br /> <a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes (You are currently reading this)</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../resources.html'>More books and other resources I found useful.</a><br /> <a class='textlink' href='../'>Back to the main site</a><br /> @@ -942,7 +1262,7 @@ http://www.gnu.org/software/src-highlite --> <br /> <span>Gogios is a lightweight and straightforward monitoring tool that is perfect for small-scale environments. With its compatibility with the Nagios Check API, email notifications, and CRON-based scheduling, Gogios offers an easy-to-use solution for those looking to monitor a limited number of resources. I personally use it to execute around 500 checks on my personal server infrastructure. I am very happy with this solution.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -1051,7 +1371,7 @@ http://www.gnu.org/software/src-highlite --> <a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes (You are currently reading this)</a><br /> <a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../resources.html'>More books and other resources I found useful.</a><br /> <a class='textlink' href='../'>Back to the main site</a><br /> @@ -1507,7 +1827,7 @@ ok codeberg<font color="#990000">.</font>org/snonux/algorithms/sort <fo <br /> <span>I won't write any benchmark for sleep sort; that will be done for the algorithms to come in this series :-).</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -1662,7 +1982,7 @@ ok codeberg<font color="#990000">.</font>org/snonux/algorithms/sort <fo <a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes</a><br /> <a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -1835,7 +2155,7 @@ The remaining content of the Gemtext file<font color="#990000">...</font> <a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again² (You are currently reading this)</a><br /> <a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -1943,7 +2263,7 @@ The remaining content of the Gemtext file<font color="#990000">...</font> <a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes</a><br /> <a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../resources.html'>More books and other resources I found useful.</a><br /> <a class='textlink' href='../'>Back to the main site</a><br /> @@ -2038,7 +2358,7 @@ The remaining content of the Gemtext file<font color="#990000">...</font> <br /> <span>There are some days at work you feel drained afterwards and think you didn't progress towards your goals at all. It's more challenging to shut down from work after such a day. A quick hack is to work on a quick win before the end of the day, giving you a sense of accomplishment after all. Another way is to make progress on your fun passion project after work. It must not be work-related, but a sense of accomplishment will still be there.</span><br /> <span> </span><br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -2195,7 +2515,7 @@ Art by Joan Stark <br /> <span>And, of course, GrapheneOS is an open-source project. This is a good thing; however, on the other side, nobody can guarantee that the OS will not break or will not damage your phone. You have to trust the GrapheneOS project and donate to the project so they can keep up with the great work. But I rather trust the GrapheneOS team than big tech. </span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -2319,7 +2639,7 @@ Art by Joan Stark <br /> <span>Am I a Java expert now? No, by far not. But I am better now than before :-).</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -2452,7 +2772,7 @@ nmap ,<b><font color="#0000FF">i</font></b> !wpbpaste<font color="#FF6600"><C <br /> <a class='textlink' href='https://github.com/NvChad/NvChad'>https://github.com/NvChad/NvChad</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -2816,8 +3136,9 @@ REMOTE|fishfinger|100|7|fstab|093f510ec5c0f512.h /usr/local ffs rw,wxallowed,nod <a class='textlink' href='./2021-04-22-dtail-the-distributed-log-tail-program.html'>2021-04-22 DTail - The distributed log tail program</a><br /> <a class='textlink' href='./2022-03-06-the-release-of-dtail-4.0.0.html'>2022-03-06 The release of DTail 4.0.0</a><br /> <a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD (You are currently reading this)</a><br /> +<a class='textlink' href='./2023-09-25-dtail-usage-examples.html'>2023-09-25 DTail usage examples</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -2931,7 +3252,7 @@ jgs (________\ \ <br /> <span>I wouldn't say I like checking social media, as it can consume a lot of time and can become addictive. But once in a while, I want to catch up with my "networks". After a bad night's sleep, it's the perfect time to check your social media. Once done, you don't have to do it anymore for the next couple of days!</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -3050,7 +3371,7 @@ http://www.gnu.org/software/src-highlite --> <a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again²</a><br /> <a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -3728,7 +4049,7 @@ rex commons <br /> <span>Why re-inventing the wheel? I love that a <span class='inlinecode'>Rexfile</span> is just a Perl DSL. Also, OpenBSD comes with Perl in the base system. So no new programming language had to be added to my mix for the configuration management system. Also, the <span class='inlinecode'>acme.sh</span> shell script is not a Bash but a standard Bourne shell script, so I didn't have to install an additional shell as OpenBSD does not come with the Bash pre-installed.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -4072,7 +4393,7 @@ v = 008 [v = p*c*(s != c ? 2 : 1)] Total logical CPUs <br /> <a class='textlink' href='./2021-04-24-welcome-to-the-geminispace.html'>Welcome to the Geminispae</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -4238,7 +4559,7 @@ v = 008 [v = p*c*(s != c ? 2 : 1)] Total logical CPUs <a class='textlink' href='./2022-05-27-perl-is-still-a-great-choice.html'>2022-05-27 Perl is still a great choice (You are currently reading this)</a><br /> <a class='textlink' href='./2023-05-01-unveiling-guprecords:-uptime-records-with-raku.html'>2023-05-01 Unveiling <span class='inlinecode'>guprecords.raku</span>: Global Uptime Records with Raku</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -4397,7 +4718,7 @@ learn () { <li>The Off Switch; Mark Cropley; Virgin Books</li> <li>Ultralearning; Scott Young; Thorsons</li> </ul><br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -4712,12 +5033,13 @@ exec /usr/local/bin/dtailhealth --server localhost:2222 <a class='textlink' href='./2021-04-22-dtail-the-distributed-log-tail-program.html'>2021-04-22 DTail - The distributed log tail program</a><br /> <a class='textlink' href='./2022-03-06-the-release-of-dtail-4.0.0.html'>2022-03-06 The release of DTail 4.0.0 (You are currently reading this)</a><br /> <a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD</a><br /> +<a class='textlink' href='./2023-09-25-dtail-usage-examples.html'>2023-09-25 DTail usage examples</a><br /> <br /> <span>Thanks!</span><br /> <br /> <span>Paul</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -4964,7 +5286,7 @@ GNU/kFreeBSD rhea.buetow.org 8.0-RELEASE-p5 FreeBSD 8.0-RELEASE-p5 #2: Sat Nov 2 <a class='textlink' href='https://www.dragonflybsd.org/'>DragonFly BSD - Fork of FreeBSD 4</a><br /> <a class='textlink' href='http://wiki.postmarketos.org/wiki/Phosh'>Phosh (on postmarketOS) - A true Linux shell for the smartphone</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -5029,7 +5351,7 @@ GNU/kFreeBSD rhea.buetow.org 8.0-RELEASE-p5 FreeBSD 8.0-RELEASE-p5 #2: Sat Nov 2 <br /> <span>The host buetow.org will stay. However, not as the primary address for this site. I will keep using it for my personal internet infrastructure as well as for most of my E-Mail addresses. I used buetow.org for that over the past 10 years already anyway and that won't change any time soon. I don't know what I am going to do with snonux.de in the long run. A .de SLD (for Germany) is pretty cheap, so I might just keep it for now. </span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -5537,7 +5859,7 @@ PAUL:X:1000:1000:PAUL BUETOW:/HOME/PAUL:/BIN/BASH <a class='textlink' href='./2021-11-29-bash-golf-part-1.html'>2021-11-29 Bash Golf Part 1</a><br /> <a class='textlink' href='./2022-01-01-bash-golf-part-2.html'>2022-01-01 Bash Golf Part 2 (You are currently reading this)</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -5681,7 +6003,7 @@ PAUL:X:1000:1000:PAUL BUETOW:/HOME/PAUL:/BIN/BASH <br /> <a class='textlink' href='https://unixsheikh.com/articles/how-to-stay-sane-in-todays-world-of-tech.html'>https://unixsheikh.com/articles/how-to-stay-sane-in-todays-world-of-tech.html</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -6171,7 +6493,7 @@ bash: line 1: 1/10.0 : syntax error: invalid arithmetic operator (error token is <a class='textlink' href='./2021-11-29-bash-golf-part-1.html'>2021-11-29 Bash Golf Part 1 (You are currently reading this)</a><br /> <a class='textlink' href='./2022-01-01-bash-golf-part-2.html'>2022-01-01 Bash Golf Part 2</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -6292,7 +6614,7 @@ bash: line 1: 1/10.0 : syntax error: invalid arithmetic operator (error token is <br /> <span>This usually means creating one or more tickets, which will be dealt with soon. Once the permanent fix is deployed, you can remove your ad-hoc automation and monitoring around it and focus on your regular work again.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -6406,7 +6728,7 @@ bash: line 1: 1/10.0 : syntax error: invalid arithmetic operator (error token is <br /> <span>Enough ranted for now!</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <span class='quote'>Controversially, a lack of features is a feature. Enjoy your peace an quiet. - Michael W Lucas </span><br /> <br /> @@ -6486,7 +6808,7 @@ bash: line 1: 1/10.0 : syntax error: invalid arithmetic operator (error token is <br /> <span>Do you need Microsoft Word? Why don't you just use the Vim text editor or GNU Emacs to write your letters? If that's too nerdy, you can still use open-source alternatives such as AbiWord or LibreOffice. Larger organizations have the tendency to standardize the software their employees have to use. Unfortunately, as Microsoft Word is the de-facto standard text processing program, most companies prefer Word over LibreOffice. Same with Microsoft Excel vs LibreOffice Calc or other spreadsheet alternatives like Gnumeric. I don't know why that is; please....</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <span>I only use free and open-source operating systems on my personal Laptops, Desktop PCs and servers (FreeBSD and Linux based ones). Most of the programs and apps I use on them are free and open-source as well, and I am comfortable with it for over twenty years. Exceptions are the BIOSes and some firmwares of my devices. I also use Skype as most of my friends and family are using it. They are, unfortunately, proprietary software still. But I will be looking into Matrix as a Skype alternative when I have time. There are also open BIOS alternatives, but they usually don't work on my devices.</span><br /> <br /> @@ -6543,7 +6865,7 @@ bash: line 1: 1/10.0 : syntax error: invalid arithmetic operator (error token is <br /> <span>You have better chances when you know how to manage your own server and install and manage alternatives to the big cloud providers by yourself. I have the advantage that I have work experience as a Linux Systems Administrator here. I mentioned NextCloud already. I use NextCloud for online photo and file storage, contact and calendar sync and as an RSS news feed server. You could do the same with your own E-Mail server, you can also host your own website and blog. I also mentioned Matrix as a Skype alternative (which could also be an alternative to WhatsApp, Skype, Telegram, Viber, ...). I don't know a lot about Matrix yet, but it seems to be a very neat alternative. I am ready to invest time in it as one of my future personal pet projects. Not only because I think it's better, but also because for fun and as a hobby. But this doesn't mean that I invest *all* of my personal free time in it.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -6666,7 +6988,7 @@ Hello World <br /> <span>Will I abandon my beloved Perl? Probably not. There are also some Perl scripts I use at work. But unfortunately I only have a limited amount of time and I have to use it wisely. I might look into Raku (formerly known as Perl 6) next year and use it for a personal pet project, who knows. :-). I also highly recommend reading the two Perl books "Modern Perl" and "Higher-Order Perl".</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -6871,7 +7193,7 @@ assert<font color="#990000">::</font>equals <font color="#FF0000">"$(generate::m <a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again²</a><br /> <a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -7278,7 +7600,7 @@ fi <a class='textlink' href='./2021-11-29-bash-golf-part-1.html'>2021-11-29 Bash Golf Part 1</a><br /> <a class='textlink' href='./2022-01-01-bash-golf-part-2.html'>2022-01-01 Bash Golf Part 2</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -7384,7 +7706,7 @@ fi <a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again²</a><br /> <a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -7515,8 +7837,9 @@ dtail –servers serverlist.txt –files ‘/var/log/*.log’ –regex ‘(?i:er <a class='textlink' href='./2021-04-22-dtail-the-distributed-log-tail-program.html'>2021-04-22 DTail - The distributed log tail program (You are currently reading this)</a><br /> <a class='textlink' href='./2022-03-06-the-release-of-dtail-4.0.0.html'>2022-03-06 The release of DTail 4.0.0</a><br /> <a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD</a><br /> +<a class='textlink' href='./2023-09-25-dtail-usage-examples.html'>2023-09-25 DTail usage examples</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -7722,7 +8045,7 @@ Total time: 1213.00s <a class='textlink' href='https://graphiteapp.org'>Graphite</a><br /> <a class='textlink' href='https://en.wikipedia.org/wiki/Memory-mapped_I/O'>Memory mapped I/O</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -7855,7 +8178,7 @@ http://www.gnu.org/software/src-highlite --> <br /> <span>C is a very old programming language with it's quirks. This might be one of the reasons why Linux will also let Rust code in.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -8110,7 +8433,7 @@ apply Service "dig6" { </ul><br /> <span>That's much more comfortable now than manually clicking at some web UIs at Schlund Technologies.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -8156,414 +8479,7 @@ apply Service "dig6" { <br /> <span>Furthermore, I added scrubbing ("zpool scrub...") to the script. It ensures that the file system is consistent and that there are no bad blocks on the disk and the file system. To increase the reliability, I also run a "zfs set copies=2 zroot". That setting is also synchronized to the offsite ZFS pool. ZFS stores every data block to disk twice now. Yes, it consumes twice as much disk space, making it better fault-tolerant against hardware errors (e.g. only individual disk sectors going bad). </span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>Jails and ZFS with Puppet on FreeBSD</title> - <link href="https://foo.zone/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html" /> - <id>https://foo.zone/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.html</id> - <updated>2016-04-09T18:29:47+01:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>Over the last couple of years I wrote quite a few Puppet modules in order to manage my personal server infrastructure. One of them manages FreeBSD Jails and another one ZFS file systems. I thought I would give a brief overview in how it looks and feels.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Jails and ZFS with Puppet on FreeBSD</h1><br /> -<br /> -<span class='quote'>Published at 2016-04-09T18:29:47+01:00</span><br /> -<br /> -<pre> - __ __ - (( \---/ )) - )__ __( - / ()___() \ - \ /(_)\ / - \ \_|_/ / - _______> <_______ - //\ |>o<| /\\ - \\/___ ___\// - | | - | | - | | - | | - `--....---' - \ \ - \ `. hjw - \ `. -</pre> -<br /> -<span>Over the last couple of years I wrote quite a few Puppet modules in order to manage my personal server infrastructure. One of them manages FreeBSD Jails and another one ZFS file systems. I thought I would give a brief overview in how it looks and feels.</span><br /> -<br /> -<h2 style='display: inline'>ZFS</h2><br /> -<br /> -<span>The ZFS module is a pretty basic one. It does not manage ZFS pools yet as I am not creating them often enough which would justify implementing an automation. But let's see how we can create a ZFS file system (on an already given ZFS pool named ztank):</span><br /> -<br /> -<span>Puppet snippet:</span><br /> -<br /> -<pre> -zfs::create { 'ztank/foo': - ensure => present, - filesystem => '/srv/foo', - - require => File['/srv'], -} -</pre> -<br /> -<span>Puppet run:</span><br /> -<br /> -<pre> -admin alphacentauri:/opt/git/server/puppet/manifests [1212]% puppet.apply -Password: -Info: Loading facts -Info: Loading facts -Info: Loading facts -Info: Loading facts -Notice: Compiled catalog for alphacentauri.home in environment production in 7.14 seconds -Info: Applying configuration version '1460189837' -Info: mount[files]: allowing * access -Info: mount[restricted]: allowing * access -Notice: /Stage[main]/Main/Node[alphacentauri]/Zfs::Create[ztank/foo]/Exec[ztank/foo_create]/returns: executed successfully -Notice: Finished catalog run in 25.41 seconds -admin alphacentauri:~ [1213]% zfs list | grep foo -ztank/foo 96K 1.13T 96K /srv/foo -admin alphacentauri:~ [1214]% df | grep foo -ztank/foo 1214493520 96 1214493424 0% /srv/foo -admin alphacentauri:~ [1215]% -</pre> -<br /> -<span>The destruction of the file system just requires to set "ensure" to "absent" in Puppet:</span><br /> -<br /> -<pre> -zfs::create { 'ztank/foo': - ensure => absent, - filesystem => '/srv/foo', - - require => File['/srv'], -}¬ -</pre> -<br /> -<span>Puppet run:</span><br /> -<br /> -<pre> -admin alphacentauri:/opt/git/server/puppet/manifests [1220]% puppet.apply -Password: -Info: Loading facts -Info: Loading facts -Info: Loading facts -Info: Loading facts -Notice: Compiled catalog for alphacentauri.home in environment production in 6.14 seconds -Info: Applying configuration version '1460190203' -Info: mount[files]: allowing * access -Info: mount[restricted]: allowing * access -Notice: /Stage[main]/Main/Node[alphacentauri]/Zfs::Create[ztank/foo]/Exec[zfs destroy -r ztank/foo]/returns: executed successfully -Notice: Finished catalog run in 22.72 seconds -admin alphacentauri:/opt/git/server/puppet/manifests [1221]% zfs list | grep foo -zsh: done zfs list | -zsh: exit 1 grep foo -admin alphacentauri:/opt/git/server/puppet/manifests [1222:1]% df | grep foo -zsh: done df | -zsh: exit 1 grep foo -</pre> -<br /> -<h2 style='display: inline'>Jails</h2><br /> -<br /> -<span>Here is an example in how a FreeBSD Jail can be created. The Jail will have its own public IPv6 address. And it will have its own internal IPv4 address with IPv4 NAT to the internet (this is due to the limitation that the host server only got one public IPv4 address which requires sharing between all the Jails).</span><br /> -<br /> -<span>Furthermore, Puppet will ensure that the Jail will have its own ZFS file system (internally it is using the ZFS module). Please notice that the NAT requires the packet filter to be setup correctly (not covered in this blog post).</span><br /> -<br /> -<pre> -include jail::freebsd - -# Cloned interface for Jail IPv4 NAT -freebsd::rc_config { 'cloned_interfaces': - value => 'lo1', -} -freebsd::rc_config { 'ipv4_addrs_lo1': - value => '192.168.0.1-24/24' -} - -freebsd::ipalias { '2a01:4f8:120:30e8::17': - ensure => up, - proto => 'inet6', - preflen => '64', - interface => 're0', - aliasnum => '8', -} - -class { 'jail': - ensure => present, - jails_config => { - sync => { - '_ensure' => present, - '_type' => 'freebsd', - '_mirror' => 'ftp://ftp.de.freebsd.org', - '_remote_path' => 'FreeBSD/releases/amd64/10.1-RELEASE', - '_dists' => [ 'base.txz', 'doc.txz', ], - '_ensure_directories' => [ '/opt', '/opt/enc' ], - '_ensure_zfs' => [ '/sync' ], - 'host.hostname' => "'sync.ian.buetow.org'", - 'ip4.addr' => '192.168.0.17', - 'ip6.addr' => '2a01:4f8:120:30e8::17', - }, - } -} -</pre> -<br /> -<span>This is how the result looks like:</span><br /> -<br /> -<pre> -admin sun:/etc [1939]% puppet.apply -Info: Loading facts -Info: Loading facts -Info: Loading facts -Info: Loading facts -Notice: Compiled catalog for sun.ian.buetow.org in environment production in 1.80 seconds -Info: Applying configuration version '1460190986' -Notice: /Stage[main]/Jail/File[/etc/jail.conf]/ensure: created -Info: mount[files]: allowing * access -Info: mount[restricted]: allowing * access -Info: Computing checksum on file /etc/motd -Info: /Stage[main]/Motd/File[/etc/motd]: Filebucketed /etc/motd to puppet with sum fced1b6e89f50ef2c40b0d7fba9defe8 -Notice: /Stage[main]/Jail/Jail::Create[sync]/File[/jail/sync]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/Zfs::Create[zroot/jail/sync]/Exec[zroot/jail/sync_create]/returns: executed successfully -Notice: /Stage[main]/Jail/Jail::Create[sync]/File[/jail/sync/opt]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/File[/jail/sync/opt/enc]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/Jail::Ensure_zfs[/sync]/Zfs::Create[zroot/jail/sync/sync]/Exec[zroot/jail/sync/sync_create]/returns: executed successfully -Notice: /Stage[main]/Jail/Jail::Create[sync]/Jail::Freebsd::Create[sync]/File[/jail/sync/.jailbootstrap]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/Jail::Freebsd::Create[sync]/File[/etc/fstab.jail.sync]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/Jail::Freebsd::Create[sync]/File[/jail/sync/.jailbootstrap/bootstrap.sh]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/Jail::Freebsd::Create[sync]/Exec[sync_bootstrap]/returns: executed successfully -Notice: Finished catalog run in 49.72 seconds -admin sun:/etc [1942]% ls -l /jail/sync -total 154 --r--r--r-- 1 root wheel 6198 11 Nov 2014 COPYRIGHT -drwxr-xr-x 2 root wheel 47 11 Nov 2014 bin -drwxr-xr-x 7 root wheel 43 11 Nov 2014 boot -dr-xr-xr-x 2 root wheel 2 11 Nov 2014 dev -drwxr-xr-x 23 root wheel 101 9 Apr 10:37 etc -drwxr-xr-x 3 root wheel 50 11 Nov 2014 lib -drwxr-xr-x 3 root wheel 4 11 Nov 2014 libexec -drwxr-xr-x 2 root wheel 2 11 Nov 2014 media -drwxr-xr-x 2 root wheel 2 11 Nov 2014 mnt -drwxr-xr-x 3 root wheel 3 9 Apr 10:36 opt -dr-xr-xr-x 2 root wheel 2 11 Nov 2014 proc -drwxr-xr-x 2 root wheel 143 11 Nov 2014 rescue -drwxr-xr-x 2 root wheel 6 11 Nov 2014 root -drwxr-xr-x 2 root wheel 132 11 Nov 2014 sbin -drwxr-xr-x 2 root wheel 2 9 Apr 10:36 sync -lrwxr-xr-x 1 root wheel 11 11 Nov 2014 sys -> usr/src/sys -drwxrwxrwt 2 root wheel 2 11 Nov 2014 tmp -drwxr-xr-x 14 root wheel 14 11 Nov 2014 usr -drwxr-xr-x 24 root wheel 24 11 Nov 2014 var -admin sun:/etc [1943]% zfs list | grep sync;df | grep sync -zroot/jail/sync 162M 343G 162M /jail/sync -zroot/jail/sync/sync 144K 343G 144K /jail/sync/sync -/opt/enc 5061624 84248 4572448 2% /jail/sync/opt/enc -zroot/jail/sync 360214972 166372 360048600 0% /jail/sync -zroot/jail/sync/sync 360048744 144 360048600 0% /jail/sync/sync -admin sun:/etc [1944]% cat /etc/fstab.jail.sync -# Generated by Puppet for a Jail. -# Can contain file systems to be mounted curing jail start. -admin sun:/etc [1945]% cat /etc/jail.conf -# Generated by Puppet - -allow.chflags = true; -exec.start = '/bin/sh /etc/rc'; -exec.stop = '/bin/sh /etc/rc.shutdown'; -mount.devfs = true; -mount.fstab = "/etc/fstab.jail.$name"; -path = "/jail/$name"; - -sync { - host.hostname = 'sync.ian.buetow.org'; - ip4.addr = 192.168.0.17; - ip6.addr = 2a01:4f8:120:30e8::17; -} -admin sun:/etc [1955]% sudo service jail start sync -Password: -Starting jails: sync. -admin sun:/etc [1956]% jls | grep sync - 103 192.168.0.17 sync.ian.buetow.org /jail/sync -admin sun:/etc [1957]% sudo jexec 103 /bin/csh -root@sync:/ # ifconfig -a -re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 - options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE> - ether 50:46:5d:9f:fd:1e - inet6 2a01:4f8:120:30e8::17 prefixlen 64 - nd6 options=8021<PERFORMNUD,AUTO_LINKLOCAL,DEFAULTIF> - media: Ethernet autoselect (1000baseT <full-duplex>) - status: active -lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 - options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> - nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> - lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 - options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> - inet 192.168.0.17 netmask 0xffffffff - nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> -</pre> -<br /> -<h2 style='display: inline'>Inside-Jail Puppet</h2><br /> -<br /> -<span>To automatically setup the applications running in the Jail I am using Puppet as well. I wrote a few scripts which bootstrap Puppet inside of a newly created Jail. It is doing the following:</span><br /> -<br /> -<ul> -<li>Mounts an encrypted container (containing a secret Puppet manifests [git repository])</li> -<li>Activates "pkg-ng", the FreeBSD binary package manager, in the Jail</li> -<li>Installs Puppet plus all dependencies in the Jail</li> -<li>Updates the Jail via "freebsd-update" to the latest version</li> -<li>Restarts the Jail and invokes Puppet.</li> -<li>Puppet then also schedules a periodic cron job for the next Puppet runs.</li> -</ul><br /> -<pre> -admin sun:~ [1951]% sudo /opt/snonux/local/etc/init.d/enc activate sync -Starting jails: dns. -The package management tool is not yet installed on your system. -Do you want to fetch and install it now? [y/N]: y -Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/freebsd:10:x86:64/latest, please wait... -Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done -[sync.ian.buetow.org] Installing pkg-1.7.2... -[sync.ian.buetow.org] Extracting pkg-1.7.2: 100% -Updating FreeBSD repository catalogue... -[sync.ian.buetow.org] Fetching meta.txz: 100% 944 B 0.9kB/s 00:01 -[sync.ian.buetow.org] Fetching packagesite.txz: 100% 5 MiB 5.6MB/s 00:01 -Processing entries: 100% -FreeBSD repository update completed. 25091 packages processed. -Updating database digests format: 100% -The following 20 package(s) will be affected (of 0 checked): - - New packages to be INSTALLED: - git: 2.7.4_1 - expat: 2.1.0_3 - python27: 2.7.11_1 - libffi: 3.2.1 - indexinfo: 0.2.4 - gettext-runtime: 0.19.7 - p5-Error: 0.17024 - perl5: 5.20.3_9 - cvsps: 2.1_1 - p5-Authen-SASL: 2.16_1 - p5-Digest-HMAC: 1.03_1 - p5-GSSAPI: 0.28_1 - curl: 7.48.0_1 - ca_root_nss: 3.22.2 - p5-Net-SMTP-SSL: 1.03 - p5-IO-Socket-SSL: 2.024 - p5-Net-SSLeay: 1.72 - p5-IO-Socket-IP: 0.37 - p5-Socket: 2.021 - p5-Mozilla-CA: 20160104 - - The process will require 144 MiB more space. - 30 MiB to be downloaded. -[sync.ian.buetow.org] Fetching git-2.7.4_1.txz: 100% 4 MiB 3.7MB/s 00:01 -[sync.ian.buetow.org] Fetching expat-2.1.0_3.txz: 100% 98 KiB 100.2kB/s 00:01 -[sync.ian.buetow.org] Fetching python27-2.7.11_1.txz: 100% 10 MiB 10.7MB/s 00:01 -[sync.ian.buetow.org] Fetching libffi-3.2.1.txz: 100% 35 KiB 36.2kB/s 00:01 -[sync.ian.buetow.org] Fetching indexinfo-0.2.4.txz: 100% 5 KiB 5.0kB/s 00:01 -[sync.ian.buetow.org] Fetching gettext-runtime-0.19.7.txz: 100% 148 KiB 151.1kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Error-0.17024.txz: 100% 24 KiB 24.8kB/s 00:01 -[sync.ian.buetow.org] Fetching perl5-5.20.3_9.txz: 100% 13 MiB 6.9MB/s 00:02 -[sync.ian.buetow.org] Fetching cvsps-2.1_1.txz: 100% 41 KiB 42.1kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Authen-SASL-2.16_1.txz: 100% 44 KiB 45.1kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Digest-HMAC-1.03_1.txz: 100% 9 KiB 9.5kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-GSSAPI-0.28_1.txz: 100% 41 KiB 41.7kB/s 00:01 -[sync.ian.buetow.org] Fetching curl-7.48.0_1.txz: 100% 2 MiB 2.2MB/s 00:01 -[sync.ian.buetow.org] Fetching ca_root_nss-3.22.2.txz: 100% 324 KiB 331.4kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Net-SMTP-SSL-1.03.txz: 100% 11 KiB 10.8kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-IO-Socket-SSL-2.024.txz: 100% 153 KiB 156.4kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Net-SSLeay-1.72.txz: 100% 234 KiB 239.3kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-IO-Socket-IP-0.37.txz: 100% 27 KiB 27.4kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Socket-2.021.txz: 100% 37 KiB 38.0kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Mozilla-CA-20160104.txz: 100% 147 KiB 150.8kB/s 00:01 -Checking integrity... -[sync.ian.buetow.org] [1/12] Installing libyaml-0.1.6_2... -[sync.ian.buetow.org] [1/12] Extracting libyaml-0.1.6_2: 100% -[sync.ian.buetow.org] [2/12] Installing libedit-3.1.20150325_2... -[sync.ian.buetow.org] [2/12] Extracting libedit-3.1.20150325_2: 100% -[sync.ian.buetow.org] [3/12] Installing ruby-2.2.4,1... -[sync.ian.buetow.org] [3/12] Extracting ruby-2.2.4,1: 100% -[sync.ian.buetow.org] [4/12] Installing ruby22-gems-2.6.2... -[sync.ian.buetow.org] [4/12] Extracting ruby22-gems-2.6.2: 100% -[sync.ian.buetow.org] [5/12] Installing libxml2-2.9.3... -[sync.ian.buetow.org] [5/12] Extracting libxml2-2.9.3: 100% -[sync.ian.buetow.org] [6/12] Installing dmidecode-3.0... -[sync.ian.buetow.org] [6/12] Extracting dmidecode-3.0: 100% -[sync.ian.buetow.org] [7/12] Installing rubygem-json_pure-1.8.3... -[sync.ian.buetow.org] [7/12] Extracting rubygem-json_pure-1.8.3: 100% -[sync.ian.buetow.org] [8/12] Installing augeas-1.4.0... -[sync.ian.buetow.org] [8/12] Extracting augeas-1.4.0: 100% -[sync.ian.buetow.org] [9/12] Installing rubygem-facter-2.4.4... -[sync.ian.buetow.org] [9/12] Extracting rubygem-facter-2.4.4: 100% -[sync.ian.buetow.org] [10/12] Installing rubygem-hiera1-1.3.4_1... -[sync.ian.buetow.org] [10/12] Extracting rubygem-hiera1-1.3.4_1: 100% -[sync.ian.buetow.org] [11/12] Installing rubygem-ruby-augeas-0.5.0_2... -[sync.ian.buetow.org] [11/12] Extracting rubygem-ruby-augeas-0.5.0_2: 100% -[sync.ian.buetow.org] [12/12] Installing puppet38-3.8.4_1... -===> Creating users and/or groups. -Creating group 'puppet' with gid '814'. -Creating user 'puppet' with uid '814'. -[sync.ian.buetow.org] [12/12] Extracting puppet38-3.8.4_1: 100% -. -. -. -. -. -Looking up update.FreeBSD.org mirrors... 4 mirrors found. -Fetching public key from update4.freebsd.org... done. -Fetching metadata signature for 10.1-RELEASE from update4.freebsd.org... done. -Fetching metadata index... done. -Fetching 2 metadata files... done. -Inspecting system... done. -Preparing to download files... done. -Fetching 874 patches.....10....20....30.... -. -. -. -Applying patches... done. -Fetching 1594 files... -Installing updates... -done. -Info: Loading facts -Info: Loading facts -Info: Loading facts -Info: Loading facts -Could not retrieve fact='pkgng_version', resolution='<anonymous>': undefined method `pkgng_enabled' for Facter:Module -Warning: Config file /usr/local/etc/puppet/hiera.yaml not found, using Hiera defaults -Notice: Compiled catalog for sync.ian.buetow.org in environment production in 1.31 seconds -Warning: Found multiple default providers for package: pkgng, gem, pip; using pkgng -Info: Applying configuration version '1460192563' -Notice: /Stage[main]/S_base_freebsd/User[root]/shell: shell changed '/bin/csh' to '/bin/tcsh' -Notice: /Stage[main]/S_user::Root_files/S_user::All_files[root_user]/File[/root/user]/ensure: created -Notice: /Stage[main]/S_user::Root_files/S_user::My_files[root]/File[/root/userfiles]/ensure: created -Notice: /Stage[main]/S_user::Root_files/S_user::My_files[root]/File[/root/.task]/ensure: created -. -. -. -. -Notice: Finished catalog run in 206.09 seconds -</pre> -<br /> -<h2 style='display: inline'>Managing multiple Jails</h2><br /> -<br /> -<span>Of course I am operating multiple Jails on the same host this way with Puppet:</span><br /> -<br /> -<ul> -<li>A Jail for the MTA</li> -<li>A Jail for the Webserver</li> -<li>A Jail for BIND DNS server</li> -<li>A Jail for syncing data forth and back between various servers</li> -<li>A Jail for other personal (experimental) use</li> -<li>...etc</li> -</ul><br /> -<span>All done in a pretty automated manor. </span><br /> -<br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> |