diff options
| author | Paul Buetow <paul@buetow.org> | 2023-09-25 15:14:36 +0300 |
|---|---|---|
| committer | Paul Buetow <paul@buetow.org> | 2023-09-25 15:14:36 +0300 |
| commit | 17507d832ccb1c5215fe6a50b30c94e7b9a08b8c (patch) | |
| tree | dd9bbf8f968511ffd2c5689ace03a79903dbb780 | |
| parent | eabbbb111270fa995392faada0d875ba02673dca (diff) | |
Update content for gemtext
19 files changed, 1148 insertions, 3323 deletions
diff --git a/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi b/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi index f42c625a..c2cd31f4 100644 --- a/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi +++ b/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi @@ -108,6 +108,7 @@ Other related posts are: => ./2021-04-22-dtail-the-distributed-log-tail-program.gmi 2021-04-22 DTail - The distributed log tail program (You are currently reading this) => ./2022-03-06-the-release-of-dtail-4.0.0.gmi 2022-03-06 The release of DTail 4.0.0 => ./2022-10-30-installing-dtail-on-openbsd.gmi 2022-10-30 Installing DTail on OpenBSD +=> ./2023-09-25-dtail-usage-examples.gmi 2023-09-25 DTail usage examples E-Mail your comments to `foo@paul.cyou` :-) diff --git a/gemfeed/2022-03-06-the-release-of-dtail-4.0.0.gmi b/gemfeed/2022-03-06-the-release-of-dtail-4.0.0.gmi index 637740a4..63c4714f 100644 --- a/gemfeed/2022-03-06-the-release-of-dtail-4.0.0.gmi +++ b/gemfeed/2022-03-06-the-release-of-dtail-4.0.0.gmi @@ -291,6 +291,7 @@ Other related posts are: => ./2021-04-22-dtail-the-distributed-log-tail-program.gmi 2021-04-22 DTail - The distributed log tail program => ./2022-03-06-the-release-of-dtail-4.0.0.gmi 2022-03-06 The release of DTail 4.0.0 (You are currently reading this) => ./2022-10-30-installing-dtail-on-openbsd.gmi 2022-10-30 Installing DTail on OpenBSD +=> ./2023-09-25-dtail-usage-examples.gmi 2023-09-25 DTail usage examples Thanks! diff --git a/gemfeed/2022-10-30-installing-dtail-on-openbsd.gmi b/gemfeed/2022-10-30-installing-dtail-on-openbsd.gmi index 79d3c0ce..90273d3d 100644 --- a/gemfeed/2022-10-30-installing-dtail-on-openbsd.gmi +++ b/gemfeed/2022-10-30-installing-dtail-on-openbsd.gmi @@ -344,6 +344,7 @@ Other related posts are: => ./2021-04-22-dtail-the-distributed-log-tail-program.gmi 2021-04-22 DTail - The distributed log tail program => ./2022-03-06-the-release-of-dtail-4.0.0.gmi 2022-03-06 The release of DTail 4.0.0 => ./2022-10-30-installing-dtail-on-openbsd.gmi 2022-10-30 Installing DTail on OpenBSD (You are currently reading this) +=> ./2023-09-25-dtail-usage-examples.gmi 2023-09-25 DTail usage examples E-Mail your comments to `foo@paul.cyou` :-) diff --git a/gemfeed/2023-09-25-dtail-usage-examples.gmi b/gemfeed/2023-09-25-dtail-usage-examples.gmi new file mode 100644 index 00000000..62b1921e --- /dev/null +++ b/gemfeed/2023-09-25-dtail-usage-examples.gmi @@ -0,0 +1,244 @@ +# DTail usage examples + +> Published at 2023-09-25T14:57:42+03:00 + +Hey there. As I am pretty busy this month personally (I am now on Paternity Leave) and as I still want to post once monthly, the blog post of this month will only be some DTail usage examples. They're from the DTail documentation, but not all readers of my blog may be aware of those! + +DTail is a distributed DevOps tool for tailing, grepping, catting logs and other text files on many remote machines at once which I programmed in Go. + +=> https://dtail.dev + +``` + ,_---~~~~~----._ + _,,_,*^____ _____``*g*\"*, + ____ _____ _ _ / __/ /' ^. / \ ^@q f + | _ \_ _|_ _(_) | @f | ((@| |@)) l 0 _/ + | | | || |/ _` | | | \`/ \~____ / __ \_____/ \ + | |_| || | (_| | | | | _l__l_ I + |____/ |_|\__,_|_|_| } [______] I + ] | | | | + ] ~ ~ | + | Let's tail those logs! | + | | +``` + +DTail consists out of a server and several client binaries. In this post, I am showcasing their use! + +* Use `dtail` to follow logs +* Use `dtail` to aggregate logs while they are followed +* Use `dcat` to display logs and other text files already written +* Use `dgrep` to grep (search) logs and other text files already written +* Use `dmap` to aggregate logs and other text files already written +* `dserver` is the DTail server, where all the clients can connect to + +## Following logs + +The following example demonstrates how to follow logs of several servers at once. The server list is provided as a flat text file. The example filters all records containing the string `INFO`. Any other Go compatible regular expression can also be used instead of `INFO`. + +```shell +% dtail --servers serverlist.txt --grep INFO --files "/var/log/dserver/*.log" +``` + +Hint: you can also provide a comma separated server list, e.g.: `servers server1.example.org,server2.example.org:PORT,...` + +=> ./dtail-usage-examples/dtail.gif Tail example + +> Hint: You can also use the shorthand version (omitting the `--files`) + +```shell +% dtail --servers serverlist.txt --grep INFO "/var/log/dserver/*.log" +``` + +## Aggregating logs + +To run ad-hoc map-reduce aggregations on newly written log lines you must add a query. The following example follows all remote log lines and prints out every few seconds the result to standard output. + +> Hint: To run a map-reduce query across log lines written in the past, please use the `dmap` command instead. + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select sum($goroutines),sum($cgocalls), + last($time),max(lifetimeConnections)' +``` + +Beware: For map-reduce queries to work, you have to ensure that DTail supports your log format. Check out the documentaiton of the DTail query language and the DTail log formats on the DTail homepage for more information. + +=> ./dtail-usage-examples/dtail-map.gif Tail map-reduce example + +> Hint: You can also use the shorthand version: + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + 'from STATS select sum($goroutines),sum($cgocalls), + last($time),max(lifetimeConnections)' +``` + +Here is another example: + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +=> ./dtail-usage-examples/dtail-map2.gif Tail map-reduce example 2 + +You can also continuously append the results to a CSV file by adding `outfile append filename.csv` to the query: + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select ... outfile append result.csv' +``` + +## How to use `dcat` + +The following example demonstrates how to cat files (display the full content of the files) on several servers at once. + +As you can see in this example, a DTail client also creates a local log file of all received data in `~/log`. You can also use the `noColor` and `-plain` flags (this all also work with other DTail commands than `dcat`). + +```shell +% dcat --servers serverlist.txt --files /etc/hostname +``` + +=> ./dtail-usage-examples/dcat.gif Cat example + +> Hint: You can also use the shorthand version: + +```shell +% dcat --servers serverlist.txt /etc/hostname +``` + +## How to use `dgrep` + +The following example demonstrates how to grep files (display only the lines which match a given regular expression) of multiple servers at once. In this example, we look after some entries in `/etc/passwd`. This time, we don't provide the server list via an file but rather via a comma separated list directly on the command line. We also explore the `-before`, `-after` and `-max` flags (see animation). + +```shell +% dgrep --servers server1.example.org:2223 \ + --files /etc/passwd \ + --regex nologin +``` + +Generally, `dgrep` is also a very useful way to search historic application logs for certain content. + +=> ./dtail-usage-examples/dgrep.gif Grep example + +> Hint: `-regex` is an alias for `-grep`. + +## How to use `dmap` + +To run a map-reduce aggregation over logs written in the past, the `dmap` command can be used. The following example aggregates all map-reduce fields `dmap` will print interim results every few seconds. You can also write the result to an CSV file by adding `outfile result.csv` to the query. + +```shell +% dmap --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +Remember: For that to work, you have to make sure that DTail supports your log format. You can either use the ones already defined in `internal/mapr/logformat` or add an extension to support a custom log format. The example here works out of the box though, as DTail understands its own log format already. + +=> ./dtail-usage-examples/dmap.gif DMap example + +## How to use the DTail serverless mode + +Until now, all examples so far required to have remote server(s) to connect to. That makes sense, as after all DTail is a *distributed* tool. However, there are circumstances where you don't really need to connect to a server remotely. For example, you already have a login shell open to the server an all what you want is to run some queries directly on local log files. + +The serverless mode does not require any `dserver` up and running and therefore there is no networking/SSH involved. + +All commands shown so far also work in a serverless mode. All what needs to be done is to omit a server list. The DTail client then starts in serverless mode. + +### Serverless map-reduce query + +The following `dmap` example is the same as the previously shown one, but the difference is that it operates on a local log file directly: + +```shell +% dmap --files /var/log/dserver/dserver.log + --query 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +As a shorthand version the following command can be used: + +```shell +% dmap 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' \ + /var/log/dsever/dserver.log +``` + +You can also use a file input pipe as follows: + +```shell +% cat /var/log/dserver/dserver.log | \ + dmap 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +### Aggregating CSV files + +In essence, this works exactly like aggregating logs. All files operated on must be valid CSV files and the first line of the CSV must be the header. E.g.: + +```shell +% cat example.csv +name,lastname,age,profession +Michael,Jordan,40,Basketball player +Michael,Jackson,100,Singer +Albert,Einstein,200,Physician +% dmap --query 'select lastname,name where age > 40 logformat csv outfile result.csv' example.csv +% cat result.csv +lastname,name +Jackson,Michael +Einstein,Albert +``` + +DMap can also be used to query and aggregate CSV files from remote servers. + +### Other serverless commands + +The serverless mode works transparently with all other DTail commands. Here are some examples: + +```shell +% dtail /var/log/dserver/dserver.log +``` + +```shell +% dtail --logLevel trace /var/log/dserver/dserver.log +``` + +```shell +% dcat /etc/passwd +``` + +```shell +% dcat --plain /etc/passwd > /etc/test +# Should show no differences. +diff /etc/test /etc/passwd +``` + +```shell +% dgrep --regex ERROR --files /var/log/dserver/dsever.log +``` + +```shell +% dgrep --before 10 --after 10 --max 10 --grep ERROR /var/log/dserver/dsever.log +``` + +Use `--help` for more available options. Or go to the DTail page for more information! Hope you find DTail useful! + +Other related posts are: + +=> ./2021-04-22-dtail-the-distributed-log-tail-program.gmi 2021-04-22 DTail - The distributed log tail program +=> ./2022-03-06-the-release-of-dtail-4.0.0.gmi 2022-03-06 The release of DTail 4.0.0 +=> ./2022-10-30-installing-dtail-on-openbsd.gmi 2022-10-30 Installing DTail on OpenBSD +=> ./2023-09-25-dtail-usage-examples.gmi 2023-09-25 DTail usage examples (You are currently reading this) + +I hope you find the tools presented in this post useful! + +Paul + +E-Mail your comments to `foo@paul.cyou` :-) + +=> ../ Back to the main site diff --git a/gemfeed/2023-09-25-dtail-usage-examples.gmi.tpl b/gemfeed/2023-09-25-dtail-usage-examples.gmi.tpl new file mode 100644 index 00000000..1fa89cdc --- /dev/null +++ b/gemfeed/2023-09-25-dtail-usage-examples.gmi.tpl @@ -0,0 +1,241 @@ +# DTail usage examples + +> Published at 2023-09-25T14:57:42+03:00 + +Hey there. As I am pretty busy this month personally (I am now on Paternity Leave) and as I still want to post once monthly, the blog post of this month will only be some DTail usage examples. They're from the DTail documentation, but not all readers of my blog may be aware of those! + +DTail is a distributed DevOps tool for tailing, grepping, catting logs and other text files on many remote machines at once which I programmed in Go. + +=> https://dtail.dev + +``` + ,_---~~~~~----._ + _,,_,*^____ _____``*g*\"*, + ____ _____ _ _ / __/ /' ^. / \ ^@q f + | _ \_ _|_ _(_) | @f | ((@| |@)) l 0 _/ + | | | || |/ _` | | | \`/ \~____ / __ \_____/ \ + | |_| || | (_| | | | | _l__l_ I + |____/ |_|\__,_|_|_| } [______] I + ] | | | | + ] ~ ~ | + | Let's tail those logs! | + | | +``` + +DTail consists out of a server and several client binaries. In this post, I am showcasing their use! + +* Use `dtail` to follow logs +* Use `dtail` to aggregate logs while they are followed +* Use `dcat` to display logs and other text files already written +* Use `dgrep` to grep (search) logs and other text files already written +* Use `dmap` to aggregate logs and other text files already written +* `dserver` is the DTail server, where all the clients can connect to + +## Following logs + +The following example demonstrates how to follow logs of several servers at once. The server list is provided as a flat text file. The example filters all records containing the string `INFO`. Any other Go compatible regular expression can also be used instead of `INFO`. + +```shell +% dtail --servers serverlist.txt --grep INFO --files "/var/log/dserver/*.log" +``` + +Hint: you can also provide a comma separated server list, e.g.: `servers server1.example.org,server2.example.org:PORT,...` + +=> ./dtail-usage-examples/dtail.gif Tail example + +> Hint: You can also use the shorthand version (omitting the `--files`) + +```shell +% dtail --servers serverlist.txt --grep INFO "/var/log/dserver/*.log" +``` + +## Aggregating logs + +To run ad-hoc map-reduce aggregations on newly written log lines you must add a query. The following example follows all remote log lines and prints out every few seconds the result to standard output. + +> Hint: To run a map-reduce query across log lines written in the past, please use the `dmap` command instead. + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select sum($goroutines),sum($cgocalls), + last($time),max(lifetimeConnections)' +``` + +Beware: For map-reduce queries to work, you have to ensure that DTail supports your log format. Check out the documentaiton of the DTail query language and the DTail log formats on the DTail homepage for more information. + +=> ./dtail-usage-examples/dtail-map.gif Tail map-reduce example + +> Hint: You can also use the shorthand version: + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + 'from STATS select sum($goroutines),sum($cgocalls), + last($time),max(lifetimeConnections)' +``` + +Here is another example: + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +=> ./dtail-usage-examples/dtail-map2.gif Tail map-reduce example 2 + +You can also continuously append the results to a CSV file by adding `outfile append filename.csv` to the query: + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select ... outfile append result.csv' +``` + +## How to use `dcat` + +The following example demonstrates how to cat files (display the full content of the files) on several servers at once. + +As you can see in this example, a DTail client also creates a local log file of all received data in `~/log`. You can also use the `noColor` and `-plain` flags (this all also work with other DTail commands than `dcat`). + +```shell +% dcat --servers serverlist.txt --files /etc/hostname +``` + +=> ./dtail-usage-examples/dcat.gif Cat example + +> Hint: You can also use the shorthand version: + +```shell +% dcat --servers serverlist.txt /etc/hostname +``` + +## How to use `dgrep` + +The following example demonstrates how to grep files (display only the lines which match a given regular expression) of multiple servers at once. In this example, we look after some entries in `/etc/passwd`. This time, we don't provide the server list via an file but rather via a comma separated list directly on the command line. We also explore the `-before`, `-after` and `-max` flags (see animation). + +```shell +% dgrep --servers server1.example.org:2223 \ + --files /etc/passwd \ + --regex nologin +``` + +Generally, `dgrep` is also a very useful way to search historic application logs for certain content. + +=> ./dtail-usage-examples/dgrep.gif Grep example + +> Hint: `-regex` is an alias for `-grep`. + +## How to use `dmap` + +To run a map-reduce aggregation over logs written in the past, the `dmap` command can be used. The following example aggregates all map-reduce fields `dmap` will print interim results every few seconds. You can also write the result to an CSV file by adding `outfile result.csv` to the query. + +```shell +% dmap --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +Remember: For that to work, you have to make sure that DTail supports your log format. You can either use the ones already defined in `internal/mapr/logformat` or add an extension to support a custom log format. The example here works out of the box though, as DTail understands its own log format already. + +=> ./dtail-usage-examples/dmap.gif DMap example + +## How to use the DTail serverless mode + +Until now, all examples so far required to have remote server(s) to connect to. That makes sense, as after all DTail is a *distributed* tool. However, there are circumstances where you don't really need to connect to a server remotely. For example, you already have a login shell open to the server an all what you want is to run some queries directly on local log files. + +The serverless mode does not require any `dserver` up and running and therefore there is no networking/SSH involved. + +All commands shown so far also work in a serverless mode. All what needs to be done is to omit a server list. The DTail client then starts in serverless mode. + +### Serverless map-reduce query + +The following `dmap` example is the same as the previously shown one, but the difference is that it operates on a local log file directly: + +```shell +% dmap --files /var/log/dserver/dserver.log + --query 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +As a shorthand version the following command can be used: + +```shell +% dmap 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' \ + /var/log/dsever/dserver.log +``` + +You can also use a file input pipe as follows: + +```shell +% cat /var/log/dserver/dserver.log | \ + dmap 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +### Aggregating CSV files + +In essence, this works exactly like aggregating logs. All files operated on must be valid CSV files and the first line of the CSV must be the header. E.g.: + +```shell +% cat example.csv +name,lastname,age,profession +Michael,Jordan,40,Basketball player +Michael,Jackson,100,Singer +Albert,Einstein,200,Physician +% dmap --query 'select lastname,name where age > 40 logformat csv outfile result.csv' example.csv +% cat result.csv +lastname,name +Jackson,Michael +Einstein,Albert +``` + +DMap can also be used to query and aggregate CSV files from remote servers. + +### Other serverless commands + +The serverless mode works transparently with all other DTail commands. Here are some examples: + +```shell +% dtail /var/log/dserver/dserver.log +``` + +```shell +% dtail --logLevel trace /var/log/dserver/dserver.log +``` + +```shell +% dcat /etc/passwd +``` + +```shell +% dcat --plain /etc/passwd > /etc/test +# Should show no differences. +diff /etc/test /etc/passwd +``` + +```shell +% dgrep --regex ERROR --files /var/log/dserver/dsever.log +``` + +```shell +% dgrep --before 10 --after 10 --max 10 --grep ERROR /var/log/dserver/dsever.log +``` + +Use `--help` for more available options. Or go to the DTail page for more information! Hope you find DTail useful! + +Other related posts are: + +<< template::inline::index dtail + +I hope you find the tools presented in this post useful! + +Paul + +E-Mail your comments to `foo@paul.cyou` :-) + +=> ../ Back to the main site diff --git a/gemfeed/DRAFT-dtail-usage-examples.gmi b/gemfeed/DRAFT-dtail-usage-examples.gmi new file mode 100644 index 00000000..77b42aa7 --- /dev/null +++ b/gemfeed/DRAFT-dtail-usage-examples.gmi @@ -0,0 +1,230 @@ +# DTail usage examples + +DTail is a distributed DevOps tool for tailing, grepping, catting logs and other text files on many remote machines at once which I programmed in Go. + +=> https://dtail.dev + +``` + ,_---~~~~~----._ + _,,_,*^____ _____``*g*\"*, + ____ _____ _ _ / __/ /' ^. / \ ^@q f + | _ \_ _|_ _(_) | @f | ((@| |@)) l 0 _/ + | | | || |/ _` | | | \`/ \~____ / __ \_____/ \ + | |_| || | (_| | | | | _l__l_ I + |____/ |_|\__,_|_|_| } [______] I + ] | | | | + ] ~ ~ | + | Let's tail those logs! | + | | +``` + +DTail consists out of a server and several client binaries. In this post, I am showcasing their use! + +## Following logs + +The following example demonstrates how to follow logs of several servers at once. The server list is provided as a flat text file. The example filters all records containing the string `INFO`. Any other Go compatible regular expression can also be used instead of `INFO`. + +```shell +% dtail --servers serverlist.txt --grep INFO --files "/var/log/dserver/*.log" +``` + +Hint: you can also provide a comma separated server list, e.g.: `servers server1.example.org,server2.example.org:PORT,...` + +=> ./dtail-usage-examples/dtail.gif Tail example + +> Hint: You can also use the shorthand version (omitting the `--files`) + +```shell +% dtail --servers serverlist.txt --grep INFO "/var/log/dserver/*.log" +``` + +## Aggregating logs + +To run ad-hoc map-reduce aggregations on newly written log lines you must add a query. The following example follows all remote log lines and prints out every few seconds the result to standard output. + +> Hint: To run a map-reduce query across log lines written in the past, please use the `dmap` command instead. + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select sum($goroutines),sum($cgocalls), + last($time),max(lifetimeConnections)' +``` + +Beware: For map-reduce queries to work, you have to ensure that DTail supports your log format. Check out the documentaiton of the DTail query language and the DTail log formats on the DTail homepage for more information. + +=> ./dtail-usage-examples/dtail-map.gif Tail map-reduce example + +> Hint: You can also use the shorthand version: + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + 'from STATS select sum($goroutines),sum($cgocalls), + last($time),max(lifetimeConnections)' +``` + +Here is another example: + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +=> ./dtail-usage-examples/dtail-map2.gif Tail map-reduce example 2 + +You can also continuously append the results to a CSV file by adding `outfile append filename.csv` to the query: + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select ... outfile append result.csv' +``` + +## How to use `dcat` + +The following example demonstrates how to cat files (display the full content of the files) on several servers at once. + +As you can see in this example, a DTail client also creates a local log file of all received data in `~/log`. You can also use the `noColor` and `-plain` flags (this all also work with other DTail commands than `dcat`). + +```shell +% dcat --servers serverlist.txt --files /etc/hostname +``` + +=> ./dtail-usage-examples/dcat.gif Cat example + +> Hint: You can also use the shorthand version: + +```shell +% dcat --servers serverlist.txt /etc/hostname +``` + +## How to use `dgrep` + +The following example demonstrates how to grep files (display only the lines which match a given regular expression) of multiple servers at once. In this example, we look after some entries in `/etc/passwd`. This time, we don't provide the server list via an file but rather via a comma separated list directly on the command line. We also explore the `-before`, `-after` and `-max` flags (see animation). + +```shell +% dgrep --servers server1.example.org:2223 \ + --files /etc/passwd \ + --regex nologin +``` + +Generally, `dgrep` is also a very useful way to search historic application logs for certain content. + +=> ./dtail-usage-examples/dgrep.gif Grep example + +Hint: `-regex` is an alias for `-grep`. + +## How to use `dmap` + +To run a map-reduce aggregation over logs written in the past, the `dmap` command can be used. The following example aggregates all map-reduce fields `dmap` will print interim results every few seconds. You can also write the result to an CSV file by adding `outfile result.csv` to the query. + +```shell +% dmap --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +Remember: For that to work, you have to make sure that DTail supports your log format. You can either use the ones already defined in `internal/mapr/logformat` or add an extension to support a custom log format. The example here works out of the box though, as DTail understands its own log format already. + +=> ./dtail-usage-examples/dmap.gif DMap example + +## How to use the DTail serverless mode + +Until now, all examples so far required to have remote server(s) to connect to. That makes sense, as after all DTail is a *distributed* tool. However, there are circumstances where you don't really need to connect to a server remotely. For example, you already have a login shell open to the server an all what you want is to run some queries directly on local log files. + +The serverless mode does not require any `dserver` up and running and therefore there is no networking/SSH involved. + +All commands shown so far also work in a serverless mode. All what needs to be done is to omit a server list. The DTail client then starts in serverless mode. + +### Serverless map-reduce query + +The following `dmap` example is the same as the previously shown one, but the difference is that it operates on a local log file directly: + +```shell +% dmap --files /var/log/dserver/dserver.log + --query 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +As a shorthand version the following command can be used: + +```shell +% dmap 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, +lifetimeConnections group by $hostname order by max($cgocalls)' /var/log/dsever/dserver.log +``` + +You can also use a file input pipe as follows: + +```shell +% cat /var/log/dserver/dserver.log | \ + dmap 'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg, + lifetimeConnections group by $hostname order by max($cgocalls)' +``` + +### Aggregating CSV files + +In essence, this works exactly like aggregating logs. All files operated on must be valid CSV files and the first line of the CSV must be the header. E.g.: + +```shell +% cat example.csv +name,lastname,age,profession +Michael,Jordan,40,Basketball player +Michael,Jackson,100,Singer +Albert,Einstein,200,Physician +% dmap --query 'select lastname,name where age > 40 logformat csv outfile result.csv' example.csv +% cat result.csv +lastname,name +Jackson,Michael +Einstein,Albert +``` + +DMap can also be used to query and aggregate CSV files from remote servers. + +### Other serverless commands + +The serverless mode works transparently with all other DTail commands. Here are some examples: + +```shell +% dtail /var/log/dserver/dserver.log +``` + +```shell +% dtail --logLevel trace /var/log/dserver/dserver.log +``` + +```shell +% dcat /etc/passwd +``` + +```shell +% dcat --plain /etc/passwd > /etc/test +# Should show no differences. +diff /etc/test /etc/passwd +``` + +```shell +% dgrep --regex ERROR --files /var/log/dserver/dsever.log +``` + +```shell +% dgrep --before 10 --after 10 --max 10 --grep ERROR /var/log/dserver/dsever.log + +Use `--help` for more available options. Or go to the DTail page for more information! Hope you find DTail useful! + +Other related posts are: + +=> ./2021-04-22-dtail-the-distributed-log-tail-program.gmi 2021-04-22 DTail - The distributed log tail program +=> ./2022-03-06-the-release-of-dtail-4.0.0.gmi 2022-03-06 The release of DTail 4.0.0 +=> ./2022-10-30-installing-dtail-on-openbsd.gmi 2022-10-30 Installing DTail on OpenBSD + +Thanks! + +Paul + +E-Mail your comments to `foo@paul.cyou` :-) + +=> ../ Back to the main site diff --git a/gemfeed/DRAFT-dtail-usage-examples.gmi.tpl b/gemfeed/W index 29538df4..7b753922 100644 --- a/gemfeed/DRAFT-dtail-usage-examples.gmi.tpl +++ b/gemfeed/W @@ -4,6 +4,20 @@ DTail is a distributed DevOps tool for tailing, grepping, catting logs and other => https://dtail.dev +``` + ,_---~~~~~----._ + _,,_,*^____ _____``*g*\"*, + ____ _____ _ _ / __/ /' ^. / \ ^@q f + | _ \_ _|_ _(_) | @f | ((@| |@)) l 0 _/ + | | | || |/ _` | | | \`/ \~____ / __ \_____/ \ + | |_| || | (_| | | | | _l__l_ I + |____/ |_|\__,_|_|_| } [______] I + ] | | | | + ] ~ ~ | + | Let's tail those logs! | + | | +``` + DTail consists out of a server and several client binaries. In this post, I am showcasing their use! ## Following logs @@ -37,7 +51,7 @@ To run ad-hoc map-reduce aggregations on newly written log lines you must add a last($time),max(lifetimeConnections)' ``` -> Beware: For map-reduce queries to work, you have to ensure that DTail supports your log format. Check out the documentaiton of the DTail query language and the DTail log formats on the DTail homepage for more information. +Beware: For map-reduce queries to work, you have to ensure that DTail supports your log format. Check out the documentaiton of the DTail query language and the DTail log formats on the DTail homepage for more information. => ./dtail-usage-examples/dtail-map.gif Tail map-reduce example diff --git a/gemfeed/atom.xml b/gemfeed/atom.xml index f41efc63..4ac849b6 100644 --- a/gemfeed/atom.xml +++ b/gemfeed/atom.xml @@ -1,12 +1,332 @@ <?xml version="1.0" encoding="utf-8"?> <feed xmlns="http://www.w3.org/2005/Atom"> - <updated>2023-08-22T00:06:26+03:00</updated> + <updated>2023-09-25T15:14:22+03:00</updated> <title>foo.zone feed</title> <subtitle>To be in the .zone!</subtitle> <link href="gemini://foo.zone/gemfeed/atom.xml" rel="self" /> <link href="gemini://foo.zone/" /> <id>gemini://foo.zone/</id> <entry> + <title>DTail usage examples</title> + <link href="gemini://foo.zone/gemfeed/2023-09-25-dtail-usage-examples.gmi" /> + <id>gemini://foo.zone/gemfeed/2023-09-25-dtail-usage-examples.gmi</id> + <updated>2023-09-25T14:57:42+03:00</updated> + <author> + <name>Paul Buetow aka snonux</name> + <email>paul@dev.buetow.org</email> + </author> + <summary>Hey there. As I am pretty busy this month personally (I am now on Paternity Leave) and as I still want to post once monthly, the blog post of this month will only be some DTail usage examples. They're from the DTail documentation, but not all readers of my blog may be aware of those!</summary> + <content type="xhtml"> + <div xmlns="http://www.w3.org/1999/xhtml"> + <h1 style='display: inline'>DTail usage examples</h1><br /> +<br /> +<span class='quote'>Published at 2023-09-25T14:57:42+03:00</span><br /> +<br /> +<span>Hey there. As I am pretty busy this month personally (I am now on Paternity Leave) and as I still want to post once monthly, the blog post of this month will only be some DTail usage examples. They're from the DTail documentation, but not all readers of my blog may be aware of those!</span><br /> +<br /> +<span>DTail is a distributed DevOps tool for tailing, grepping, catting logs and other text files on many remote machines at once which I programmed in Go.</span><br /> +<br /> +<a class='textlink' href='https://dtail.dev'>https://dtail.dev</a><br /> +<br /> +<pre> + ,_---~~~~~----._ + _,,_,*^____ _____``*g*\"*, + ____ _____ _ _ / __/ /' ^. / \ ^@q f + | _ \_ _|_ _(_) | @f | ((@| |@)) l 0 _/ + | | | || |/ _` | | | \`/ \~____ / __ \_____/ \ + | |_| || | (_| | | | | _l__l_ I + |____/ |_|\__,_|_|_| } [______] I + ] | | | | + ] ~ ~ | + | Let's tail those logs! | + | | +</pre> +<br /> +<span>DTail consists out of a server and several client binaries. In this post, I am showcasing their use!</span><br /> +<br /> +<ul> +<li>Use <span class='inlinecode'>dtail</span> to follow logs</li> +<li>Use <span class='inlinecode'>dtail</span> to aggregate logs while they are followed</li> +<li>Use <span class='inlinecode'>dcat</span> to display logs and other text files already written</li> +<li>Use <span class='inlinecode'>dgrep</span> to grep (search) logs and other text files already written</li> +<li>Use <span class='inlinecode'>dmap</span> to aggregate logs and other text files already written</li> +<li><span class='inlinecode'>dserver</span> is the DTail server, where all the clients can connect to</li> +</ul><br /> +<h2 style='display: inline'>Following logs</h2><br /> +<br /> +<span>The following example demonstrates how to follow logs of several servers at once. The server list is provided as a flat text file. The example filters all records containing the string <span class='inlinecode'>INFO</span>. Any other Go compatible regular expression can also be used instead of <span class='inlinecode'>INFO</span>.</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt --grep INFO --files <font color="#FF0000">"/var/log/dserver/*.log"</font> +</pre> +<br /> +<span>Hint: you can also provide a comma separated server list, e.g.: <span class='inlinecode'>servers server1.example.org,server2.example.org:PORT,...</span></span><br /> +<br /> +<a href='./dtail-usage-examples/dtail.gif'><img alt='Tail example' title='Tail example' src='./dtail-usage-examples/dtail.gif' /></a><br /> +<br /> +<span class='quote'>Hint: You can also use the shorthand version (omitting the <span class='inlinecode'>--files</span>)</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt --grep INFO <font color="#FF0000">"/var/log/dserver/*.log"</font> +</pre> +<br /> +<h2 style='display: inline'>Aggregating logs</h2><br /> +<br /> +<span>To run ad-hoc map-reduce aggregations on newly written log lines you must add a query. The following example follows all remote log lines and prints out every few seconds the result to standard output.</span><br /> +<br /> +<span class='quote'>Hint: To run a map-reduce query across log lines written in the past, please use the <span class='inlinecode'>dmap</span> command instead.</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt <font color="#990000">\</font> + --files <font color="#FF0000">'/var/log/dserver/*.log'</font> <font color="#990000">\</font> + --query <font color="#FF0000">'from STATS select sum($goroutines),sum($cgocalls),</font> +<font color="#FF0000"> last($time),max(lifetimeConnections)'</font> +</pre> +<br /> +<span>Beware: For map-reduce queries to work, you have to ensure that DTail supports your log format. Check out the documentaiton of the DTail query language and the DTail log formats on the DTail homepage for more information.</span><br /> +<br /> +<a href='./dtail-usage-examples/dtail-map.gif'><img alt='Tail map-reduce example' title='Tail map-reduce example' src='./dtail-usage-examples/dtail-map.gif' /></a><br /> +<br /> +<span class='quote'>Hint: You can also use the shorthand version:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt <font color="#990000">\</font> + --files <font color="#FF0000">'/var/log/dserver/*.log'</font> <font color="#990000">\</font> + <font color="#FF0000">'from STATS select sum($goroutines),sum($cgocalls),</font> +<font color="#FF0000"> last($time),max(lifetimeConnections)'</font> +</pre> +<br /> +<span>Here is another example:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt <font color="#990000">\</font> + --files <font color="#FF0000">'/var/log/dserver/*.log'</font> <font color="#990000">\</font> + --query <font color="#FF0000">'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg,</font> +<font color="#FF0000"> lifetimeConnections group by $hostname order by max($cgocalls)'</font> +</pre> +<br /> +<a href='./dtail-usage-examples/dtail-map2.gif'><img alt='Tail map-reduce example 2' title='Tail map-reduce example 2' src='./dtail-usage-examples/dtail-map2.gif' /></a><br /> +<br /> +<span>You can also continuously append the results to a CSV file by adding <span class='inlinecode'>outfile append filename.csv</span> to the query:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --servers serverlist<font color="#990000">.</font>txt <font color="#990000">\</font> + --files <font color="#FF0000">'/var/log/dserver/*.log'</font> <font color="#990000">\</font> + --query <font color="#FF0000">'from STATS select ... outfile append result.csv'</font> +</pre> +<br /> +<h2 style='display: inline'>How to use <span class='inlinecode'>dcat</span></h2><br /> +<br /> +<span>The following example demonstrates how to cat files (display the full content of the files) on several servers at once.</span><br /> +<br /> +<span>As you can see in this example, a DTail client also creates a local log file of all received data in <span class='inlinecode'>~/log</span>. You can also use the <span class='inlinecode'>noColor</span> and <span class='inlinecode'>-plain</span> flags (this all also work with other DTail commands than <span class='inlinecode'>dcat</span>).</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dcat --servers serverlist<font color="#990000">.</font>txt --files /etc/hostname +</pre> +<br /> +<a href='./dtail-usage-examples/dcat.gif'><img alt='Cat example' title='Cat example' src='./dtail-usage-examples/dcat.gif' /></a><br /> +<br /> +<span class='quote'>Hint: You can also use the shorthand version:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dcat --servers serverlist<font color="#990000">.</font>txt /etc/hostname +</pre> +<br /> +<h2 style='display: inline'>How to use <span class='inlinecode'>dgrep</span></h2><br /> +<br /> +<span>The following example demonstrates how to grep files (display only the lines which match a given regular expression) of multiple servers at once. In this example, we look after some entries in <span class='inlinecode'>/etc/passwd</span>. This time, we don't provide the server list via an file but rather via a comma separated list directly on the command line. We also explore the <span class='inlinecode'>-before</span>, <span class='inlinecode'>-after</span> and <span class='inlinecode'>-max</span> flags (see animation).</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dgrep --servers server1<font color="#990000">.</font>example<font color="#990000">.</font>org<font color="#990000">:</font><font color="#993399">2223</font> <font color="#990000">\</font> + --files /etc/passwd <font color="#990000">\</font> + --regex nologin +</pre> +<br /> +<span>Generally, <span class='inlinecode'>dgrep</span> is also a very useful way to search historic application logs for certain content.</span><br /> +<br /> +<a href='./dtail-usage-examples/dgrep.gif'><img alt='Grep example' title='Grep example' src='./dtail-usage-examples/dgrep.gif' /></a><br /> +<br /> +<span class='quote'>Hint: <span class='inlinecode'>-regex</span> is an alias for <span class='inlinecode'>-grep</span>.</span><br /> +<br /> +<h2 style='display: inline'>How to use <span class='inlinecode'>dmap</span></h2><br /> +<br /> +<span>To run a map-reduce aggregation over logs written in the past, the <span class='inlinecode'>dmap</span> command can be used. The following example aggregates all map-reduce fields <span class='inlinecode'>dmap</span> will print interim results every few seconds. You can also write the result to an CSV file by adding <span class='inlinecode'>outfile result.csv</span> to the query.</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dmap --servers serverlist<font color="#990000">.</font>txt <font color="#990000">\</font> + --files <font color="#FF0000">'/var/log/dserver/*.log'</font> <font color="#990000">\</font> + --query <font color="#FF0000">'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg,</font> +<font color="#FF0000"> lifetimeConnections group by $hostname order by max($cgocalls)'</font> +</pre> +<br /> +<span>Remember: For that to work, you have to make sure that DTail supports your log format. You can either use the ones already defined in <span class='inlinecode'>internal/mapr/logformat</span> or add an extension to support a custom log format. The example here works out of the box though, as DTail understands its own log format already. </span><br /> +<br /> +<a href='./dtail-usage-examples/dmap.gif'><img alt='DMap example' title='DMap example' src='./dtail-usage-examples/dmap.gif' /></a><br /> +<br /> +<h2 style='display: inline'>How to use the DTail serverless mode</h2><br /> +<br /> +<span>Until now, all examples so far required to have remote server(s) to connect to. That makes sense, as after all DTail is a *distributed* tool. However, there are circumstances where you don't really need to connect to a server remotely. For example, you already have a login shell open to the server an all what you want is to run some queries directly on local log files.</span><br /> +<br /> +<span>The serverless mode does not require any <span class='inlinecode'>dserver</span> up and running and therefore there is no networking/SSH involved. </span><br /> +<br /> +<span>All commands shown so far also work in a serverless mode. All what needs to be done is to omit a server list. The DTail client then starts in serverless mode.</span><br /> +<br /> +<h3 style='display: inline'>Serverless map-reduce query</h3><br /> +<br /> +<span>The following <span class='inlinecode'>dmap</span> example is the same as the previously shown one, but the difference is that it operates on a local log file directly:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dmap --files /var/log/dserver/dserver<font color="#990000">.</font>log + --query <font color="#FF0000">'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg,</font> +<font color="#FF0000"> lifetimeConnections group by $hostname order by max($cgocalls)'</font> +</pre> +<br /> +<span>As a shorthand version the following command can be used:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dmap <font color="#FF0000">'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg,</font> +<font color="#FF0000"> lifetimeConnections group by $hostname order by max($cgocalls)'</font> <font color="#990000">\</font> + /var/log/dsever/dserver<font color="#990000">.</font>log +</pre> +<br /> +<span>You can also use a file input pipe as follows:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> cat /var/log/dserver/dserver<font color="#990000">.</font>log <font color="#990000">|</font> <font color="#990000">\</font> + dmap <font color="#FF0000">'from STATS select $hostname,max($goroutines),max($cgocalls),$loadavg,</font> +<font color="#FF0000"> lifetimeConnections group by $hostname order by max($cgocalls)'</font> +</pre> +<br /> +<h3 style='display: inline'>Aggregating CSV files</h3><br /> +<br /> +<span>In essence, this works exactly like aggregating logs. All files operated on must be valid CSV files and the first line of the CSV must be the header. E.g.:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> cat example<font color="#990000">.</font>csv +name<font color="#990000">,</font>lastname<font color="#990000">,</font>age<font color="#990000">,</font>profession +Michael<font color="#990000">,</font>Jordan<font color="#990000">,</font><font color="#993399">40</font><font color="#990000">,</font>Basketball player +Michael<font color="#990000">,</font>Jackson<font color="#990000">,</font><font color="#993399">100</font><font color="#990000">,</font>Singer +Albert<font color="#990000">,</font>Einstein<font color="#990000">,</font><font color="#993399">200</font><font color="#990000">,</font>Physician +<font color="#990000">%</font> dmap --query <font color="#FF0000">'select lastname,name where age > 40 logformat csv outfile result.csv'</font> example<font color="#990000">.</font>csv +<font color="#990000">%</font> cat result<font color="#990000">.</font>csv +lastname<font color="#990000">,</font>name +Jackson<font color="#990000">,</font>Michael +Einstein<font color="#990000">,</font>Albert +</pre> +<br /> +<span>DMap can also be used to query and aggregate CSV files from remote servers.</span><br /> +<br /> +<h3 style='display: inline'>Other serverless commands</h3><br /> +<br /> +<span>The serverless mode works transparently with all other DTail commands. Here are some examples:</span><br /> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail /var/log/dserver/dserver<font color="#990000">.</font>log +</pre> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dtail --logLevel trace /var/log/dserver/dserver<font color="#990000">.</font>log +</pre> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dcat /etc/passwd +</pre> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dcat --plain /etc/passwd <font color="#990000">></font> /etc/test +<i><font color="#9A1900"># Should show no differences.</font></i> +diff /etc/test /etc/passwd +</pre> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dgrep --regex ERROR --files /var/log/dserver/dsever<font color="#990000">.</font>log +</pre> +<br /> +<!-- Generator: GNU source-highlight 3.1.9 +by Lorenzo Bettini +http://www.lorenzobettini.it +http://www.gnu.org/software/src-highlite --> +<pre><font color="#990000">%</font> dgrep --before <font color="#993399">10</font> --after <font color="#993399">10</font> --max <font color="#993399">10</font> --grep ERROR /var/log/dserver/dsever<font color="#990000">.</font>log +</pre> +<br /> +<span>Use <span class='inlinecode'>--help</span> for more available options. Or go to the DTail page for more information! Hope you find DTail useful!</span><br /> +<br /> +<span>Other related posts are:</span><br /> +<br /> +<a class='textlink' href='./2021-04-22-dtail-the-distributed-log-tail-program.html'>2021-04-22 DTail - The distributed log tail program</a><br /> +<a class='textlink' href='./2022-03-06-the-release-of-dtail-4.0.0.html'>2022-03-06 The release of DTail 4.0.0</a><br /> +<a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD</a><br /> +<a class='textlink' href='./2023-09-25-dtail-usage-examples.html'>2023-09-25 DTail usage examples (You are currently reading this)</a><br /> +<br /> +<span>I hope you find the tools presented in this post useful!</span><br /> +<br /> +<span>Paul</span><br /> +<br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> +<br /> +<a class='textlink' href='../'>Back to the main site</a><br /> + </div> + </content> + </entry> + <entry> <title>Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect</title> <link href="gemini://foo.zone/gemfeed/2023-08-20-site-reliability-engineering-part-3.gmi" /> <id>gemini://foo.zone/gemfeed/2023-08-20-site-reliability-engineering-part-3.gmi</id> @@ -74,7 +394,7 @@ <br /> <span>The fourth part of this blog series will be published soon :-)</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -139,7 +459,7 @@ <br /> <a class='textlink' href='./2023-08-20-site-reliability-engineering-part-3.html'>2023-08-20 Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -213,7 +533,7 @@ DC on fire: <br /> <a class='textlink' href='./2023-08-19-site-reliability-engineering-part-2.html'>2023-08-19 Site Reliability Engineering - Part 2: Operational Balance in SRE</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -341,7 +661,7 @@ http://www.gnu.org/software/src-highlite --> <a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again²</a><br /> <a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³ (You are currently reading this)</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -664,7 +984,7 @@ http://www.gnu.org/software/src-highlite --> <a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes</a><br /> <a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes (You are currently reading this)</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../resources.html'>More books and other resources I found useful.</a><br /> <a class='textlink' href='../'>Back to the main site</a><br /> @@ -942,7 +1262,7 @@ http://www.gnu.org/software/src-highlite --> <br /> <span>Gogios is a lightweight and straightforward monitoring tool that is perfect for small-scale environments. With its compatibility with the Nagios Check API, email notifications, and CRON-based scheduling, Gogios offers an easy-to-use solution for those looking to monitor a limited number of resources. I personally use it to execute around 500 checks on my personal server infrastructure. I am very happy with this solution.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -1051,7 +1371,7 @@ http://www.gnu.org/software/src-highlite --> <a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes (You are currently reading this)</a><br /> <a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../resources.html'>More books and other resources I found useful.</a><br /> <a class='textlink' href='../'>Back to the main site</a><br /> @@ -1507,7 +1827,7 @@ ok codeberg<font color="#990000">.</font>org/snonux/algorithms/sort <fo <br /> <span>I won't write any benchmark for sleep sort; that will be done for the algorithms to come in this series :-).</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -1662,7 +1982,7 @@ ok codeberg<font color="#990000">.</font>org/snonux/algorithms/sort <fo <a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes</a><br /> <a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -1835,7 +2155,7 @@ The remaining content of the Gemtext file<font color="#990000">...</font> <a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again² (You are currently reading this)</a><br /> <a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -1943,7 +2263,7 @@ The remaining content of the Gemtext file<font color="#990000">...</font> <a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes</a><br /> <a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../resources.html'>More books and other resources I found useful.</a><br /> <a class='textlink' href='../'>Back to the main site</a><br /> @@ -2038,7 +2358,7 @@ The remaining content of the Gemtext file<font color="#990000">...</font> <br /> <span>There are some days at work you feel drained afterwards and think you didn't progress towards your goals at all. It's more challenging to shut down from work after such a day. A quick hack is to work on a quick win before the end of the day, giving you a sense of accomplishment after all. Another way is to make progress on your fun passion project after work. It must not be work-related, but a sense of accomplishment will still be there.</span><br /> <span> </span><br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -2195,7 +2515,7 @@ Art by Joan Stark <br /> <span>And, of course, GrapheneOS is an open-source project. This is a good thing; however, on the other side, nobody can guarantee that the OS will not break or will not damage your phone. You have to trust the GrapheneOS project and donate to the project so they can keep up with the great work. But I rather trust the GrapheneOS team than big tech. </span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -2319,7 +2639,7 @@ Art by Joan Stark <br /> <span>Am I a Java expert now? No, by far not. But I am better now than before :-).</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -2452,7 +2772,7 @@ nmap ,<b><font color="#0000FF">i</font></b> !wpbpaste<font color="#FF6600"><C <br /> <a class='textlink' href='https://github.com/NvChad/NvChad'>https://github.com/NvChad/NvChad</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -2816,8 +3136,9 @@ REMOTE|fishfinger|100|7|fstab|093f510ec5c0f512.h /usr/local ffs rw,wxallowed,nod <a class='textlink' href='./2021-04-22-dtail-the-distributed-log-tail-program.html'>2021-04-22 DTail - The distributed log tail program</a><br /> <a class='textlink' href='./2022-03-06-the-release-of-dtail-4.0.0.html'>2022-03-06 The release of DTail 4.0.0</a><br /> <a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD (You are currently reading this)</a><br /> +<a class='textlink' href='./2023-09-25-dtail-usage-examples.html'>2023-09-25 DTail usage examples</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -2931,7 +3252,7 @@ jgs (________\ \ <br /> <span>I wouldn't say I like checking social media, as it can consume a lot of time and can become addictive. But once in a while, I want to catch up with my "networks". After a bad night's sleep, it's the perfect time to check your social media. Once done, you don't have to do it anymore for the next couple of days!</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -3050,7 +3371,7 @@ http://www.gnu.org/software/src-highlite --> <a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again²</a><br /> <a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -3728,7 +4049,7 @@ rex commons <br /> <span>Why re-inventing the wheel? I love that a <span class='inlinecode'>Rexfile</span> is just a Perl DSL. Also, OpenBSD comes with Perl in the base system. So no new programming language had to be added to my mix for the configuration management system. Also, the <span class='inlinecode'>acme.sh</span> shell script is not a Bash but a standard Bourne shell script, so I didn't have to install an additional shell as OpenBSD does not come with the Bash pre-installed.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -4072,7 +4393,7 @@ v = 008 [v = p*c*(s != c ? 2 : 1)] Total logical CPUs <br /> <a class='textlink' href='./2021-04-24-welcome-to-the-geminispace.html'>Welcome to the Geminispae</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -4238,7 +4559,7 @@ v = 008 [v = p*c*(s != c ? 2 : 1)] Total logical CPUs <a class='textlink' href='./2022-05-27-perl-is-still-a-great-choice.html'>2022-05-27 Perl is still a great choice (You are currently reading this)</a><br /> <a class='textlink' href='./2023-05-01-unveiling-guprecords:-uptime-records-with-raku.html'>2023-05-01 Unveiling <span class='inlinecode'>guprecords.raku</span>: Global Uptime Records with Raku</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -4397,7 +4718,7 @@ learn () { <li>The Off Switch; Mark Cropley; Virgin Books</li> <li>Ultralearning; Scott Young; Thorsons</li> </ul><br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -4712,12 +5033,13 @@ exec /usr/local/bin/dtailhealth --server localhost:2222 <a class='textlink' href='./2021-04-22-dtail-the-distributed-log-tail-program.html'>2021-04-22 DTail - The distributed log tail program</a><br /> <a class='textlink' href='./2022-03-06-the-release-of-dtail-4.0.0.html'>2022-03-06 The release of DTail 4.0.0 (You are currently reading this)</a><br /> <a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD</a><br /> +<a class='textlink' href='./2023-09-25-dtail-usage-examples.html'>2023-09-25 DTail usage examples</a><br /> <br /> <span>Thanks!</span><br /> <br /> <span>Paul</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -4964,7 +5286,7 @@ GNU/kFreeBSD rhea.buetow.org 8.0-RELEASE-p5 FreeBSD 8.0-RELEASE-p5 #2: Sat Nov 2 <a class='textlink' href='https://www.dragonflybsd.org/'>DragonFly BSD - Fork of FreeBSD 4</a><br /> <a class='textlink' href='http://wiki.postmarketos.org/wiki/Phosh'>Phosh (on postmarketOS) - A true Linux shell for the smartphone</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -5029,7 +5351,7 @@ GNU/kFreeBSD rhea.buetow.org 8.0-RELEASE-p5 FreeBSD 8.0-RELEASE-p5 #2: Sat Nov 2 <br /> <span>The host buetow.org will stay. However, not as the primary address for this site. I will keep using it for my personal internet infrastructure as well as for most of my E-Mail addresses. I used buetow.org for that over the past 10 years already anyway and that won't change any time soon. I don't know what I am going to do with snonux.de in the long run. A .de SLD (for Germany) is pretty cheap, so I might just keep it for now. </span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -5537,7 +5859,7 @@ PAUL:X:1000:1000:PAUL BUETOW:/HOME/PAUL:/BIN/BASH <a class='textlink' href='./2021-11-29-bash-golf-part-1.html'>2021-11-29 Bash Golf Part 1</a><br /> <a class='textlink' href='./2022-01-01-bash-golf-part-2.html'>2022-01-01 Bash Golf Part 2 (You are currently reading this)</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -5681,7 +6003,7 @@ PAUL:X:1000:1000:PAUL BUETOW:/HOME/PAUL:/BIN/BASH <br /> <a class='textlink' href='https://unixsheikh.com/articles/how-to-stay-sane-in-todays-world-of-tech.html'>https://unixsheikh.com/articles/how-to-stay-sane-in-todays-world-of-tech.html</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -6171,7 +6493,7 @@ bash: line 1: 1/10.0 : syntax error: invalid arithmetic operator (error token is <a class='textlink' href='./2021-11-29-bash-golf-part-1.html'>2021-11-29 Bash Golf Part 1 (You are currently reading this)</a><br /> <a class='textlink' href='./2022-01-01-bash-golf-part-2.html'>2022-01-01 Bash Golf Part 2</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -6292,7 +6614,7 @@ bash: line 1: 1/10.0 : syntax error: invalid arithmetic operator (error token is <br /> <span>This usually means creating one or more tickets, which will be dealt with soon. Once the permanent fix is deployed, you can remove your ad-hoc automation and monitoring around it and focus on your regular work again.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -6406,7 +6728,7 @@ bash: line 1: 1/10.0 : syntax error: invalid arithmetic operator (error token is <br /> <span>Enough ranted for now!</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <span class='quote'>Controversially, a lack of features is a feature. Enjoy your peace an quiet. - Michael W Lucas </span><br /> <br /> @@ -6486,7 +6808,7 @@ bash: line 1: 1/10.0 : syntax error: invalid arithmetic operator (error token is <br /> <span>Do you need Microsoft Word? Why don't you just use the Vim text editor or GNU Emacs to write your letters? If that's too nerdy, you can still use open-source alternatives such as AbiWord or LibreOffice. Larger organizations have the tendency to standardize the software their employees have to use. Unfortunately, as Microsoft Word is the de-facto standard text processing program, most companies prefer Word over LibreOffice. Same with Microsoft Excel vs LibreOffice Calc or other spreadsheet alternatives like Gnumeric. I don't know why that is; please....</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <span>I only use free and open-source operating systems on my personal Laptops, Desktop PCs and servers (FreeBSD and Linux based ones). Most of the programs and apps I use on them are free and open-source as well, and I am comfortable with it for over twenty years. Exceptions are the BIOSes and some firmwares of my devices. I also use Skype as most of my friends and family are using it. They are, unfortunately, proprietary software still. But I will be looking into Matrix as a Skype alternative when I have time. There are also open BIOS alternatives, but they usually don't work on my devices.</span><br /> <br /> @@ -6543,7 +6865,7 @@ bash: line 1: 1/10.0 : syntax error: invalid arithmetic operator (error token is <br /> <span>You have better chances when you know how to manage your own server and install and manage alternatives to the big cloud providers by yourself. I have the advantage that I have work experience as a Linux Systems Administrator here. I mentioned NextCloud already. I use NextCloud for online photo and file storage, contact and calendar sync and as an RSS news feed server. You could do the same with your own E-Mail server, you can also host your own website and blog. I also mentioned Matrix as a Skype alternative (which could also be an alternative to WhatsApp, Skype, Telegram, Viber, ...). I don't know a lot about Matrix yet, but it seems to be a very neat alternative. I am ready to invest time in it as one of my future personal pet projects. Not only because I think it's better, but also because for fun and as a hobby. But this doesn't mean that I invest *all* of my personal free time in it.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -6666,7 +6988,7 @@ Hello World <br /> <span>Will I abandon my beloved Perl? Probably not. There are also some Perl scripts I use at work. But unfortunately I only have a limited amount of time and I have to use it wisely. I might look into Raku (formerly known as Perl 6) next year and use it for a personal pet project, who knows. :-). I also highly recommend reading the two Perl books "Modern Perl" and "Higher-Order Perl".</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -6871,7 +7193,7 @@ assert<font color="#990000">::</font>equals <font color="#FF0000">"$(generate::m <a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again²</a><br /> <a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -7278,7 +7600,7 @@ fi <a class='textlink' href='./2021-11-29-bash-golf-part-1.html'>2021-11-29 Bash Golf Part 1</a><br /> <a class='textlink' href='./2022-01-01-bash-golf-part-2.html'>2022-01-01 Bash Golf Part 2</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -7384,7 +7706,7 @@ fi <a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again²</a><br /> <a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -7515,8 +7837,9 @@ dtail –servers serverlist.txt –files ‘/var/log/*.log’ –regex ‘(?i:er <a class='textlink' href='./2021-04-22-dtail-the-distributed-log-tail-program.html'>2021-04-22 DTail - The distributed log tail program (You are currently reading this)</a><br /> <a class='textlink' href='./2022-03-06-the-release-of-dtail-4.0.0.html'>2022-03-06 The release of DTail 4.0.0</a><br /> <a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD</a><br /> +<a class='textlink' href='./2023-09-25-dtail-usage-examples.html'>2023-09-25 DTail usage examples</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -7722,7 +8045,7 @@ Total time: 1213.00s <a class='textlink' href='https://graphiteapp.org'>Graphite</a><br /> <a class='textlink' href='https://en.wikipedia.org/wiki/Memory-mapped_I/O'>Memory mapped I/O</a><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -7855,7 +8178,7 @@ http://www.gnu.org/software/src-highlite --> <br /> <span>C is a very old programming language with it's quirks. This might be one of the reasons why Linux will also let Rust code in.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -8110,7 +8433,7 @@ apply Service "dig6" { </ul><br /> <span>That's much more comfortable now than manually clicking at some web UIs at Schlund Technologies.</span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> @@ -8156,414 +8479,7 @@ apply Service "dig6" { <br /> <span>Furthermore, I added scrubbing ("zpool scrub...") to the script. It ensures that the file system is consistent and that there are no bad blocks on the disk and the file system. To increase the reliability, I also run a "zfs set copies=2 zroot". That setting is also synchronized to the offsite ZFS pool. ZFS stores every data block to disk twice now. Yes, it consumes twice as much disk space, making it better fault-tolerant against hardware errors (e.g. only individual disk sectors going bad). </span><br /> <br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>Jails and ZFS with Puppet on FreeBSD</title> - <link href="gemini://foo.zone/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi" /> - <id>gemini://foo.zone/gemfeed/2016-04-09-jails-and-zfs-on-freebsd-with-puppet.gmi</id> - <updated>2016-04-09T18:29:47+01:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>Over the last couple of years I wrote quite a few Puppet modules in order to manage my personal server infrastructure. One of them manages FreeBSD Jails and another one ZFS file systems. I thought I would give a brief overview in how it looks and feels.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Jails and ZFS with Puppet on FreeBSD</h1><br /> -<br /> -<span class='quote'>Published at 2016-04-09T18:29:47+01:00</span><br /> -<br /> -<pre> - __ __ - (( \---/ )) - )__ __( - / ()___() \ - \ /(_)\ / - \ \_|_/ / - _______> <_______ - //\ |>o<| /\\ - \\/___ ___\// - | | - | | - | | - | | - `--....---' - \ \ - \ `. hjw - \ `. -</pre> -<br /> -<span>Over the last couple of years I wrote quite a few Puppet modules in order to manage my personal server infrastructure. One of them manages FreeBSD Jails and another one ZFS file systems. I thought I would give a brief overview in how it looks and feels.</span><br /> -<br /> -<h2 style='display: inline'>ZFS</h2><br /> -<br /> -<span>The ZFS module is a pretty basic one. It does not manage ZFS pools yet as I am not creating them often enough which would justify implementing an automation. But let's see how we can create a ZFS file system (on an already given ZFS pool named ztank):</span><br /> -<br /> -<span>Puppet snippet:</span><br /> -<br /> -<pre> -zfs::create { 'ztank/foo': - ensure => present, - filesystem => '/srv/foo', - - require => File['/srv'], -} -</pre> -<br /> -<span>Puppet run:</span><br /> -<br /> -<pre> -admin alphacentauri:/opt/git/server/puppet/manifests [1212]% puppet.apply -Password: -Info: Loading facts -Info: Loading facts -Info: Loading facts -Info: Loading facts -Notice: Compiled catalog for alphacentauri.home in environment production in 7.14 seconds -Info: Applying configuration version '1460189837' -Info: mount[files]: allowing * access -Info: mount[restricted]: allowing * access -Notice: /Stage[main]/Main/Node[alphacentauri]/Zfs::Create[ztank/foo]/Exec[ztank/foo_create]/returns: executed successfully -Notice: Finished catalog run in 25.41 seconds -admin alphacentauri:~ [1213]% zfs list | grep foo -ztank/foo 96K 1.13T 96K /srv/foo -admin alphacentauri:~ [1214]% df | grep foo -ztank/foo 1214493520 96 1214493424 0% /srv/foo -admin alphacentauri:~ [1215]% -</pre> -<br /> -<span>The destruction of the file system just requires to set "ensure" to "absent" in Puppet:</span><br /> -<br /> -<pre> -zfs::create { 'ztank/foo': - ensure => absent, - filesystem => '/srv/foo', - - require => File['/srv'], -}¬ -</pre> -<br /> -<span>Puppet run:</span><br /> -<br /> -<pre> -admin alphacentauri:/opt/git/server/puppet/manifests [1220]% puppet.apply -Password: -Info: Loading facts -Info: Loading facts -Info: Loading facts -Info: Loading facts -Notice: Compiled catalog for alphacentauri.home in environment production in 6.14 seconds -Info: Applying configuration version '1460190203' -Info: mount[files]: allowing * access -Info: mount[restricted]: allowing * access -Notice: /Stage[main]/Main/Node[alphacentauri]/Zfs::Create[ztank/foo]/Exec[zfs destroy -r ztank/foo]/returns: executed successfully -Notice: Finished catalog run in 22.72 seconds -admin alphacentauri:/opt/git/server/puppet/manifests [1221]% zfs list | grep foo -zsh: done zfs list | -zsh: exit 1 grep foo -admin alphacentauri:/opt/git/server/puppet/manifests [1222:1]% df | grep foo -zsh: done df | -zsh: exit 1 grep foo -</pre> -<br /> -<h2 style='display: inline'>Jails</h2><br /> -<br /> -<span>Here is an example in how a FreeBSD Jail can be created. The Jail will have its own public IPv6 address. And it will have its own internal IPv4 address with IPv4 NAT to the internet (this is due to the limitation that the host server only got one public IPv4 address which requires sharing between all the Jails).</span><br /> -<br /> -<span>Furthermore, Puppet will ensure that the Jail will have its own ZFS file system (internally it is using the ZFS module). Please notice that the NAT requires the packet filter to be setup correctly (not covered in this blog post).</span><br /> -<br /> -<pre> -include jail::freebsd - -# Cloned interface for Jail IPv4 NAT -freebsd::rc_config { 'cloned_interfaces': - value => 'lo1', -} -freebsd::rc_config { 'ipv4_addrs_lo1': - value => '192.168.0.1-24/24' -} - -freebsd::ipalias { '2a01:4f8:120:30e8::17': - ensure => up, - proto => 'inet6', - preflen => '64', - interface => 're0', - aliasnum => '8', -} - -class { 'jail': - ensure => present, - jails_config => { - sync => { - '_ensure' => present, - '_type' => 'freebsd', - '_mirror' => 'ftp://ftp.de.freebsd.org', - '_remote_path' => 'FreeBSD/releases/amd64/10.1-RELEASE', - '_dists' => [ 'base.txz', 'doc.txz', ], - '_ensure_directories' => [ '/opt', '/opt/enc' ], - '_ensure_zfs' => [ '/sync' ], - 'host.hostname' => "'sync.ian.buetow.org'", - 'ip4.addr' => '192.168.0.17', - 'ip6.addr' => '2a01:4f8:120:30e8::17', - }, - } -} -</pre> -<br /> -<span>This is how the result looks like:</span><br /> -<br /> -<pre> -admin sun:/etc [1939]% puppet.apply -Info: Loading facts -Info: Loading facts -Info: Loading facts -Info: Loading facts -Notice: Compiled catalog for sun.ian.buetow.org in environment production in 1.80 seconds -Info: Applying configuration version '1460190986' -Notice: /Stage[main]/Jail/File[/etc/jail.conf]/ensure: created -Info: mount[files]: allowing * access -Info: mount[restricted]: allowing * access -Info: Computing checksum on file /etc/motd -Info: /Stage[main]/Motd/File[/etc/motd]: Filebucketed /etc/motd to puppet with sum fced1b6e89f50ef2c40b0d7fba9defe8 -Notice: /Stage[main]/Jail/Jail::Create[sync]/File[/jail/sync]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/Zfs::Create[zroot/jail/sync]/Exec[zroot/jail/sync_create]/returns: executed successfully -Notice: /Stage[main]/Jail/Jail::Create[sync]/File[/jail/sync/opt]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/File[/jail/sync/opt/enc]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/Jail::Ensure_zfs[/sync]/Zfs::Create[zroot/jail/sync/sync]/Exec[zroot/jail/sync/sync_create]/returns: executed successfully -Notice: /Stage[main]/Jail/Jail::Create[sync]/Jail::Freebsd::Create[sync]/File[/jail/sync/.jailbootstrap]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/Jail::Freebsd::Create[sync]/File[/etc/fstab.jail.sync]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/Jail::Freebsd::Create[sync]/File[/jail/sync/.jailbootstrap/bootstrap.sh]/ensure: created -Notice: /Stage[main]/Jail/Jail::Create[sync]/Jail::Freebsd::Create[sync]/Exec[sync_bootstrap]/returns: executed successfully -Notice: Finished catalog run in 49.72 seconds -admin sun:/etc [1942]% ls -l /jail/sync -total 154 --r--r--r-- 1 root wheel 6198 11 Nov 2014 COPYRIGHT -drwxr-xr-x 2 root wheel 47 11 Nov 2014 bin -drwxr-xr-x 7 root wheel 43 11 Nov 2014 boot -dr-xr-xr-x 2 root wheel 2 11 Nov 2014 dev -drwxr-xr-x 23 root wheel 101 9 Apr 10:37 etc -drwxr-xr-x 3 root wheel 50 11 Nov 2014 lib -drwxr-xr-x 3 root wheel 4 11 Nov 2014 libexec -drwxr-xr-x 2 root wheel 2 11 Nov 2014 media -drwxr-xr-x 2 root wheel 2 11 Nov 2014 mnt -drwxr-xr-x 3 root wheel 3 9 Apr 10:36 opt -dr-xr-xr-x 2 root wheel 2 11 Nov 2014 proc -drwxr-xr-x 2 root wheel 143 11 Nov 2014 rescue -drwxr-xr-x 2 root wheel 6 11 Nov 2014 root -drwxr-xr-x 2 root wheel 132 11 Nov 2014 sbin -drwxr-xr-x 2 root wheel 2 9 Apr 10:36 sync -lrwxr-xr-x 1 root wheel 11 11 Nov 2014 sys -> usr/src/sys -drwxrwxrwt 2 root wheel 2 11 Nov 2014 tmp -drwxr-xr-x 14 root wheel 14 11 Nov 2014 usr -drwxr-xr-x 24 root wheel 24 11 Nov 2014 var -admin sun:/etc [1943]% zfs list | grep sync;df | grep sync -zroot/jail/sync 162M 343G 162M /jail/sync -zroot/jail/sync/sync 144K 343G 144K /jail/sync/sync -/opt/enc 5061624 84248 4572448 2% /jail/sync/opt/enc -zroot/jail/sync 360214972 166372 360048600 0% /jail/sync -zroot/jail/sync/sync 360048744 144 360048600 0% /jail/sync/sync -admin sun:/etc [1944]% cat /etc/fstab.jail.sync -# Generated by Puppet for a Jail. -# Can contain file systems to be mounted curing jail start. -admin sun:/etc [1945]% cat /etc/jail.conf -# Generated by Puppet - -allow.chflags = true; -exec.start = '/bin/sh /etc/rc'; -exec.stop = '/bin/sh /etc/rc.shutdown'; -mount.devfs = true; -mount.fstab = "/etc/fstab.jail.$name"; -path = "/jail/$name"; - -sync { - host.hostname = 'sync.ian.buetow.org'; - ip4.addr = 192.168.0.17; - ip6.addr = 2a01:4f8:120:30e8::17; -} -admin sun:/etc [1955]% sudo service jail start sync -Password: -Starting jails: sync. -admin sun:/etc [1956]% jls | grep sync - 103 192.168.0.17 sync.ian.buetow.org /jail/sync -admin sun:/etc [1957]% sudo jexec 103 /bin/csh -root@sync:/ # ifconfig -a -re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 - options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE> - ether 50:46:5d:9f:fd:1e - inet6 2a01:4f8:120:30e8::17 prefixlen 64 - nd6 options=8021<PERFORMNUD,AUTO_LINKLOCAL,DEFAULTIF> - media: Ethernet autoselect (1000baseT <full-duplex>) - status: active -lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 - options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> - nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> - lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 - options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> - inet 192.168.0.17 netmask 0xffffffff - nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> -</pre> -<br /> -<h2 style='display: inline'>Inside-Jail Puppet</h2><br /> -<br /> -<span>To automatically setup the applications running in the Jail I am using Puppet as well. I wrote a few scripts which bootstrap Puppet inside of a newly created Jail. It is doing the following:</span><br /> -<br /> -<ul> -<li>Mounts an encrypted container (containing a secret Puppet manifests [git repository])</li> -<li>Activates "pkg-ng", the FreeBSD binary package manager, in the Jail</li> -<li>Installs Puppet plus all dependencies in the Jail</li> -<li>Updates the Jail via "freebsd-update" to the latest version</li> -<li>Restarts the Jail and invokes Puppet.</li> -<li>Puppet then also schedules a periodic cron job for the next Puppet runs.</li> -</ul><br /> -<pre> -admin sun:~ [1951]% sudo /opt/snonux/local/etc/init.d/enc activate sync -Starting jails: dns. -The package management tool is not yet installed on your system. -Do you want to fetch and install it now? [y/N]: y -Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/freebsd:10:x86:64/latest, please wait... -Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done -[sync.ian.buetow.org] Installing pkg-1.7.2... -[sync.ian.buetow.org] Extracting pkg-1.7.2: 100% -Updating FreeBSD repository catalogue... -[sync.ian.buetow.org] Fetching meta.txz: 100% 944 B 0.9kB/s 00:01 -[sync.ian.buetow.org] Fetching packagesite.txz: 100% 5 MiB 5.6MB/s 00:01 -Processing entries: 100% -FreeBSD repository update completed. 25091 packages processed. -Updating database digests format: 100% -The following 20 package(s) will be affected (of 0 checked): - - New packages to be INSTALLED: - git: 2.7.4_1 - expat: 2.1.0_3 - python27: 2.7.11_1 - libffi: 3.2.1 - indexinfo: 0.2.4 - gettext-runtime: 0.19.7 - p5-Error: 0.17024 - perl5: 5.20.3_9 - cvsps: 2.1_1 - p5-Authen-SASL: 2.16_1 - p5-Digest-HMAC: 1.03_1 - p5-GSSAPI: 0.28_1 - curl: 7.48.0_1 - ca_root_nss: 3.22.2 - p5-Net-SMTP-SSL: 1.03 - p5-IO-Socket-SSL: 2.024 - p5-Net-SSLeay: 1.72 - p5-IO-Socket-IP: 0.37 - p5-Socket: 2.021 - p5-Mozilla-CA: 20160104 - - The process will require 144 MiB more space. - 30 MiB to be downloaded. -[sync.ian.buetow.org] Fetching git-2.7.4_1.txz: 100% 4 MiB 3.7MB/s 00:01 -[sync.ian.buetow.org] Fetching expat-2.1.0_3.txz: 100% 98 KiB 100.2kB/s 00:01 -[sync.ian.buetow.org] Fetching python27-2.7.11_1.txz: 100% 10 MiB 10.7MB/s 00:01 -[sync.ian.buetow.org] Fetching libffi-3.2.1.txz: 100% 35 KiB 36.2kB/s 00:01 -[sync.ian.buetow.org] Fetching indexinfo-0.2.4.txz: 100% 5 KiB 5.0kB/s 00:01 -[sync.ian.buetow.org] Fetching gettext-runtime-0.19.7.txz: 100% 148 KiB 151.1kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Error-0.17024.txz: 100% 24 KiB 24.8kB/s 00:01 -[sync.ian.buetow.org] Fetching perl5-5.20.3_9.txz: 100% 13 MiB 6.9MB/s 00:02 -[sync.ian.buetow.org] Fetching cvsps-2.1_1.txz: 100% 41 KiB 42.1kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Authen-SASL-2.16_1.txz: 100% 44 KiB 45.1kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Digest-HMAC-1.03_1.txz: 100% 9 KiB 9.5kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-GSSAPI-0.28_1.txz: 100% 41 KiB 41.7kB/s 00:01 -[sync.ian.buetow.org] Fetching curl-7.48.0_1.txz: 100% 2 MiB 2.2MB/s 00:01 -[sync.ian.buetow.org] Fetching ca_root_nss-3.22.2.txz: 100% 324 KiB 331.4kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Net-SMTP-SSL-1.03.txz: 100% 11 KiB 10.8kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-IO-Socket-SSL-2.024.txz: 100% 153 KiB 156.4kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Net-SSLeay-1.72.txz: 100% 234 KiB 239.3kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-IO-Socket-IP-0.37.txz: 100% 27 KiB 27.4kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Socket-2.021.txz: 100% 37 KiB 38.0kB/s 00:01 -[sync.ian.buetow.org] Fetching p5-Mozilla-CA-20160104.txz: 100% 147 KiB 150.8kB/s 00:01 -Checking integrity... -[sync.ian.buetow.org] [1/12] Installing libyaml-0.1.6_2... -[sync.ian.buetow.org] [1/12] Extracting libyaml-0.1.6_2: 100% -[sync.ian.buetow.org] [2/12] Installing libedit-3.1.20150325_2... -[sync.ian.buetow.org] [2/12] Extracting libedit-3.1.20150325_2: 100% -[sync.ian.buetow.org] [3/12] Installing ruby-2.2.4,1... -[sync.ian.buetow.org] [3/12] Extracting ruby-2.2.4,1: 100% -[sync.ian.buetow.org] [4/12] Installing ruby22-gems-2.6.2... -[sync.ian.buetow.org] [4/12] Extracting ruby22-gems-2.6.2: 100% -[sync.ian.buetow.org] [5/12] Installing libxml2-2.9.3... -[sync.ian.buetow.org] [5/12] Extracting libxml2-2.9.3: 100% -[sync.ian.buetow.org] [6/12] Installing dmidecode-3.0... -[sync.ian.buetow.org] [6/12] Extracting dmidecode-3.0: 100% -[sync.ian.buetow.org] [7/12] Installing rubygem-json_pure-1.8.3... -[sync.ian.buetow.org] [7/12] Extracting rubygem-json_pure-1.8.3: 100% -[sync.ian.buetow.org] [8/12] Installing augeas-1.4.0... -[sync.ian.buetow.org] [8/12] Extracting augeas-1.4.0: 100% -[sync.ian.buetow.org] [9/12] Installing rubygem-facter-2.4.4... -[sync.ian.buetow.org] [9/12] Extracting rubygem-facter-2.4.4: 100% -[sync.ian.buetow.org] [10/12] Installing rubygem-hiera1-1.3.4_1... -[sync.ian.buetow.org] [10/12] Extracting rubygem-hiera1-1.3.4_1: 100% -[sync.ian.buetow.org] [11/12] Installing rubygem-ruby-augeas-0.5.0_2... -[sync.ian.buetow.org] [11/12] Extracting rubygem-ruby-augeas-0.5.0_2: 100% -[sync.ian.buetow.org] [12/12] Installing puppet38-3.8.4_1... -===> Creating users and/or groups. -Creating group 'puppet' with gid '814'. -Creating user 'puppet' with uid '814'. -[sync.ian.buetow.org] [12/12] Extracting puppet38-3.8.4_1: 100% -. -. -. -. -. -Looking up update.FreeBSD.org mirrors... 4 mirrors found. -Fetching public key from update4.freebsd.org... done. -Fetching metadata signature for 10.1-RELEASE from update4.freebsd.org... done. -Fetching metadata index... done. -Fetching 2 metadata files... done. -Inspecting system... done. -Preparing to download files... done. -Fetching 874 patches.....10....20....30.... -. -. -. -Applying patches... done. -Fetching 1594 files... -Installing updates... -done. -Info: Loading facts -Info: Loading facts -Info: Loading facts -Info: Loading facts -Could not retrieve fact='pkgng_version', resolution='<anonymous>': undefined method `pkgng_enabled' for Facter:Module -Warning: Config file /usr/local/etc/puppet/hiera.yaml not found, using Hiera defaults -Notice: Compiled catalog for sync.ian.buetow.org in environment production in 1.31 seconds -Warning: Found multiple default providers for package: pkgng, gem, pip; using pkgng -Info: Applying configuration version '1460192563' -Notice: /Stage[main]/S_base_freebsd/User[root]/shell: shell changed '/bin/csh' to '/bin/tcsh' -Notice: /Stage[main]/S_user::Root_files/S_user::All_files[root_user]/File[/root/user]/ensure: created -Notice: /Stage[main]/S_user::Root_files/S_user::My_files[root]/File[/root/userfiles]/ensure: created -Notice: /Stage[main]/S_user::Root_files/S_user::My_files[root]/File[/root/.task]/ensure: created -. -. -. -. -Notice: Finished catalog run in 206.09 seconds -</pre> -<br /> -<h2 style='display: inline'>Managing multiple Jails</h2><br /> -<br /> -<span>Of course I am operating multiple Jails on the same host this way with Puppet:</span><br /> -<br /> -<ul> -<li>A Jail for the MTA</li> -<li>A Jail for the Webserver</li> -<li>A Jail for BIND DNS server</li> -<li>A Jail for syncing data forth and back between various servers</li> -<li>A Jail for other personal (experimental) use</li> -<li>...etc</li> -</ul><br /> -<span>All done in a pretty automated manor. </span><br /> -<br /> -<span>E-Mail your comments to paul at buetow.org :-)</span><br /> +<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> <br /> <a class='textlink' href='../'>Back to the main site</a><br /> </div> diff --git a/gemfeed/atom.xml.tmp b/gemfeed/atom.xml.tmp deleted file mode 100644 index 59e0ee54..00000000 --- a/gemfeed/atom.xml.tmp +++ /dev/null @@ -1,2825 +0,0 @@ -<?xml version="1.0" encoding="utf-8"?> -<feed xmlns="http://www.w3.org/2005/Atom"> - <updated>2023-08-27T13:59:36+03:00</updated> - <title>foo.zone feed</title> - <subtitle>To be in the .zone!</subtitle> - <link href="gemini://foo.zone/gemfeed/atom.xml" rel="self" /> - <link href="gemini://foo.zone/" /> - <id>gemini://foo.zone/</id> - <entry> - <title>Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect</title> - <link href="gemini://foo.zone/gemfeed/2023-08-20-site-reliability-engineering-part-3.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-08-20-site-reliability-engineering-part-3.gmi</id> - <updated>2023-08-20T12:17:56+03:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>This is the third part of my Site Reliability Engineering (SRE) series. I am currently employed as a Principal Site Reliability Engineer and will try to share what SRE is about in this blog series.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect</h1><br /> -<br /> -<span class='quote'>Published at 2023-08-20T12:17:56+03:00</span><br /> -<br /> -<span>This is the third part of my Site Reliability Engineering (SRE) series. I am currently employed as a Principal Site Reliability Engineer and will try to share what SRE is about in this blog series.</span><br /> -<br /> -<a class='textlink' href='./2023-08-18-site-reliability-engineering-part-1.html'>2023-08-18 Site Reliability Engineering - Part 1: SRE and Organizational Culture</a><br /> -<a class='textlink' href='./2023-08-19-site-reliability-engineering-part-2.html'>2023-08-19 Site Reliability Engineering - Part 2: Operational Balance in SRE</a><br /> -<a class='textlink' href='./2023-08-20-site-reliability-engineering-part-3.html'>2023-08-20 Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect (You are currently reading this)</a><br /> -<br /> -<pre> - ..--""""----.. - .-" ..--""""--.j-. - .-" .-" .--.""--.. - .-" .-" ..--"-. \/ ; - .-" .-"_.--..--"" ..--' "-. : - .' .' / `. \..--"" __ _ \ ; - :.__.-" \ / .' ( )"-. Y - ; ;: ( ) ( ). \ - .': /:: : \ \ - .'.-"\._ _.-" ; ; ( ) .-. ( ) \ - " `.""" .j" : : \ ; ; \ - bug /"""""/ ; ( ) "" :.( ) \ - /\ / : \ \`.: _ \ - : `. / ; `( ) (\/ :" \ \ - \ `. : "-.(_)_.' t-' ; - \ `. ; ..--": - `. `. : ..--"" : - `. "-. ; ..--"" ; - `. "-.:_..--"" ..--" - `. : ..--"" - "-. : ..--"" - "-.;_..--"" - -</pre> -<br /> -<h2 style='display: inline'>On-Call Culture and the Human Aspect: Prioritising Well-being in the Realm of Reliability</h2><br /> -<br /> -<span>Site Reliability Engineering is synonymous with ensuring system reliability, but the human factor is an often-underestimated part of this discipline. Ensuring an healthy on-call culture is as critical as any technical solution. The well-being of the engineers is an important factor.</span><br /> -<br /> -<span>Firstly, a healthy on-call rotation is about more than just managing and responding to incidents. It's about the entire ecosystem that supports this practice. This involves reducing pain points, offering mentorship, rapid iteration, and ensuring that engineers have the right tools and processes. One ceavat is, that engineers should be willing to learn. Especially in on-call rotation embedding SREs with other engineers (for example Software Engineers or QA Engineers), it's difficult to motivate everyone to engage. QA Engineers want to test the software, Software Engineers want to implement new features; they don't want to troubleshoot and debug production incidents. It can be depressing for the mentoring SRE.</span><br /> -<br /> -<span>Furthermore, the metrics that measure the success of an on-call experience are only sometimes straightforward. While one might assume that fewer pages translate to better on-call expertise (which is true to a degree, as who wants to receive a page out of office hours?), it's not always the volume of pages that matters most. Trust, ownership, accountability, and effective communication play the important roles.</span><br /> -<br /> -<span>An important part is giving feedback about the on-call experience to ensure continuous learning. If alerts are mostly noise, they should be tuned or even eliminated. If alerts are actionable, can recurring tasks be automated? If there are knowledge gaps, is the documentation not good enough? Continuous retrospection ensures that not only do systems evolve, but the experience for the on-call engineers becomes progressively better.</span><br /> -<br /> -<span>Onboarding for on-call duties is a crucial aspect of ensuring the reliability and efficiency of systems. This process involves equipping new team members with the knowledge, tools, and support to handle incidents confidently. It begins with an overview of the system architecture and common challenges, followed by training on monitoring tools, alerting mechanisms, and incident response protocols. Shadowing experienced on-call engineers can offer practical exposure. Too often, new engineers are thrown into the cold water without proper onboarding and training because the more experienced engineers are too busy fire-fighting production issues in the first place.</span><br /> -<br /> -<span>An always-on, always-alert culture can lead to burnout. Engineers should be encouraged to recognise their limits, take breaks, and seek support when needed. This isn't just about individual health; a burnt-out engineer can have cascading effects on the entire team and the systems they manage. A successful on-call culture ensures that while systems are kept running, the engineers are kept happy, healthy, and supported. The more experienced engineers should take time to mentor the junior engineers, but the junior engineers should also be fully engaged, try to investigate and learn new things by themselves.</span><br /> -<br /> -<span>For the junior engineer, it's too easy to fall back and ask the experts in the team every time an issue arises. This seems reasonable, but serving recipes for solving production issues on a silver tablet won't scale forever, as there are infinite scenarios of how production systems can break. So every engineer should learn to debug, troubleshoot and resolve production incidents independently. The experts will still be there for guidance and step in when the junior gets stuck after trying, but the experts should also learn to step down so that lesser experienced engineers can step up and learn. But mistakes can always happen here; that's why having a blameless on-call culture is essential.</span><br /> -<br /> -<span>A blameless on-call culture is a must for a safe and collaborative environment where engineers can effectively respond to incidents without fear of retribution. This approach acknowledges that mistakes are a natural part of the learning and innovation process. When individuals are assured they won't be punished for errors, they're more likely to openly discuss mistakes, allowing the entire team to learn and grow from each incident. Furthermore, a blameless culture promotes psychological safety, enhances job satisfaction, reduces burnout, and ensures that talent remains committed and engaged.</span><br /> -<br /> -<span>The fourth part of this blog series will be published soon :-)</span><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>Site Reliability Engineering - Part 2: Operational Balance in SRE</title> - <link href="gemini://foo.zone/gemfeed/2023-08-19-site-reliability-engineering-part-2.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-08-19-site-reliability-engineering-part-2.gmi</id> - <updated>2023-08-19T00:18:18+03:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>This is the second part of my Site Reliability Engineering (SRE) series. I am currently employed as a Principal Site Reliability Engineer and will try to share what SRE is about in this blog series.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Site Reliability Engineering - Part 2: Operational Balance in SRE</h1><br /> -<br /> -<span class='quote'>Published at 2023-08-19T00:18:18+03:00</span><br /> -<br /> -<span>This is the second part of my Site Reliability Engineering (SRE) series. I am currently employed as a Principal Site Reliability Engineer and will try to share what SRE is about in this blog series.</span><br /> -<br /> -<a class='textlink' href='./2023-08-18-site-reliability-engineering-part-1.html'>2023-08-18 Site Reliability Engineering - Part 1: SRE and Organizational Culture</a><br /> -<a class='textlink' href='./2023-08-19-site-reliability-engineering-part-2.html'>2023-08-19 Site Reliability Engineering - Part 2: Operational Balance in SRE (You are currently reading this)</a><br /> -<a class='textlink' href='./2023-08-20-site-reliability-engineering-part-3.html'>2023-08-20 Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect</a><br /> -<br /> -<pre> -⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣠⣾⣷⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ -⠀⠀⠀⠀⣾⠿⠿⠿⠶⠾⠿⠿⣿⣿⣿⣿⣿⣿⠿⠿⠶⠶⠿⠿⠿⣷⠀⠀⠀⠀ -⠀⠀⠀⣸⢿⣆⠀⠀⠀⠀⠀⠀⠀⠙⢿⡿⠉⠀⠀⠀⠀⠀⠀⠀⣸⣿⡆⠀⠀⠀ -⠀⠀⢠⡟⠀⢻⣆⠀⠀⠀⠀⠀⠀⠀⣾⣧⠀⠀⠀⠀⠀⠀⠀⣰⡟⠀⢻⡄⠀⠀ -⠀⢀⣾⠃⠀⠀⢿⡄⠀⠀⠀⠀⠀⢠⣿⣿⡀⠀⠀⠀⠀⠀⢠⡿⠀⠀⠘⣷⡀⠀ -⠀⣼⣏⣀⣀⣀⣈⣿⡀⠀⠀⠀⠀⣸⣿⣿⡇⠀⠀⠀⠀⢀⣿⣃⣀⣀⣀⣸⣧⠀ -⠀⢻⣿⣿⣿⣿⣿⣿⠃⠀⠀⠀⠀⣿⣿⣿⣿⠀⠀⠀⠀⠈⢿⣿⣿⣿⣿⣿⡿⠀ -⠀⠀⠉⠛⠛⠛⠋⠁⠀⠀⠀⠀⢸⣿⣿⣿⣿⡆⠀⠀⠀⠀⠈⠙⠛⠛⠛⠉⠀⠀ -⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠸⣿⣿⣿⣿⠇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ -⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⣾⣿⣿⣷⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ -⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣸⣿⣿⣿⣿⣿⣿⣆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ -⠀⠀⠀⠀⠀⠀⠴⠶⠿⠿⠿⠿⠿⠿⠿⠿⠿⠿⠿⠿⠿⠿⠶⠦⠀⠀ -</pre> -<br /> -<h2 style='display: inline'>Operational Balance in SRE: Finding the Equilibrium in Reliability and Velocity</h2><br /> -<br /> -<span>Site Reliability Engineering has established itself as more than just a set of best practices or methodologies. Instead, it stands as a beacon of operational excellence, which guides engineering teams through the turbulent waters of modern software development and system management.</span><br /> -<br /> -<span>In the universe of software production, two fundamental forces are often at odds: The drive for rapid feature release (velocity) and the need for system reliability. Traditionally, the faster teams moved, the more risk was introduced into systems. SRE offers a approach to mitigate these conflicting drives through concepts like error budgets and SLIs/SLOs. These mechanisms offer a tangible metric, allowing teams to quantify how much they can push changes while ensuring they don't compromise system health. Thus, the error budget becomes a balancing act, where teams weigh the trade-offs between innovation and reliability.</span><br /> -<br /> -<span>An important part of this balance is the dichotomy between operations and coding. According to SRE principles, an engineer should ideally spend an equal amount of time on operations work and coding - 50% on each. This isn't just a random metric; it's a reflection of the value SRE places on both maintaining operational excellence and progressing forward with innovations. This balance ensures that while SREs are solving today's problems, they are also preparing for tomorrow's challenges. </span><br /> -<br /> -<span>However, not all operational tasks are equal. SRE differentiates between "ops work" and "toil". While ops work is integral to system maintenance and can provide value, toil represents repetitive, mundane tasks which offer little value in the long run. Recognising and minimising toil is crucial. A culture that allows engineers to drown in toil stifles innovation and growth. Hence, an organisation's approach to toil indicates its operational health and commitment to balance.</span><br /> -<br /> -<span>A cornerstone of achieving operational balance lies in the tools and processes SREs use. Effective monitoring, observability tools, and ensuring that tools can handle high cardinality data are foundational. These aren't just technical requisites but reflective of an organisational culture prioritising proactive problem-solving. By having systems that effectively flag potential issues before they escalate, SREs can maintain the balance between system stability and forward momentum.</span><br /> -<br /> -<span>Moreover, operational balance isn't just a technological or process challenge; it's a human one. The health of on-call engineers is as crucial as the health of the services they manage. On-call postmortems, continuous feedback loops, and recognising gaps (be it tooling, operational expertise, or resources) ensure that the human elements of operations are noticed. </span><br /> -<br /> -<span>In conclusion, operational balance in SRE isn't static thing but an ongoing journey. It requires organisations to constantly evaluate their practices, tools, and, most importantly, their culture. By achieving this balance, organisations can ensure that they have time for innovation while maintaining the robustness and reliability of their systems, resulting in sustainable long-term success.</span><br /> -<br /> -<span>That all sounds very romantic. The truth is, it's brutal to archive the perfect balance. No system will ever be perfect. But at least we should aim for it!</span><br /> -<br /> -<span>Continue with the third part of this series:</span><br /> -<br /> -<a class='textlink' href='./2023-08-20-site-reliability-engineering-part-3.html'>2023-08-20 Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>Site Reliability Engineering - Part 1: SRE and Organizational Culture</title> - <link href="gemini://foo.zone/gemfeed/2023-08-18-site-reliability-engineering-part-1.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-08-18-site-reliability-engineering-part-1.gmi</id> - <updated>2023-08-18T22:43:47+03:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>The universe of Site Reliability Engineering (SRE) is like an intricate tapestry woven with diverse technology, culture, and personal grit threads. Site Reliability Engineering is one of the most demanding jobs. With all the facets, it's impossible to get bored. There is always a new challenge to master, and there is always a new technology to tinker with. It's not just technical; it's also about communication, collaboration and teamwork. I am currently employed as a Principal Site Reliability Engineer and will try to share what SRE is about in this blog series.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Site Reliability Engineering - Part 1: SRE and Organizational Culture</h1><br /> -<br /> -<span class='quote'>Published at 2023-08-18T22:43:47+03:00</span><br /> -<br /> -<span>The universe of Site Reliability Engineering (SRE) is like an intricate tapestry woven with diverse technology, culture, and personal grit threads. Site Reliability Engineering is one of the most demanding jobs. With all the facets, it's impossible to get bored. There is always a new challenge to master, and there is always a new technology to tinker with. It's not just technical; it's also about communication, collaboration and teamwork. I am currently employed as a Principal Site Reliability Engineer and will try to share what SRE is about in this blog series.</span><br /> -<br /> -<a class='textlink' href='./2023-08-18-site-reliability-engineering-part-1.html'>2023-08-18 Site Reliability Engineering - Part 1: SRE and Organizational Culture (You are currently reading this)</a><br /> -<a class='textlink' href='./2023-08-19-site-reliability-engineering-part-2.html'>2023-08-19 Site Reliability Engineering - Part 2: Operational Balance in SRE</a><br /> -<a class='textlink' href='./2023-08-20-site-reliability-engineering-part-3.html'>2023-08-20 Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect</a><br /> -<br /> -<pre> -▓▓▓▓░░ - -DC on fire: - - ▓▓ ▓▓ ▓▓ - ░░ ░░ ▓▓▓▓ ██ ░░ ▓▓▓▓ ▓▓ - ▓▓░░░░ ░░ ▓▓▓▓ ▓▓░░ ▓▓▓▓ - ░░░░ ▓▓▓▓▓▓ ▓▓ ▓▓ ▓▓ ▓▓▓▓▓▓ ▓▓ - ▓▓░░ ▓▓▒▒▒▒▓▓▓▓ ▓▓ ▓▓▓▓ ▓▓▓▓▓▓ ▓▓▒▒▒▒▓▓▓▓ ▓▓▓▓ - ██▓▓ ▓▓▒▒░░▒▒▓▓ ▓▓██ ▓▓▓▓▓▓ ▓▓▒▒▓▓ ▓▓▒▒░░▒▒▓▓ ██▓▓▓▓ - ▓▓▓▓██ ▓▓▒▒░░░░▒▒▓▓ ▓▓▓▓ ▓▓▒▒▒▒▓▓ ▓▓▒▒░░▒▒▓▓██▓▓ ▓▓▒▒░░░░▒▒▓▓ ▓▓▒▒▒▒▓▓ - ▓▓▒▒▒▒▓▓▓▓▒▒░░▒▒▓▓▓▓▓▓▒▒▒▒▓▓ ▓▓▓▓░░▒▒▓▓ ▓▓▒▒░░▒▒▓▓▒▒▒▒▓▓ ▓▓▒▒░░▒▒▓▓▓▓▓▓▓▓░░▒▒▓▓ - ▒▒░░▒▒▓▓▓▓▒▒░░▒▒▓▓▓▓▒▒░░▒▒▓▓ ▓▓▒▒░░▒▒▓▓ ▓▓░░░░▒▒▒▒░░░░▒▒██████▒▒░░▒▒██▓▓▓▓▒▒░░▒▒▓▓██ - ░░░░▒▒▓▓▒▒░░▒▒▓▓▓▓▓▓▒▒░░▒▒▓▓██▒▒░░░░▒▒▓▓ ▓▓▒▒░░▒▒▓▓▒▒▒▒░░▒▒▓▓▓▓▒▒░░▒▒▓▓▓▓▓▓▒▒░░░░▒▒▓▓▓▓ - ░░░░▒▒▓▓▒▒░░░░▓▓██▒▒░░░░▒▒▓▓██▒▒░░░░▒▒██▓▓▓▓▒▒░░▒▒▓▓▓▓▒▒░░░░▒▒▓▓▒▒░░░░██▓▓▓▓▒▒░░░░▒▒████ - ▒▒░░▒▒▓▓▓▓░░░░▒▒▓▓▒▒▒▒░░░░▒▒▓▓▓▓▒▒░░░░▒▒▓▓▓▓▒▒░░░░▒▒▓▓▒▒░░▒▒▓▓▓▓▓▓░░░░▒▒▓▓▓▓▓▓▒▒░░░░▒▒▓▓ - ▒▒░░▒▒▓▓▒▒▒▒░░▒▒██▒▒▒▒░░▒▒▒▒██▒▒▒▒░░░░░░▒▒▓▓▒▒░░░░▒▒▒▒░░░░▒▒████▒▒▒▒░░▒▒██▓▓▒▒▒▒░░░░░░▒▒ - ░░░░░░▒▒░░░░░░░░▒▒▒▒▒▒░░░░▒▒▒▒▒▒░░░░░░░░▒▒▒▒░░░░░░▒▒▒▒░░░░░░▒▒▒▒░░░░░░░░▒▒▒▒▒▒░░░░░░░░▒▒ - ░░░░░░░░░░▒▒░░░░░░░░░░░░░░░░░░░░░░░░▒▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▒▒░░░░░░░░░░░░░░░░░░ -</pre> -<br /> -<h2 style='display: inline'>SRE and Organizational Culture: Navigating the Nexus</h2><br /> -<br /> -<span>At the heart of SRE lies the proactive mindset of "prevention over cure." Traditional IT models focused predominantly on reactive solutions, but SRE mandates a shift towards foresight. By adopting Service Level Indicators (SLIs) and Service Level Objectives (SLOs), teams are equipped with clear metrics and goals that guide them toward ensuring reliability and user satisfaction. They reflect an organisational culture prioritising user experience and constant system alignment with user needs. </span><br /> -<br /> -<span>Another defining SRE idea concept the "error budget." This ingenious framework accepts that no system is flawless. Failures are inevitable. However, instead of being punitive, the culture here is to accept, learn, and iterate. By providing teams with a "budget" for errors, organisations create an environment where innovation is encouraged, and failures are viewed as learning opportunities.</span><br /> -<br /> -<span>But SRE isn't just about technology and metrics; it's deeply human. It challenges the "hero culture" that plagues many IT teams. While individual heroics might occasionally save the day, a sustainable model requires collective expertise. An SRE culture recognises that heroes achieve their best within teams, negating the need for a hero-centric environment. This philosophy promotes a balanced on-call experience, emphasising the importance of trust, ownership, effective communication, and collaboration as cornerstones of team success. I personally have fallen into the hero trap, and know it's unsustainable to be the only go-to person for every problem.</span><br /> -<br /> -<span>Additionally, the SRE model requires good documentation. However, it's essential ensuring that this documentation undergoes the same quality checks as code, reinforcing effective onboarding, training and communication.</span><br /> -<br /> -<span>Organisations might face a significant challenge when adopting SRE. Some might feel SRE principles counter their goals. They might prioritise feature rollouts over reliability or view SRE practices as cumbersome. Hence, creating an SRE culture often demands patient explanations and showcasing benefits, such as increased release velocity and improved user experience.</span><br /> -<br /> -<span>Monitoring and observability form another SRE aspect, emphasising the need for high-quality tools to query and analyse data. This ties back to the cultural emphasis on continuous learning and adaptability. SREs, by nature, need to be curious, ready to delve into anomalies, and keen on adopting new tools and practices. </span><br /> -<br /> -<span>The success of SRE within any organisation depends on the broader acceptance of its principles. It demands a move away from siloed operations, where SRE acts as a bandage on flawed systems, to a model where reliability is everyone's responsibility.</span><br /> -<br /> -<span>In essence, the integration of SRE principles transcends technical practices. It paves the way for a shift in organisational culture that values proactive prevention, continuous learning, collaboration, and transparent communication. The successful melding of SRE and corporate culture promises not just reliable systems but also a robust, resilient, and progressive work environment.</span><br /> -<br /> -<span>Organisations with the implementation of SLIs, SLOs and error budgets are already advanced in their SRE journey. It takes a lot of communication, convincing, and patience until that point is reached.</span><br /> -<br /> -<span>Continue with the second part of this series:</span><br /> -<br /> -<a class='textlink' href='./2023-08-19-site-reliability-engineering-part-2.html'>2023-08-19 Site Reliability Engineering - Part 2: Operational Balance in SRE</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>Gemtexter 2.1.0 - Let's Gemtext again³</title> - <link href="gemini://foo.zone/gemfeed/2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.gmi</id> - <updated>2023-07-21T10:19:31+03:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>I proudly announce that I've released Gemtexter version `2.1.0`. What is Gemtexter? It's my minimalist static site generator for Gemini Gemtext, HTML and Markdown, written in GNU Bash.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Gemtexter 2.1.0 - Let's Gemtext again³</h1><br /> -<br /> -<span class='quote'>Published at 2023-07-21T10:19:31+03:00</span><br /> -<br /> -<pre> --=[ typewriters ]=- 1/98 - .-------. - .-------. _|~~ ~~ |_ - _|~~ ~~ |_ .-------. =(_|_______|_) - =(_|_______|_)= _|~~ ~~ |_ |:::::::::| - |:::::::::| =(_|_______|_) |:::::::[]| - |:::::::[]| |:::::::::| |o=======.| - |o=======.| |:::::::[]| `"""""""""` - jgs `"""""""""` |o=======.| - mod. by Paul Buetow `"""""""""` -</pre> -<br /> -<span>I proudly announce that I've released Gemtexter version <span class='inlinecode'>2.1.0</span>. What is Gemtexter? It's my minimalist static site generator for Gemini Gemtext, HTML and Markdown, written in GNU Bash.</span><br /> -<br /> -<a class='textlink' href='https://codeberg.org/snonux/gemtexter'>https://codeberg.org/snonux/gemtexter</a><br /> -<br /> -<h2 style='display: inline'>Why Bash?</h2><br /> -<br /> -<span>This project is too complex for a Bash script. Writing it in Bash was to try out how maintainable a "larger" Bash script could be. It's still pretty maintainable and helps me try new Bash tricks here and then!</span><br /> -<br /> -<span>Let's list what's new!</span><br /> -<br /> -<h2 style='display: inline'>Switch to GPL3 license</h2><br /> -<br /> -<span>Many (almost all) of the tools and commands (GNU Bash, GMU Sed, GNU Date, GNU Grep, GNU Source Highlight) used by <span class='inlinecode'>Gemtexter</span> are licensed under the GPL anyway. So why not use the same? This was an easy switch, as I was the only code contributor so far!</span><br /> -<br /> -<h2 style='display: inline'>Source code highlighting support</h2><br /> -<br /> -<span>The HTML output now supports source code highlighting, which is pretty neat if your site is about programming. The requirement is to have the <span class='inlinecode'>source-highlight</span> command, which is GNU Source Highlight, to be installed. Once done, you can annotate a bare block with the language to be highlighted. E.g.:</span><br /> -<br /> -<pre> - ```bash - if [ -n "$foo" ]; then - echo "$foo" - fi - ``` -</pre> -<br /> -<span>The result will look like this (you can see the code highlighting only in the Web version, not in the Geminispace version of this site):</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">if</font></b> <font color="#990000">[</font> -n <font color="#FF0000">"$foo"</font> <font color="#990000">];</font> <b><font color="#0000FF">then</font></b> - echo <font color="#FF0000">"$foo"</font> -<b><font color="#0000FF">fi</font></b> -</pre> -<br /> -<span>Please run <span class='inlinecode'>source-highlight --lang-list</span> for a list of all supported languages.</span><br /> -<br /> -<h2 style='display: inline'>HTML exact variant</h2><br /> -<br /> -<span>Gemtexter is there to convert your Gemini Capsule into other formats, such as HTML and Markdown. An HTML exact variant can now be enabled in the <span class='inlinecode'>gemtexter.conf</span> by adding the line <span class='inlinecode'>declare -rx HTML_VARIANT=exact</span>. The HTML/CSS output changed to reflect a more exact Gemtext appearance and to respect the same spacing as you would see in the Geminispace. </span><br /> -<br /> -<h2 style='display: inline'>Use of Hack webfont by default</h2><br /> -<br /> -<span>The Hack web font is a typeface designed explicitly for source code. It's a derivative of the Bitstream Vera and DejaVu Mono lineage, but it features many improvements and refinements that make it better suited to reading and writing code.</span><br /> -<br /> -<span>The font has distinctive glyphs for every character, which helps to reduce confusion between similar-looking characters. For example, the characters "0" (zero), "O" (capital o), and "o" (lowercase o), or "1" (one), "l" (lowercase L), and "I" (capital i) all have distinct looks in Hack, making it easier to read and understand code at a glance.</span><br /> -<br /> -<span>Hack is open-source and freely available for use and modification under the MIT License.</span><br /> -<br /> -<h2 style='display: inline'>HTML Mastodon verification support</h2><br /> -<br /> -<span>The following link explains how URL verification works in Mastodon:</span><br /> -<br /> -<a class='textlink' href='https://joinmastodon.org/verification'>https://joinmastodon.org/verification</a><br /> -<br /> -<span>So we have to hyperlink to the Mastodon profile to be verified and also to include a <span class='inlinecode'>rel='me'</span> into the tag. In order to do that add this to the <span class='inlinecode'>gemtexter.conf</span> (replace the URI to your Mastodon profile accordingly):</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">declare</font></b> -xr <font color="#009900">MASTODON_URI</font><font color="#990000">=</font><font color="#FF0000">'https://fosstodon.org/@snonux'</font> -</pre> -<br /> -<span>and add the following into your <span class='inlinecode'>index.gmi</span>:</span><br /> -<br /> -<pre> -=> https://fosstodon.org/@snonux Me at Mastodon -</pre> -<br /> -<span>The resulting line in the HTML output will be something as follows:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF"><a</font></b> <font color="#009900">href</font><font color="#990000">=</font><font color="#FF0000">'https://fosstodon.org/@snonux'</font> <font color="#009900">rel</font><font color="#990000">=</font><font color="#FF0000">'me'</font><b><font color="#0000FF">></font></b>Me at Mastodon<b><font color="#0000FF"></a></font></b> -</pre> -<br /> -<h2 style='display: inline'>More</h2><br /> -<br /> -<span>Additionally, there were a couple of bug fixes, refactorings and overall improvements in the documentation made. </span><br /> -<br /> -<span>Other related posts are:</span><br /> -<br /> -<a class='textlink' href='./2021-04-24-welcome-to-the-geminispace.html'>2021-04-24 Welcome to the Geminispace</a><br /> -<a class='textlink' href='./2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html'>2021-06-05 Gemtexter - One Bash script to rule it all</a><br /> -<a class='textlink' href='./2022-08-27-gemtexter-1.1.0-lets-gemtext-again.html'>2022-08-27 Gemtexter 1.1.0 - Let's Gemtext again</a><br /> -<a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again²</a><br /> -<a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³ (You are currently reading this)</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>'Software Developmers Career Guide and Soft Skills' book notes</title> - <link href="gemini://foo.zone/gemfeed/2023-07-17-career-guide-and-soft-skills-book-notes.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-07-17-career-guide-and-soft-skills-book-notes.gmi</id> - <updated>2023-07-17T04:56:20+03:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>These notes are of two books by 'John Sommez' I found helpful. I also added some of my own keypoints to it. These notes are mainly for my own use, but you might find them helpful, too.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>"Software Developmers Career Guide and Soft Skills" book notes</h1><br /> -<br /> -<span class='quote'>Published at 2023-07-17T04:56:20+03:00</span><br /> -<br /> -<span>These notes are of two books by "John Sommez" I found helpful. I also added some of my own keypoints to it. These notes are mainly for my own use, but you might find them helpful, too.</span><br /> -<br /> -<pre> - ,.......... .........., - ,..,' '.' ',.., - ,' ,' : ', ', - ,' ,' : ', ', - ,' ,' : ', ', - ,' ,'............., : ,.............', ', -,' '............ '.' ............' ', - '''''''''''''''''';''';'''''''''''''''''' - ''' -</pre> -<br /> -<h1 style='display: inline'>Improve</h1><br /> -<br /> -<h2 style='display: inline'>Always learn new things</h2><br /> -<br /> -<span>When you learn something new, e.g. a programming language, first gather an overview, learn from multiple sources, play around and learn by doing and not consuming and form your own questions. Don't read too much upfront. A large amount of time is spent in learning technical skills which were never use. You want to have a practical set of skills you are actually using. You need to know 20 percent to get out 80 percent of the results.</span><br /> -<br /> -<ul> -<li>Learn a technology with a goal, e.g. implement a tool. Practice practise practice.</li> -<li>"I know X can do Y, I don't know exactly how, but I can look it up."</li> -<li>Read what experts are writing, for example follow blogs. Stay up to date and spent half an hour per day trading blogs and books.</li> -<li>Pick an open source application, read the code and try to understand it to get a feel of the syntax of the programming language.</li> -<li>Understand, that the standard library makes you a much better programmer.</li> -<li>Self learning is the top skill a programmer can have and is also useful in other aspects in your life.</li> -<li>Keep learning skills every day. Code every day. Don't be overconfident for job security. Read blogs, read books.</li> -<li>If you want to learn, then do it by exploring. Also teach what you learned (for example write a blog post or hold a presentation).</li> -</ul><br /> -<span>Fake it until you make it. But be honest about your abilities or lack of. There is however only time between now and until you make it. Refer to your abilities to learn.</span><br /> -<br /> -<span>Boot camps: The advantage of a boot camp is to pragmatically learn things fast. We almost always overestimate what we can do in a day. Especially during boot camps. Connect to others during the boot camps</span><br /> -<br /> -<h2 style='display: inline'>Set goals</h2><br /> -<br /> -<span>Your own goals are important but the manager also looks at how the team performs and how someone can help the team perform better. Check whether you are on track with your goals every 2 weeks in order to avoid surprises for the annual review. Make concrete goals for next review. Track and document your progress. Invest in your education. Make your goals known. If you want something, then ask for it. Nobody but you knows what you want.</span><br /> -<br /> -<h2 style='display: inline'>Ratings</h2><br /> -<br /> -<span>That's a trap: If you have to rate yourself, that's a trap. That never works in an unbiased way. Rate yourself always the best way but rate your weakest part as high as possible minus one point. Rate yourself as good as you can otherwise. Nobody is putting for fun a gun on his own head. </span><br /> -<br /> -<ul> -<li>Don't do peer rating, it can fire back on you. What if the colleague becomes your new boss?</li> -<li>Cooperate rankings are unfortunately HR guidelines and politics and only mirror a little your actual performance.</li> -</ul><br /> -<h2 style='display: inline'>Promotions</h2><br /> -<br /> -<span>The most valuable employees are the ones who make themselves obsolete and automate all away. Keep a safety net of 3 to 6 months of finances. Safe at least 10 percent of your earnings. Also, if you make money it does not mean that you have to spent more money. Is a new car better than a used car which both can bring you from A to B? Liability vs assets.</span><br /> -<br /> -<ul> -<li>Raise or promotion, what's better? Promotion is better as money will follow anyway then.</li> -<li>Take projects no-one wants and make them shine. A promotion will follow.</li> -<li>A promotion is not going to come to you because you deserve it. You have to hunt and ask for it.</li> -<li>Track all kudos (e.g. ask for emails from your colleagues).</li> -<li>Big corporations HRs don't expect a figjit. That's why it's so important to keep track of your accomplishments and kudos'.</li> -<li>If you want a raise be specific how much and know to back your demands. Don't make a thread and no ultimatums.</li> -<li>Best way for a promotion is to switch jobs. You can even switch back with a better salary.</li> -</ul><br /> -<h2 style='display: inline'>Finish things</h2><br /> -<br /> -<span>Hard work is necessary for accomplish results. However, work smarter not harder. Furthermore, working smart is not a substitute for working hard. Work both, hard and smart.</span><br /> -<br /> -<ul> -<li>Learn to finish things without motivation. Things will pay off when you stick to stuff and eventually motivation can also come back.</li> -<li>You will fail if you don't plan realistically. Set also a schedule and follow to it as of life depends on it.</li> -<li>Advances come only of you give more than asked. Consistency, commitment and knowing what you need to do is more key than hard work.</li> -<li>Any action is better than no action. If you get stuck you have gained nothing.</li> -<li>You need to know the unknowns. Identify as many unknown not known things as possible. </li> -</ul><br /> -<span>Hard vs fun: Both engage the brain (video games vs work). Some work is hard and other is easy. Hard work is boring. The harsh truth is you have to put in hard and boring work in order to accomplish and be successful. Work won't be always boring though, as joy will follow with mastery.</span><br /> -<br /> -<span>Defeat is finally give up. Failure is the road to success, embrace it. Failure does not define you but how you respond to it. Events don't make your unhappy, but how you react to events do.</span><br /> -<br /> -<h2 style='display: inline'>Expand the empire</h2><br /> -<br /> -<span>The larger your empire is, the larger your circle of influence is. The larger the circle of influence is, the more opportunities you have.</span><br /> -<br /> -<ul> -<li>Do the dirty work if you want to expand the empire. That's there the opportunities are.</li> -<li>SCRUM often fails due to the lack to commitment. The backlog just becomes a wish to get completed.</li> -<li>Apply work on your quality standards. Don't cross the line of compromise. Always improve your skills. Never be happy being good enough.</li> -</ul><br /> -<span>Become visible, keep track that you accomplishments. E.g. write a weekly summary. Do presentations, be seen. Learn new things and share your learnings. Be the problem solver and not the blamer.</span><br /> -<br /> -<h2 style='display: inline'>Be pragmatic and also manage your time</h2><br /> -<br /> -<span>Make use of time boxing via the Pomodoro technique: Set a target of rounds and track the rounds. That give you exact focused work time. That's really the trick. For example set a goal of 6 daily pomodores.</span><br /> -<br /> -<ul> -<li>Every time you do something question why does it make sense be pragmatic and don't follow because it is best practice.</li> -<li>You can also apply the time boxing technique (Cal Newport) for focused deep work.</li> -</ul><br /> -<span>You should feel good of the work done even if you don't finished the task. You will feel good about pomodoro wise even you don't finish the task on hand yet. Helps you to enjoy time off more. Working longer may not sell anything.</span><br /> -<br /> -<h3 style='display: inline'>The quota system</h3><br /> -<br /> -<span>Defined quota of things done. E.g. N runs per week or M Blog posts per month or O pomodoros per week. This helps with consistency. Truly commit to these quotas. Failure is not an option. Start with small commitments. Don't commit to something you can't fulfill otherwise you set yourself up for failure.</span><br /> -<br /> -<ul> -<li>Why does the quota System work? Slow and consistent pace is the key. It also overcomes willpower weaknesses as goals are preset.</li> -<li>Internal motivation is more important over external motivation. Check out Daniels book drive.</li> -<li>Multitasking: Batching is effective. E.g. emails twice daily at pre-set times..</li> -</ul><br /> -<h3 style='display: inline'>Don't waste time</h3><br /> -<br /> -<span>The biggest time waster is TV watching. The TV is programming you. It's insane that Americans watch so much TV as they work full time. Schedule one show at a time and watch it when you want to watch it. Most movies are crap anyways. The good movies will come to you as people will talk about them.</span><br /> -<br /> -<ul> -<li>Social media is time waster as well. Schedule your Social Media times. For example be on Facebook only for max one hour on Saturdays.</li> -<li>Meetings can waste time as well. Simply don't go to them. Try to cancel meeting if it can be dealt with via email.</li> -<li>Enjoying things is not a waste of time. E.g. you could still play a game once in a while. It is important not to cut away all you enjoy from your life.</li> -</ul><br /> -<h3 style='display: inline'>Habits</h3><br /> -<br /> -<span>Try to have as many good habits as possible. Start with easy habits, and make them a little bit more challenging over time. Set ankers and rewards. Over time the routines will become habits naturally.</span><br /> -<br /> -<span>Habit stacking is effective, which is combining multiple habits at the same time. For example you can workout on a circular trainer while while watching a learning video on O'Reilly Safari Online while getting closer to your weekly step goal.</span><br /> -<br /> -<ul> -<li>We don't have control over our habits but our own routines.</li> -<li>Routines help to form the habits, though.</li> -</ul><br /> -<h1 style='display: inline'>Work-life balance</h1><br /> -<br /> -<span>Avoid overwork hours. That's not as beneficial as you might think and comes only with very small rewards. Invest rather in yourself and not in your employer.</span><br /> -<br /> -<ul> -<li>Work-life balance is a myth. Make it so that you enjoy work and your personal life and not just personal life.</li> -<li>Maintain fewer but good relationships. As a reward, better and integrated your life will be.</li> -<li>Life in the present Moment. Make the best of every moment of your life.</li> -<li>Enjoy every aspect of your life. If you want to take away one thing from this book that is it.</li> -</ul><br /> -<span>Use your most productive hours to work on you. Make that your priority. Take care of yourself a priority (E.g. do workouts or learn a new language). You can always workout 2 or 1 hour per day, but will you pay the price?</span><br /> -<br /> -<h2 style='display: inline'>Mental health</h2><br /> -<br /> -<ul> -<li>Friendships and positive thinking help to have and maintain better health, longer Life, better productivity and increased happiness.</li> -<li>Positive thinking can be trained and be a habit. Read the book "The Power of Positive Thinking".</li> -<li>Stoicism helps. Meditation helps. Playing for fun helps too.</li> -</ul><br /> -<span>Become the person you want to become (your self image). Program your brain unconsciously. Don't become the person other people want you to be. Embrace yourself, you are you.</span><br /> -<br /> -<span>In most cases burnout is just an illusion. If you don't have motivation push through the wall. People usually don't pass the wall as they feel they are burned out. After pushing through the wall you will have the most fun, for example you will be able playing the guitar greatly.</span><br /> -<br /> -<h2 style='display: inline'>Physical health</h2><br /> -<br /> -<span>Utilise a standing desk and treadmill (you could walk and type at the same time). Increase the incline in order to burn more calories. Even on the standing desk you burn more calories than sitting. When you use pomodoro then you can use the small breaks for push-ups (maybe won't do as good when you are in a fasted state).</span><br /> -<br /> -<ul> -<li>You can only do one thing, lose fat or gain muscles. Not both at the same time.</li> -<li>Train your strength by heavy lifting, but only with a very few repetitions (e.g. 5 max for each exercise, everything over this is body building).</li> -<li>If you want to increase the muscle mass use medium weights but lift them more often. If you want to increase your endurance lift light weights but with even more reps.</li> -<li>Avoid highly processed foods</li> -</ul><br /> -<span>Intermittent fasting is an effective method to maintain weight and health. But it does not mean that you can only eat junk food in the feeding windows. Also, diet and nutrition is the most important for health and fitness. They make it also easier to stay focused and positive.</span><br /> -<br /> -<h2 style='display: inline'>No drama</h2><br /> -<br /> -<span>Avoid drama at work. Where are humans there is drama. You can decide where to spent your energy in. But don't avoid conflict. Conflict is healthy in any kind of relationship. Be tactful and state your opinion. The goal is to find the best solution to the problem.</span><br /> -<br /> -<span>Don't worry about other people what they do and don't do. You only worry about you. Shut up and get your own things done. But you could help to inspire a not working colleague.</span><br /> -<br /> -<ul> -<li>During an argument, take the opponent's position and see how your opinion changes.</li> -<li>If you they to convince someone else it's an argument. Of you try to find the best solution it is a good resolution.</li> -<li>If someone is hurting the team let the manager know but phrase it nicely.</li> -<li>How to get rid of a never ending talking person? Set up focus hours officially where you don't want to be interrupted. Present as if it is your defect that you get interrupted easily.</li> -<li>TOXIC PEOPLE: AVOID THEM. RUN.</li> -<li>Boss likes if you get shit done without getting asked all the time about things and also without drama.</li> -</ul><br /> -<span>You have to learn how to work in a team. Be honest but tactful. It's not too be the loudest but about selling your ideas. Don't argue otherwise you won't sell anything. Be persuasive by finding the common ground. Or lead the colleagues to your idea and don't sell it upfront. Communicate clearly.</span><br /> -<br /> -<h1 style='display: inline'>Personal brand</h1><br /> -<br /> -<ul> -<li>Invest your value outside the company. Build your personal brand. Show how valuable you are, also to other companies. Become an asset.</li> -<li>Invest in your education. Make your goals known. If you want something ask for it (see also the sections about goals in this document).</li> -</ul><br /> -<h2 style='display: inline'>Market yourself</h2><br /> -<br /> -<ul> -<li>The best way to market yourself is to make you usable.</li> -<li>Create a brand. Decide your focus. Throw your name out as often as possible.</li> -</ul><br /> -<span>Have a blog. Schedule your posts. Consistency beats every other factor. E.g. post once a month a new post. Find your voice, you don't have to sound academic. Keep writing, if you keep it long enough the rewards will be coming. Your own blog can take 5 years to take off. Most people give up too soon.</span><br /> -<br /> -<ul> -<li>Consistency of your blog is key. Also write quality content. Don't try to be a man of success but try to be a man of value.</li> -<li>Have an elevator pitch: "buetow.org - Having fun with computers!"</li> -<li>Have social media accounts, especially the ones which are more tech related.</li> -</ul><br /> -<h2 style='display: inline'>Networking</h2><br /> -<br /> -<span>Ask people so they talk about themselves. They are not really interested in you. Use meetup.com to find groups you are interested and build up the network over time. Don't drink on social networking events even when others do. Talking to other people at events only has upsides. Just saying "hi" and introducing yourself is enough. What worse can happen? If the person rejects you so what, life goes on. Ask open questions and no "yes" and "no" questions. E.g.: "What is your story, why are you here?".</span><br /> -<br /> -<h2 style='display: inline'>Public speaking</h2><br /> -<br /> -<span>Before your talk go on stage 10 minutes in advance. Introduce yourself to the front row people. During the talk they will smile at you and encourage you during your talk.</span><br /> -<br /> -<ul> -<li>Try at least 5 times before giving up public speaking. You can also start small, e.g. present a topic at work you are learning.</li> -<li>Practise your talk and timing. You can also record your practicing.</li> -</ul><br /> -<span>Just do it. Just go to conferences. Even if you are not speaking. Sell your boss what you would learn and "this and that" and you would present the learnings to the team afterwards.</span><br /> -<br /> -<h1 style='display: inline'>New job</h1><br /> -<br /> -<h2 style='display: inline'>For the interview</h2><br /> -<br /> -<ul> -<li>Build up a network before the interview. E.g., follow and comment blogs. Or go to meet-ups and conferences. Join user groups.</li> -<li>Ask to touch base before the real interview and ask questions about the company. Do "pre-interviews".</li> -<li>Have a blog, a CV can only be 2 pages and an interview only can last only 2 hours. A blog helps you also to be a better communicator.</li> -</ul><br /> -<span>If you are specialized then there is a better chance to get a fitting job. No one will hire a general lawyer if there are specialized lawyers available. Even if you are specialized, you will have a wide range of skills (T-shape knowledge).</span><br /> -<br /> -<h2 style='display: inline'>Find the right type of company</h2><br /> -<br /> -<span>Not all companies are equal. They have individual cultures and guidelines.</span><br /> -<br /> -<ul> -<li>Startup: dynamic and larger impact. Many hats on.</li> -<li>Medium size companies: most stable ones. Not cutting edge technologies. No crazy working hours.</li> -<li>Large company: very established with a lot of structure however constant layoffs and restructurings. Less impact you can have. Complex politics.</li> -<li>Working for yourself: This is harder than you think, probably much harder.</li> -</ul><br /> -<span>Work in a tech. company if you want to work on/with cutting edge technologies.</span><br /> -<br /> -<h2 style='display: inline'>Apply for the new job</h2><br /> -<br /> -<span>Get a professional resume writer. Get referrals of writers and get samples from there. Get sufficient with algorithm and data structures interview questions. Cracking the coding interview book and blog </span><br /> -<br /> -<ul> -<li>Apply for each job with a specialised CV each. Each CV fits the job better.</li> -<li>Best get a job via a personal referral or inbound marketing. The latter is somehow rare.</li> -<li>Inbound marketing is for example someone responds to your blog and offers you a job.</li> -<li>Interview the interviewer. Be persistent.</li> -<li>Create creative looking resumes, see simple programmer website. Action-result style for a resume.</li> -</ul><br /> -<span>Invest in your dress code as appearance masters. It does make sense to invest in your style. You could even hire a professional stylist (not my personal way though).</span><br /> -<br /> -<h2 style='display: inline'>Negotiation</h2><br /> -<br /> -<ul> -<li>Whoever names the number first loses. You don't know what someone else is expecting unless told. Low ball number may be an issue but you have to know the market.</li> -<li>Salary is not about what you need but what you are worth. Try to find out what you are worth.</li> -<li>Big tech companies have a pay scale. You can ask for this.</li> -<li>Don't tell your current salary. Only do one counter offer and say "If you do X then I commit today". Be tactful and not rude. Nobody wants to be taken advantage of. Also, don't be arrogant.</li> -<li>If the company wants to know your range, respond: "I would rather learn more about the job and compensation. You have a range in mind, correct?" Be brave and just pause here.</li> -<li>Otherwise, if the company refuses then say "if you tell me what the range is and although I am not yet sure yet what are my exact salary requirements are I can see if the range is of what I am looking for. If they absolute refuse give a high ball range you would expect and make it conditional to the overall compensation package. E.g. 70k to 100k depending on the compensation package. THE LOW END SHOULD BE YOUR REAL LOW END. Play a little bit of hardball here and be brave. Practise it.</li> -<li>Put 10 percent on top of the salary range into a counter offer.</li> -<li>Everything is negotiable, not only the salary.</li> -<li>Job markup rate: Check it regarding the recruitment rate negotiation.</li> -<li>Don't make a rushed decision based on deadlines. Make a fairly high counter offer shortly before deadline.</li> -<li>You should also cope with rejections while selling yourself. There is no such thing as job security.</li> -</ul><br /> -<ul> -<li>Never spilt the difference is the best book for learning negotiation techniques..</li> -</ul><br /> -<h2 style='display: inline'>Leaving the old job</h2><br /> -<br /> -<span>When leaving a job make a clean and non personal as possible. Never complain and never explain. Don't worry about abandonment of the team. Everybody is replacement and you make a business decision. Don't threaten to quit as you are replaceable.</span><br /> -<br /> -<h1 style='display: inline'>Other things</h1><br /> -<br /> -<ul> -<li>As a leader lead by example and don't lead from the Eiffel tower.</li> -<li>As a leader you are responsible for the team. If the team fails then it's your fault only.</li> -</ul><br /> -<h2 style='display: inline'>Testing</h2><br /> -<br /> -<span>Unit testing Vs regression testing: Unit tests test the smallest possible unit and get rewritten if the unit gets changed. It's like programming against a specification n. Regression tests test whether the software still works after the change. Now you know more than most software engineers.</span><br /> -<br /> -<h2 style='display: inline'>Books to read</h2><br /> -<br /> -<ul> -<li>Clean Code</li> -<li>Code Complete</li> -<li>Cracking the Interview - Lessons and Solutions.</li> -<li>Daniels Book "Drive" (about internal and external motivation)</li> -<li>God's degree (inventor of Dilbert)</li> -<li>Head first Design Patterns</li> -<li>How to win Friends and influence People</li> -<li>Never Split the Difference [X]</li> -<li>Structure and programming functional programs</li> -<li>The obstacle is the way [X]</li> -<li>The passionate programmer</li> -<li>The Power of Positive Thinking (Highly religious - I personally don't like it)</li> -<li>The Pragmatic Programmer [X]</li> -<li>The war of Art (to combat procrastination)</li> -<li>Willpower Instinct</li> -</ul><br /> -<span>Other book notes of mine are:</span><br /> -<br /> -<a class='textlink' href='./2023-03-16-the-pragmatic-programmer-book-notes.html'>2023-03-16 "The Pragmatic Programmer" book notes</a><br /> -<a class='textlink' href='./2023-04-01-never-split-the-difference-book-notes.html'>2023-04-01 "Never split the difference" book notes</a><br /> -<a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes</a><br /> -<a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes (You are currently reading this)</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../resources.html'>More books and other resources I found useful.</a><br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>KISS server monitoring with Gogios</title> - <link href="gemini://foo.zone/gemfeed/2023-06-01-kiss-server-monitoring-with-gogios.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-06-01-kiss-server-monitoring-with-gogios.gmi</id> - <updated>2023-06-01T21:10:17+03:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>Gogios is a minimalistic and easy-to-use monitoring tool I programmed in Google Go designed specifically for small-scale self-hosted servers and virtual machines. The primary purpose of Gogios is to monitor my personal server infrastructure for `foo.zone`, my MTAs, my authoritative DNS servers, my NextCloud, Wallabag and Anki sync server installations, etc.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>KISS server monitoring with Gogios</h1><br /> -<br /> -<span class='quote'>Published at 2023-06-01T21:10:17+03:00</span><br /> -<br /> -<a href='./2023-06-01-kiss-server-monitoring-with-gogios/gogios-small.png'><img alt='Gogios logo' title='Gogios logo' src='./2023-06-01-kiss-server-monitoring-with-gogios/gogios-small.png' /></a><br /> -<br /> -<h2 style='display: inline'>Introduction</h2><br /> -<br /> -<span>Gogios is a minimalistic and easy-to-use monitoring tool I programmed in Google Go designed specifically for small-scale self-hosted servers and virtual machines. The primary purpose of Gogios is to monitor my personal server infrastructure for <span class='inlinecode'>foo.zone</span>, my MTAs, my authoritative DNS servers, my NextCloud, Wallabag and Anki sync server installations, etc.</span><br /> -<br /> -<span>With compatibility with the Nagios Check API, Gogios offers a simple yet effective solution to monitor a limited number of resources. In theory, Gogios scales to a couple of thousand checks, though. You can clone it from Codeberg here:</span><br /> -<br /> -<a class='textlink' href='https://codeberg.org/snonux/gogios'>https://codeberg.org/snonux/gogios</a><br /> -<br /> -<pre> - _____________________________ ____________________________ - / \ / \ - | _______________________ || ______________________ | - | / \ || / \ | - | | # Alerts with status c| || | # Unhandled alerts: | | - | | hanged: | || | | | - | | | || | CRITICAL: Check Pizza| | - | | OK->CRITICAL: Check Pi| || | : Late delivery | | - | | zza: Late delivery | || | | | - | | | || | WARNING: Check Thirst| | - | | | || | : OutofKombuchaExcept| | - | \_______________________/ || \______________________/ | - | /|\ GOGIOS MONITOR 1 _ || /|\ GOGIOS MONITOR 2 _ | - \_____________________________/ \____________________________/ - !_________________________! !________________________! - ------------------------------------------------- -ASCII art was modified by Paul Buetow -The original can be found at -https://asciiart.website/index.php?art=objects/computers -</pre> -<br /> -<h2 style='display: inline'>Motivation</h2><br /> -<br /> -<span>With experience in monitoring solutions like Nagios, Icinga, Prometheus and OpsGenie, these tools often came with many features that I didn't necessarily need for personal use. Contact groups, host groups, check clustering, and the requirement of operating a DBMS and a WebUI added complexity and bloat to my monitoring setup.</span><br /> -<br /> -<span>My primary goal was to have a single email address for notifications and a simple mechanism to periodically execute standard Nagios check scripts and notify me of any state changes. I wanted the most minimalistic monitoring solution possible but wasn't satisfied with the available options.</span><br /> -<br /> -<span>This led me to create Gogios, a lightweight monitoring tool tailored to my specific needs. I chose the Go programming language for this project as it comes, in my opinion, with the best balance of ease to use and performance.</span><br /> -<br /> -<h2 style='display: inline'>Features</h2><br /> -<br /> -<ul> -<li>Compatible with Nagios Check scripts: Gogios leverages the widely-used Nagios Check API, allowing to use existing Nagios plugins.</li> -<li>Lightweight and Minimalistic: Gogios is designed to be simple and fairly easy to set up.</li> -<li>Configurable Check Timeout and Concurrency: Gogios allows you to set a timeout for checks and configure the number of concurrent checks, offering flexibility in monitoring your resources.</li> -<li>Configurable check dependency: A check can depend on another check, which enables scenarios like not executing an HTTP check when the server isn't pingable.</li> -<li>Retries: Check retry and retry intervals are configurable per check.</li> -<li>Email Notifications: Gogios can send email notifications regarding the status of monitored services, ensuring you stay informed about potential issues.</li> -<li>CRON-based Execution: Gogios can be quickly scheduled to run periodically via CRON, allowing you to automate monitoring without needing a complex setup.</li> -</ul><br /> -<h2 style='display: inline'>Example alert</h2><br /> -<br /> -<span>This is an example alert report received via E-Mail. Whereas, <span class='inlinecode'>[C:2 W:0 U:0 OK:51]</span> means that we've got two alerts in status critical, 0 warnings, 0 unknowns and 51 OKs.</span><br /> -<br /> -<pre> -Subject: GOGIOS Report [C:2 W:0 U:0 OK:51] - -This is the recent Gogios report! - -# Alerts with status changed: - -OK->CRITICAL: Check ICMP4 vulcan.buetow.org: Check command timed out -OK->CRITICAL: Check ICMP6 vulcan.buetow.org: Check command timed out - -# Unhandled alerts: - -CRITICAL: Check ICMP4 vulcan.buetow.org: Check command timed out -CRITICAL: Check ICMP6 vulcan.buetow.org: Check command timed out - -Have a nice day! -</pre> -<br /> -<h2 style='display: inline'>Installation</h2><br /> -<br /> -<h3 style='display: inline'>Compiling and installing Gogios</h3><br /> -<br /> -<span>This document is primarily written for OpenBSD, but applying the corresponding steps to any Unix-like (e.g. Linux-based) operating system should be easy. On systems other than OpenBSD, you may always have to replace <span class='inlinecode'>does</span> with the <span class='inlinecode'>sudo</span> command and replace the <span class='inlinecode'>/usr/local/bin</span> path with <span class='inlinecode'>/usr/bin</span>.</span><br /> -<br /> -<span>To compile and install Gogios on OpenBSD, follow these steps:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>git clone https<font color="#990000">:</font>//codeberg<font color="#990000">.</font>org/snonux/gogios<font color="#990000">.</font>git -cd gogios -go build -o gogios cmd/gogios/main<font color="#990000">.</font>go -doas cp gogios /usr/local/bin/gogios -doas chmod <font color="#993399">755</font> /usr/local/bin/gogios -</pre> -<br /> -<span>You can use cross-compilation if you want to compile Gogios for OpenBSD on a Linux system without installing the Go compiler on OpenBSD. Follow these steps:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">export</font></b> <font color="#009900">GOOS</font><font color="#990000">=</font>openbsd -<b><font color="#0000FF">export</font></b> <font color="#009900">GOARCH</font><font color="#990000">=</font>amd64 -go build -o gogios cmd/gogios/main<font color="#990000">.</font>go -</pre> -<br /> -<span>On your OpenBSD system, copy the binary to <span class='inlinecode'>/usr/local/bin/gogios</span> and set the correct permissions as described in the previous section. All steps described here you could automate with your configuration management system of choice. I use Rexify, the friendly configuration management system, to automate the installation, but that is out of the scope of this document.</span><br /> -<br /> -<a class='textlink' href='https://www.rexify.org'>https://www.rexify.org</a><br /> -<br /> -<h3 style='display: inline'>Setting up user, group and directories</h3><br /> -<br /> -<span>It is best to create a dedicated system user and group for Gogios to ensure proper isolation and security. Here are the steps to create the <span class='inlinecode'>_gogios</span> user and group under OpenBSD:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>doas adduser -group _gogios -batch _gogios -doas usermod -d /var/run/gogios _gogios -doas mkdir -p /var/run/gogios -doas chown _gogios<font color="#990000">:</font>_gogios /var/run/gogios -doas chmod <font color="#993399">750</font> /var/run/gogios -</pre> -<br /> -<span>Please note that creating a user and group might differ depending on your operating system. For other operating systems, consult their documentation for creating system users and groups.</span><br /> -<br /> -<h3 style='display: inline'>Installing monitoring plugins</h3><br /> -<br /> -<span>Gogios relies on external Nagios or Icinga monitoring plugin scripts. On OpenBSD, you can install the <span class='inlinecode'>monitoring-plugins</span> package with Gogios. The monitoring-plugins package is a collection of monitoring plugins, similar to Nagios plugins, that can be used to monitor various services and resources:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>doas pkg_add monitoring-plugins -doas pkg_add nrpe <i><font color="#9A1900"># If you want to execute checks remotely via NRPE.</font></i> -</pre> -<br /> -<span>Once the installation is complete, you can find the monitoring plugins in the <span class='inlinecode'>/usr/local/libexec/nagios</span> directory, which then can be configured to be used in <span class='inlinecode'>gogios.json</span>.</span><br /> -<br /> -<h2 style='display: inline'>Configuration</h2><br /> -<br /> -<h3 style='display: inline'>MTA</h3><br /> -<br /> -<span>Gogios requires a local Mail Transfer Agent (MTA) such as Postfix or OpenBSD SMTPD running on the same server where the CRON job (see about the CRON job further below) is executed. The local MTA handles email delivery, allowing Gogios to send email notifications to monitor status changes. Before using Gogios, ensure that you have a properly configured MTA installed and running on your server to facilitate the sending of emails. Once the MTA is set up and functioning correctly, Gogios can leverage it to send email notifications.</span><br /> -<br /> -<span>You can use the mail command to send an email via the command line on OpenBSD. Here's an example of how to send a test email to ensure that your email server is working correctly:</span><br /> -<br /> -<pre> -echo 'This is a test email from OpenBSD.' | mail -s 'Test Email' your-email@example.com -</pre> -<br /> -<span>Check the recipient's inbox to confirm the delivery of the test email. If the email is delivered successfully, it indicates that your email server is configured correctly and functioning. Please check your MTA logs in case of issues.</span><br /> -<br /> -<h3 style='display: inline'>Configuring Gogios</h3><br /> -<br /> -<span>To configure Gogios, create a JSON configuration file (e.g., <span class='inlinecode'>/etc/gogios.json</span>). Here's an example configuration:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><font color="#990000">{</font> - "EmailTo": "<font color="#FF0000">paul@dev.buetow.org</font>"<font color="#990000">,</font> - "EmailFrom": "<font color="#FF0000">gogios@buetow.org</font>"<font color="#990000">,</font> - "CheckTimeoutS": <font color="#993399">10</font><font color="#990000">,</font> - "CheckConcurrency": <font color="#993399">2</font><font color="#990000">,</font> - "StateDir": "<font color="#FF0000">/var/run/gogios</font>"<font color="#990000">,</font> - "Checks": <font color="#990000">{</font> - "Check ICMP4 www.foo.zone": <font color="#990000">{</font> - "Plugin": "<font color="#FF0000">/usr/local/libexec/nagios/check_ping</font>"<font color="#990000">,</font> - "Args": <font color="#990000">[</font> "-H"<font color="#990000">,</font> "www.foo.zone"<font color="#990000">,</font> "-4"<font color="#990000">,</font> "-w"<font color="#990000">,</font> "50,10%"<font color="#990000">,</font> "-c"<font color="#990000">,</font> "100,15%" <font color="#990000">],</font> - "Retries": <font color="#993399">3</font><font color="#990000">,</font> - "RetryInterval": <font color="#993399">10</font> - <font color="#990000">},</font> - "Check ICMP6 www.foo.zone": <font color="#990000">{</font> - "Plugin": "<font color="#FF0000">/usr/local/libexec/nagios/check_ping</font>"<font color="#990000">,</font> - "Args": <font color="#990000">[</font> "-H"<font color="#990000">,</font> "www.foo.zone"<font color="#990000">,</font> "-6"<font color="#990000">,</font> "-w"<font color="#990000">,</font> "50,10%"<font color="#990000">,</font> "-c"<font color="#990000">,</font> "100,15%" <font color="#990000">],</font> - "Retries": <font color="#993399">3</font><font color="#990000">,</font> - "RetryInterval": <font color="#993399">10</font> - <font color="#990000">},</font> - "www.foo.zone HTTP IPv4": <font color="#990000">{</font> - "Plugin": "<font color="#FF0000">/usr/local/libexec/nagios/check_http</font>"<font color="#990000">,</font> - "Args": <font color="#990000">[</font>"www.foo.zone"<font color="#990000">,</font> "-4"<font color="#990000">],</font> - "DependsOn": <font color="#990000">[</font>"Check ICMP4 www.foo.zone"<font color="#990000">]</font> - <font color="#990000">},</font> - "www.foo.zone HTTP IPv6": <font color="#990000">{</font> - "Plugin": "<font color="#FF0000">/usr/local/libexec/nagios/check_http</font>"<font color="#990000">,</font> - "Args": <font color="#990000">[</font>"www.foo.zone"<font color="#990000">,</font> "-6"<font color="#990000">],</font> - "DependsOn": <font color="#990000">[</font>"Check ICMP6 www.foo.zone"<font color="#990000">]</font> - <font color="#990000">}</font> - "Check NRPE Disk Usage foo.zone": <font color="#990000">{</font> - "Plugin": "<font color="#FF0000">/usr/local/libexec/nagios/check_nrpe</font>"<font color="#990000">,</font> - "Args": <font color="#990000">[</font>"-H"<font color="#990000">,</font> "foo.zone"<font color="#990000">,</font> "-c"<font color="#990000">,</font> "check_disk"<font color="#990000">,</font> "-p"<font color="#990000">,</font> "5666"<font color="#990000">,</font> "-4"<font color="#990000">]</font> - <font color="#990000">}</font> - <font color="#990000">}</font> -<font color="#990000">}</font> -</pre> -<br /> -<ul> -<li><span class='inlinecode'>EmailTo</span>: Specifies the recipient of the email notifications.</li> -<li><span class='inlinecode'>EmailFrom</span>: Indicates the sender's email address for email notifications.</li> -<li><span class='inlinecode'>CheckTimeoutS</span>: Sets the timeout for checks in seconds.</li> -<li><span class='inlinecode'>CheckConcurrency</span>: Determines the number of concurrent checks that can run simultaneously.</li> -<li><span class='inlinecode'>StateDir</span>: Specifies the directory where Gogios stores its persistent state in a <span class='inlinecode'>state.json</span> file. </li> -<li><span class='inlinecode'>Checks</span>: Defines a list of checks to be performed, each with a unique name, plugin path, and arguments.</li> -</ul><br /> -<span>Adjust the configuration file according to your needs, specifying the checks you want Gogios to perform.</span><br /> -<br /> -<span>If you want to execute checks only when another check succeeded (status OK), use <span class='inlinecode'>DependsOn</span>. In the example above, the HTTP checks won't run when the hosts aren't pingable. They will show up as <span class='inlinecode'>UNKNOWN</span> in the report.</span><br /> -<br /> -<span><span class='inlinecode'>Retries</span> and <span class='inlinecode'>RetryInterval</span> are optional check configuration parameters. In case of failure, Gogios will retry <span class='inlinecode'>Retries</span> times each <span class='inlinecode'>RetryInterval</span> seconds.</span><br /> -<br /> -<span>For remote checks, use the <span class='inlinecode'>check_nrpe</span> plugin. You also need to have the NRPE server set up correctly on the target host (out of scope for this document).</span><br /> -<br /> -<span>The <span class='inlinecode'>state.json</span> file mentioned above keeps track of the monitoring state and check results between Gogios runs, enabling Gogios only to send email notifications when there are changes in the check status.</span><br /> -<br /> -<h2 style='display: inline'>Running Gogios</h2><br /> -<br /> -<span>Now it is time to give it a first run. On OpenBSD, do:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>doas -u _gogios /usr/local/bin/gogios -cfg /etc/gogios<font color="#990000">.</font>json -</pre> -<br /> -<span>To run Gogios via CRON on OpenBSD as the <span class='inlinecode'>gogios</span> user and check all services once per minute, follow these steps:</span><br /> -<br /> -<span>Type <span class='inlinecode'>doas crontab -e -u _gogios</span> and press Enter to open the crontab file for the <span class='inlinecode'>_gogios</span> user for editing and add the following lines to the crontab file:</span><br /> -<br /> -<pre> -*/5 8-22 * * * /usr/local/bin/gogios -cfg /etc/gogios.json -0 7 * * * /usr/local/bin/gogios -renotify -cfg /etc/gogios.json -</pre> -<br /> -<span>Gogios is now configured to run every five minutes from 8 am to 10 pm via CRON as the <span class='inlinecode'>_gogios</span> user. It will execute the checks and send monitoring status whenever a check status changes via email according to your configuration. Also, Gogios will run once at 7 am every morning and re-notify all unhandled alerts as a reminder.</span><br /> -<br /> -<h3 style='display: inline'>High-availability</h3><br /> -<br /> -<span>To create a high-availability Gogios setup, you can install Gogios on two servers that will monitor each other using the NRPE (Nagios Remote Plugin Executor) plugin. By running Gogios in alternate CRON intervals on both servers, you can ensure that even if one server goes down, the other will continue monitoring your infrastructure and sending notifications.</span><br /> -<br /> -<ul> -<li>Install Gogios on both servers following the compilation and installation instructions provided earlier.</li> -<li>Install the NRPE server (out of scope for this document) and plugin on both servers. This plugin allows you to execute Nagios check scripts on remote hosts.</li> -<li>Configure Gogios on both servers to monitor each other using the NRPE plugin. Add a check to the Gogios configuration file (<span class='inlinecode'>/etc/gogios.json</span>) on both servers that uses the NRPE plugin to execute a check script on the other server. For example, if you have Server A and Server B, the configuration on Server A should include a check for Server B, and vice versa.</li> -<li>Set up alternate CRON intervals on both servers. Configure the CRON job on Server A to run Gogios at minutes 0, 10, 20, ..., and on Server B to run at minutes 5, 15, 25, ... This will ensure that if one server goes down, the other server will continue monitoring and sending notifications. </li> -<li>Gogios doesn't support clustering. So it means when both servers are up, unhandled alerts will be notified via E-Mail twice; from each server once. That's the trade-off for simplicity.</li> -</ul><br /> -<span>There are plans to make it possible to execute certain checks only on certain nodes (e.g. on elected leader or master nodes). This is still in progress (check out my Gorum Git project).</span><br /> -<br /> -<h2 style='display: inline'>Conclusion:</h2><br /> -<br /> -<span>Gogios is a lightweight and straightforward monitoring tool that is perfect for small-scale environments. With its compatibility with the Nagios Check API, email notifications, and CRON-based scheduling, Gogios offers an easy-to-use solution for those looking to monitor a limited number of resources. I personally use it to execute around 500 checks on my personal server infrastructure. I am very happy with this solution.</span><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>'The Obstacle is the Way' book notes</title> - <link href="gemini://foo.zone/gemfeed/2023-05-06-the-obstacle-is-the-way-book-notes.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-05-06-the-obstacle-is-the-way-book-notes.gmi</id> - <updated>2023-05-06T17:23:16+03:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>These are my personal takeaways after reading 'The Obstacle Is the Way' by Ryan Holiday. This is mainly for my own use, but you might find it helpful too.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>"The Obstacle is the Way" book notes</h1><br /> -<br /> -<span class='quote'>Published at 2023-05-06T17:23:16+03:00</span><br /> -<br /> -<span>These are my personal takeaways after reading "The Obstacle Is the Way" by Ryan Holiday. This is mainly for my own use, but you might find it helpful too.</span><br /> -<br /> -<br /> -<pre> - ,.......... .........., - ,..,' '.' ',.., - ,' ,' : ', ', - ,' ,' : ', ', - ,' ,' : ', ', - ,' ,'............., : ,.............', ', -,' '............ '.' ............' ', - '''''''''''''''''';''';'''''''''''''''''' - ''' -</pre> -<br /> -<span>"The obstacle is the way" is a powerful statement that encapsulates the wisdom of turning challenges into opportunities for growth and success. We will explore using obstacles as fuel, transforming weaknesses into strengths, and adopting a mindset that allows us to be creative and persistent in the face of adversity.</span><br /> -<br /> -<h2 style='display: inline'>Reframe your perspective</h2><br /> -<br /> -<span>The obstacle in your path can become your path to success. Instead of being paralyzed by challenges, see them as opportunities to learn and grow. Remember, the things that hurt us often instruct us. </span><br /> -<br /> -<span>We spend a lot of time trying to get things perfect and look at the rules, but what matters is that it works; it doesn't need to be after the book. Focus on results rather than on beautiful methods. In Jujitsu, it does matter that you bring your opponent down, but not how. There are many ways from point A to point B; it doesn't need to be a straight line. So many try to find the best solution but need to catch up on what is in Infront of them. Think progress and not perfection.</span><br /> -<br /> -<span>Don't always try to use the front door; a backdoor could open. It's nonsense. Don't fight the judo master with judo. Non-action can be action, exposing the weaknesses of others.</span><br /> -<br /> -<br /> -<h2 style='display: inline'>Embrace rationality</h2><br /> -<br /> -<span>It is a superpower to see things rationally when others are fearful. Focus on the reality of the situation without letting emotions, such as anger, cloud your judgment. This ability will enable you to make better decisions in adversity. Ability to see things what they really are. E.g. wine is old fermented grapes, or other people behaving like animals during a fight. Show the middle finger if someone persists on the stupid rules occasionally.</span><br /> -<br /> -<h2 style='display: inline'>Control your response</h2><br /> -<br /> -<span>You can choose how you respond to obstacles. Focus on what you can control, and don't let yourself feel harmed by external circumstances. Remember, you decide how things affect you; nobody else does. Choose to feel good in response to any situation. Embrace the challenges and obstacles that come your way, as they are opportunities for growth and learning.</span><br /> -<br /> -<h2 style='display: inline'>Practice emotional and physical resilience</h2><br /> -<br /> -<span>Martial artists know the importance of developing physical and emotional strength. Cultivate the art of not panicking; it will help you avoid making mistakes during high-pressure situations.</span><br /> -<br /> -<span>Focus on what you can control. Don't choose to feel harmed, and then you won't be harmed. I decide things that affect me; nobody else does. E.g., in prison, your mind stays your own. Don't ignore fear but explain it away, have a different view.</span><br /> -<br /> -<h2 style='display: inline'>Persistence and patience</h2><br /> -<br /> -<span>Practice persistence and patience in your pursuits. Focus on the process rather than the prize and take one step at a time. Remember, the journey is about finishing tasks, projects, or workouts to the best of your ability. Never be in a hurry and never be desperate. There is no reason to be rushed; there are all in the long haul. Follow the process and not the price. Take it one step at a time. The process is about finishing (workout, task, project, etc.).</span><br /> -<br /> -<h2 style='display: inline'>Embrace failure</h2><br /> -<br /> -<span>Failure is a natural part of life and can make us stronger. Treat defeat as a stepping stone to success and education. What is defeat? The first step to education. Failure makes you stronger. If we do our best, we can be proud of it, regardless of the result. Do your job, but do it right. Only an asshole thinks he is too good at the things he does. Also, asking for forgiveness is easier than asking for permission. </span><br /> -<br /> -<h2 style='display: inline'>Be adaptable</h2><br /> -<br /> -<span>There are many ways to achieve your goals; sometimes, unconventional methods are necessary. Feel free to break the rules or go off the beaten path if it will lead to better results. Transform weaknesses into strengths. We have a choice of how to respond to things. It's not about being positive but to be creative. Aim high, but stuff will happen; E.g., surprises will always happen.</span><br /> -<br /> -<h2 style='display: inline'>Embrace non-action</h2><br /> -<br /> -<span>We constantly push to the next thing. Sometimes the best course of action is standing still or even going backwards. Obstacles might resolve by themselves. Or going sideways. Sometimes, the best action is to stand still, go sideways, or even go backwards. Obstacles may resolve themselves or present new opportunities if you're patient and observant. People always want your input before you have all the facts. They want you to play after their rules. The question is, do you let them? The English call it the cool head. Being in control of Stress; requires practice. Appear, the absence of fear (Greek). When all others do it one way, it does not mean it is the correct or best practice.</span><br /> -<br /> -<h2 style='display: inline'>Leverage crisis</h2><br /> -<br /> -<span>In times of crisis, seize the chance to do things never done before. Great people use negative situations to their advantage and become the most effective in challenging circumstances.</span><br /> -<br /> -<span>The art of not panicking; otherwise, you will make mistakes. When overs are shocked, you know which way to take due to your thinking of the problem at Hand. A crisis gives you a chance to do things which never done before. Ordinary people shy from negative situations; great people use these for their benefit and are the most effective. The obstacle is not just turned upside down but used as a catapult.</span><br /> -<br /> -<span>Be prepared for nothing to work. Problems are an opportunity to do your best, not to do miracles. Always manage your expectations. It will suck, but it will be ok. Be prepared to begin from the beginning. Be cheerful and eagerly work on the next obstacle. Each time you become better. Life is not a sprint but a marathon. After each obstacle lies another obstacle, there won't be anything without obstacles. Passing one means you are ready for the next.</span><br /> -<br /> -<h2 style='display: inline'>Build your inner citadel</h2><br /> -<br /> -<span>Develop your inner strength during good times so you can rely on it in bad times. Always prepare for adversity and face it with calmness and resilience. Be humble enough that things which happen will happen. Build your inner citadel. In good times strengthen it. In bad times rely on it.</span><br /> -<br /> -<span>We should always prepare for things to get tough. Your house burns down: no worries, we eliminated much rubbish. Imagine what can go wrong before things go wrong. We are prepared for adversity; it's other people who aren't. Phil Jackson's hip problem example. To receive unexpected benefits, you must first accept the unexpected obstacles. Meditate on death. It's a universal obstacle. Use it as a reminder to do your best.</span><br /> -<br /> -<h2 style='display: inline'>Love everything that happens</h2><br /> -<br /> -<span>Turn an obstacle the other way around for your benefit. Use it at fuel. It's simple but challenging. Most are paralyzed instead. The obstacle in the path becomes the path. Obstacles are neither good nor bad. The things which hurt, instruct.</span><br /> -<br /> -<span>Should I hate people who hate me? That's their problem and not mine. Be always calm and relaxed during the fight. The story of the battle is the story of the smile. Cheerfulness in all situations, especially the bad ones. Love for everything that happens; if it happens, it was meant to happen. We can choose how we react to things, so why not choose to feel good? I love everything that happens. You must never lower yourself to the person you don't like.</span><br /> -<br /> -<h2 style='display: inline'>Conclusion</h2><br /> -<br /> -<span>Life is a marathon, not a sprint. Each obstacle we overcome prepares us for the next one. Remember, the obstacle is not just a barrier to be turned upside down; it can also be used as a catapult to propel us forward. By embracing challenges and using them as opportunities for growth, we become stronger, more adaptable, and, ultimately, more successful.</span><br /> -<br /> -<span>Other book notes of mine are:</span><br /> -<br /> -<a class='textlink' href='./2023-03-16-the-pragmatic-programmer-book-notes.html'>2023-03-16 "The Pragmatic Programmer" book notes</a><br /> -<a class='textlink' href='./2023-04-01-never-split-the-difference-book-notes.html'>2023-04-01 "Never split the difference" book notes</a><br /> -<a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes (You are currently reading this)</a><br /> -<a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../resources.html'>More books and other resources I found useful.</a><br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>Unveiling `guprecords.raku`: Global Uptime Records with Raku</title> - <link href="gemini://foo.zone/gemfeed/2023-05-01-unveiling-guprecords:-uptime-records-with-raku.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-05-01-unveiling-guprecords:-uptime-records-with-raku.gmi</id> - <updated>2023-04-30T13:10:26+03:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>For fun, I am tracking the uptime of various personal machines (servers, laptops, workstations...). I have been doing this for over ten years now, so I have a lot of statistics collected.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Unveiling <span class='inlinecode'>guprecords.raku</span>: Global Uptime Records with Raku</h1><br /> -<br /> -<span class='quote'>Published at 2023-04-30T13:10:26+03:00</span><br /> -<br /> -<pre> -+-----+-----------------+-----------------------------+ -| Pos | Host | Lifespan | -+-----+-----------------+-----------------------------+ -| 1. | dionysus | 8 years, 6 months, 17 days | -| 2. | uranus | 7 years, 2 months, 16 days | -| 3. | alphacentauri | 6 years, 9 months, 13 days | -| 4. | *vulcan | 4 years, 5 months, 6 days | -| 5. | sun | 3 years, 10 months, 2 days | -| 6. | uugrn | 3 years, 5 months, 5 days | -| 7. | deltavega | 3 years, 1 months, 21 days | -| 8. | pluto | 2 years, 10 months, 30 days | -| 9. | tauceti | 2 years, 3 months, 22 days | -| 10. | callisto | 2 years, 3 months, 13 days | -+-----+-----------------+-----------------------------+ -</pre> -<br /> -<h1 style='display: inline'>Introduction</h1><br /> -<br /> -<span>For fun, I am tracking the uptime of various personal machines (servers, laptops, workstations...). I have been doing this for over ten years now, so I have a lot of statistics collected.</span><br /> -<br /> -<span>As a result of this, I am introducing <span class='inlinecode'>guprecords.raku</span>, a handy Raku script that helps me combine uptime statistics from multiple servers into one comprehensive report. In this blog post, I'll explore what Guprecords is and some examples of its application. I will also add some notes on Raku.</span><br /> -<br /> -<span>Guprecords, or global uptime records, is a Raku script designed to generate a consolidated uptime report from multiple hosts:</span><br /> -<br /> -<a class='textlink' href='https://codeberg.org/snonux/guprecords'>https://codeberg.org/snonux/guprecords</a><br /> -<a class='textlink' href='https://raku.org'>The Raku Programming Language</a><br /> -<br /> -<span>A previous version of Guprecords was actually written in Perl, the older and more established language from which Raku was developed. One of the primary motivations for rewriting Guprecords in Raku was to learn the language and explore its features. Raku is a more modern and powerful language compared to Perl, and working on a real-world project like Guprecords provided a practical and engaging way to learn the language.</span><br /> -<br /> -<span>Over the last years, I have been reading the following books and resources about Raku:</span><br /> -<br /> -<ul> -<li>Raku Guide (at raku.guide)</li> -<li>Think Perl 6</li> -<li>Raku Fundamentals</li> -<li>Raku Recipes</li> -</ul><br /> -<span>And I have been following the Raku newsletter, and sometimes I have been lurking around in the IRC channels, too. Watching Raku coding challenges on YouTube was pretty fun, too. However, nothing beats actually using Raku to learn the language. After reading all of these resources, I may have a good idea about the features and paradigms, but I am by far not an expert.</span><br /> -<br /> -<h2 style='display: inline'>How Guprecords works</h2><br /> -<br /> -<span>Guprecords works in three stages:</span><br /> -<br /> -<ul> -<li>1. Generating uptime statistics using <span class='inlinecode'>uptimed</span>: First, I need to install and run <span class='inlinecode'>uptimed</span> on each host to generate uptime statistics. This tool is available for most common Linux and *BSD distributions and macOS via Homebrew.</li> -<li>2. Collecting uptime records to a central location: The next step involves collecting the raw uptime statistics files generated by <span class='inlinecode'>uptimed</span> on each host. It's a good idea to store all record files in a central git repository. The records file contains information about the total uptime since boot, boot time, and the operating system and kernel version. Guprecords itself does not do the collection part, but have a look at the <span class='inlinecode'>README.md</span> in the git repository for some guidance.</li> -<li>3. Generating global uptime stats: Finally, run the <span class='inlinecode'>guprecords.raku</span> script with the appropriate flags to create a global uptime report. For example, I can use the following command:</li> -</ul><br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>$ raku guprecords<font color="#990000">.</font>raku --stats<font color="#990000">=</font><font color="#009900">dir</font><font color="#990000">=</font><font color="#009900">$HOME</font>/git/uprecords/stats --all -</pre> -<br /> -<span>This command will generate a comprehensive uptime report from the collected statistics, making it easy to review and enjoy the data.</span><br /> -<br /> -<span>Guprecords supports the following features:</span><br /> -<br /> -<ul> -<li>Supports multiple categories: Host, Kernel, KernelMajor, and KernelName</li> -<li>Supports multiple metrics: Boots, Uptime, Score, Downtime, and Lifespan</li> -<li>Output formats available: Plaintext, Markdown, and Gemtext</li> -<li>Provides top entries based on the specified limit</li> -</ul><br /> -<h2 style='display: inline'>Example</h2><br /> -<br /> -<span>You have already seen an example at the very top of this post, where the hosts were grouped by their total lifespans (uptime+downtime). Here's an example of what the global uptime report (grouped by total host uptimes) might look like:</span><br /> -<br /> -<pre> -Top 20 Uptime's by Host - -+-----+-----------------+-----------------------------+ -| Pos | Host | Uptime | -+-----+-----------------+-----------------------------+ -| 1. | *vulcan | 4 years, 5 months, 6 days | -| 2. | uranus | 3 years, 11 months, 21 days | -| 3. | sun | 3 years, 9 months, 26 days | -| 4. | uugrn | 3 years, 5 months, 5 days | -| 5. | deltavega | 3 years, 1 months, 21 days | -| 6. | pluto | 2 years, 10 months, 29 days | -| 7. | tauceti | 2 years, 3 months, 19 days | -| 8. | tauceti-f | 1 years, 9 months, 18 days | -| 9. | *ultramega15289 | 1 years, 8 months, 17 days | -| 10. | *earth | 1 years, 5 months, 22 days | -| 11. | *blowfish | 1 years, 4 months, 20 days | -| 12. | ultramega8477 | 1 years, 3 months, 25 days | -| 13. | host0 | 1 years, 3 months, 9 days | -| 14. | tauceti-e | 1 years, 2 months, 20 days | -| 15. | makemake | 1 years, 1 months, 6 days | -| 16. | callisto | 0 years, 10 months, 31 days | -| 17. | alphacentauri | 0 years, 10 months, 28 days | -| 18. | london | 0 years, 9 months, 16 days | -| 19. | twofish | 0 years, 8 months, 31 days | -| 20. | *fishfinger | 0 years, 8 months, 17 days | -+-----+-----------------+-----------------------------+ -</pre> -<br /> -<span>This table ranks the top 20 hosts based on their total uptime, with the host having the highest uptime at the top. The hosts marked with <span class='inlinecode'>*</span> are still active, means stats were collected within the last couple of months. </span><br /> -<br /> -<span>My up to date stats can be seen here:</span><br /> -<br /> -<a class='textlink' href='../uptime-stats.html'>My machine uptime stats</a><br /> -<br /> -<span>Just recently, I decommissioned <span class='inlinecode'>vulcan</span> (the number one stop from above), which used to be my CentOS 7 (initially CentOS 6) VM hosting my personal NextCloud and Wallabag (which I modernised just recently with a brand new shiny Rocky Linux 9 VM). This was the last <span class='inlinecode'>uptimed</span> output before shutting it down (it always makes me feel sentimental decommissioning one of my machines <span class='inlinecode'>:'-(</span>):</span><br /> -<br /> -<pre> - # Uptime | System Boot up -----------------------------+--------------------------------------------------- - 1 545 days, 17:58:15 | Linux 3.10.0-1160.15.2.e Sun Jul 25 19:32:25 2021 - 2 279 days, 10:12:14 | Linux 3.10.0-957.21.3.el Sun Jun 30 12:43:41 2019 - 3 161 days, 06:08:43 | Linux 3.10.0-1160.15.2.e Sun Feb 14 11:05:38 2021 - 4 107 days, 01:26:35 | Linux 3.10.0-957.1.3.el7 Thu Dec 20 09:29:13 2018 - 5 96 days, 21:13:49 | Linux 3.10.0-1127.13.1.e Sat Jul 25 17:56:22 2020 --> 6 89 days, 23:05:32 | Linux 3.10.0-1160.81.1.e Sun Jan 22 12:39:36 2023 - 7 63 days, 18:30:45 | Linux 3.10.0-957.10.1.el Sat Apr 27 18:12:43 2019 - 8 63 days, 06:53:33 | Linux 3.10.0-1127.8.2.el Sat May 23 10:41:08 2020 - 9 48 days, 11:44:49 | Linux 3.10.0-1062.18.1.e Sat Apr 4 22:56:07 2020 - 10 42 days, 08:00:13 | Linux 3.10.0-1127.19.1.e Sat Nov 7 11:47:33 2020 - 11 36 days, 22:57:19 | Linux 3.10.0-1160.6.1.el Sat Dec 19 19:47:57 2020 - 12 21 days, 06:16:28 | Linux 3.10.0-957.10.1.el Sat Apr 6 11:56:01 2019 - 13 12 days, 20:11:53 | Linux 3.10.0-1160.11.1.e Mon Jan 25 18:45:27 2021 - 14 7 days, 21:29:18 | Linux 3.10.0-1127.13.1.e Fri Oct 30 14:18:04 2020 - 15 6 days, 20:07:18 | Linux 3.10.0-1160.15.2.e Sun Feb 7 14:57:35 2021 - 16 1 day , 21:46:41 | Linux 3.10.0-957.1.3.el7 Tue Dec 18 11:42:19 2018 - 17 0 days, 01:25:57 | Linux 3.10.0-957.1.3.el7 Tue Dec 18 10:16:08 2018 - 18 0 days, 00:42:34 | Linux 3.10.0-1160.15.2.e Sun Jul 25 18:49:38 2021 - 19 0 days, 00:08:32 | Linux 3.10.0-1160.81.1.e Sun Jan 22 12:30:52 2023 -----------------------------+--------------------------------------------------- -1up in 6 days, 22:08:18 | at Sat Apr 29 10:53:25 2023 -no1 in 455 days, 18:52:44 | at Sun Jul 21 07:37:51 2024 - up 1586 days, 00:20:28 | since Tue Dec 18 10:16:08 2018 - down 0 days, 01:08:32 | since Tue Dec 18 10:16:08 2018 - %up 99.997 | since Tue Dec 18 10:16:08 2018 -</pre> -<br /> -<h2 style='display: inline'>Conclusion</h2><br /> -<br /> -<span>Guprecords is a small, yet powerful tool for analyzing uptime statistics. While developing Guprecords, I have come to truly appreciate and love Raku's expressiveness. The language is designed to be both powerful and flexible, allowing developers to express their intentions and logic more clearly and concisely.</span><br /> -<br /> -<span>Raku's expressive syntax, support for multiple programming paradigms, and unique features, such as grammars and lazy evaluation, make it a joy to work with. </span><br /> -<br /> -<span>Working on Guprecords in Raku has been an enjoyable experience, and I've found that Raku's expressiveness has significantly contributed to the overall quality and effectiveness of the script. The language's ability to elegantly express complex logic and data manipulation tasks makes it an excellent choice for developing tools like these, where expressiveness and productiveness are of the utmost importance.</span><br /> -<br /> -<span>So far, I have only scratched the surface of what Raku can do. I hope to find more time to become a regular Rakoon (a Raku Programmer). I have many Ideas for other small tools like Guprecords, but the challenge is finding the time. I'd love to explore Raku Grammars and also I would love to explore writing concurrent code in Raku (I also love Go (Golang), btw!). Ideas for future Raku personal projects include:</span><br /> -<br /> -<ul> -<li>A log file analyzer, for generating anonymized <span class='inlinecode'>foo.zone</span> visitor stats for both, the Web and Gemini.</li> -<li>A social media sharing scheduler a la <span class='inlinecode'>buffer.com</span>. I am using Buffer at the moment to share posts on Mastadon, Twitter, Telegram and LinkedIn, but it is proprietary and also it's not really reliable.</li> -<li>Rewrite the static photo album generator of <span class='inlinecode'>irregular.ninja</span> in Raku (from Bash).</li> -</ul><br /> -<span>Other related posts are:</span><br /> -<br /> -<a class='textlink' href='./2008-06-26-perl-poetry.html'>2008-06-26 Perl Poetry</a><br /> -<a class='textlink' href='./2011-05-07-perl-daemon-service-framework.html'>2011-05-07 Perl Daemon (Service Framework)</a><br /> -<a class='textlink' href='./2022-05-27-perl-is-still-a-great-choice.html'>2022-05-27 Perl is still a great choice</a><br /> -<a class='textlink' href='./2022-06-15-sweating-the-small-stuff.html'>2022-06-15 Sweating the small stuff - Tiny projects of mine</a><br /> -<a class='textlink' href='./2023-05-01-unveiling-guprecords:-uptime-records-with-raku.html'>2023-05-01 Unveiling <span class='inlinecode'>guprecords.raku</span>: Global Uptime Records with Raku (You are currently reading this)</a><br /> -<br /> -<span>E-Mail your comments to hi@foo.zone :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>Algorithms and Data Structures in Go - Part 1</title> - <link href="gemini://foo.zone/gemfeed/2023-04-09-algorithms-and-data-structures-in-golang-part-1.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-04-09-algorithms-and-data-structures-in-golang-part-1.gmi</id> - <updated>2023-04-09T22:31:42+03:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>This is the first blog post about my Algorithms and Data Structures in Go series. I am not a Software Developer in my day job. In my current role, programming and scripting skills are desirable but not mandatory. I have been learning about Data Structures and Algorithms many years ago at University. I thought it would be fun to revisit/refresh my knowledge here and implement many of the algorithms in Go.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Algorithms and Data Structures in Go - Part 1</h1><br /> -<br /> -<span class='quote'>Published at 2023-04-09T22:31:42+03:00</span><br /> -<br /> -<pre> - ,_---~~~~~----._ - _,,_,*^____ _____``*g*\"*, - / __/ /' ^. / \ ^@q f -[ @f | @)) | | @)) l 0 _/ - \`/ \~____ / __ \_____/ \ - | _l__l_ I - } [______] I - ] | | | | - ] ~ ~ | - | | - | | -</pre> -<br /> -<span>This is the first blog post about my Algorithms and Data Structures in Go series. I am not a Software Developer in my day job. In my current role, programming and scripting skills are desirable but not mandatory. I have been learning about Data Structures and Algorithms many years ago at University. I thought it would be fun to revisit/refresh my knowledge here and implement many of the algorithms in Go.</span><br /> -<br /> -<a class='textlink' href='./2023-04-09-algorithms-and-data-structures-in-golang-part-1.html'>2023-04-09 Algorithms and Data Structures in Go - Part 1 (You are currently reading this)</a><br /> -<br /> -<span>This post is about setting up some basic data structures and methods for this blog series. I promise, everything will be easy to follow in this post. It will become more interesting later in this series.</span><br /> -<br /> -<h2 style='display: inline'>Type constraints</h2><br /> -<br /> -<span>First, the package <span class='inlinecode'>ds</span> (data structures) defines the <span class='inlinecode'>types.go</span>. All examples will either operate on the <span class='inlinecode'>Integer</span> or <span class='inlinecode'>Number</span> type:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">package</font></b> ds - -<b><font color="#0000FF">import</font></b> <font color="#990000">(</font> - <font color="#FF0000">"golang.org/x/exp/constraints"</font> -<font color="#990000">)</font> - -<b><font color="#0000FF">type</font></b> Integer <b><font color="#0000FF">interface</font></b> <font color="#FF0000">{</font> - constraints<font color="#990000">.</font>Integer -<font color="#FF0000">}</font> - -<b><font color="#0000FF">type</font></b> Number <b><font color="#0000FF">interface</font></b> <font color="#FF0000">{</font> - constraints<font color="#990000">.</font>Integer <font color="#990000">|</font> constraints<font color="#990000">.</font>Float -<font color="#FF0000">}</font> - -</pre> -<br /> -<h2 style='display: inline'>ArrayList</h2><br /> -<br /> -<span>Next comes the <span class='inlinecode'>arraylist.go</span>, which defines the underlying data structure all the algorithms of this series will use. <span class='inlinecode'>ArrayList</span> is just a type alias of a Go array (or slice) with custom methods on it:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">package</font></b> ds - -<b><font color="#0000FF">import</font></b> <font color="#990000">(</font> - <font color="#FF0000">"fmt"</font> - <font color="#FF0000">"math/rand"</font> - <font color="#FF0000">"strings"</font> -<font color="#990000">)</font> - -<b><font color="#0000FF">type</font></b> ArrayList<font color="#990000">[</font>V Number<font color="#990000">]</font> <font color="#990000">[]</font>V - -<b><font color="#0000FF">func</font></b> NewArrayList<font color="#990000">[</font>V Number<font color="#990000">](</font>l int<font color="#990000">)</font> ArrayList<font color="#990000">[</font>V<font color="#990000">]</font> <font color="#FF0000">{</font> - <b><font color="#0000FF">return</font></b> <b><font color="#000000">make</font></b><font color="#990000">(</font>ArrayList<font color="#990000">[</font>V<font color="#990000">],</font> l<font color="#990000">)</font> -<font color="#FF0000">}</font> -</pre> -<br /> -<span>As you can see, the code uses Go generics, which I refactored recently. Besides the default constructor (which only returns an empty <span class='inlinecode'>ArrayList</span> with a given capacity), there are also a bunch of special constructors. <span class='inlinecode'>NewRandomArrayList</span> is returning an <span class='inlinecode'>ArrayList</span> with random numbers, <span class='inlinecode'>NewAscendingArrayList</span> and <span class='inlinecode'>NewDescendingArrayList</span> are returning <span class='inlinecode'>ArrayList</span>s in either ascending or descending order. They all will be used later on for testing and benchmarking the algorithms.</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">func</font></b> NewRandomArrayList<font color="#990000">[</font>V Number<font color="#990000">](</font>l<font color="#990000">,</font> max int<font color="#990000">)</font> ArrayList<font color="#990000">[</font>V<font color="#990000">]</font> <font color="#FF0000">{</font> - a <font color="#990000">:=</font> <b><font color="#000000">make</font></b><font color="#990000">(</font>ArrayList<font color="#990000">[</font>V<font color="#990000">],</font> l<font color="#990000">)</font> - <b><font color="#0000FF">for</font></b> i <font color="#990000">:=</font> <font color="#993399">0</font><font color="#990000">;</font> i <font color="#990000"><</font> l<font color="#990000">;</font> i<font color="#990000">++</font> <font color="#FF0000">{</font> - <b><font color="#0000FF">if</font></b> max <font color="#990000">></font> <font color="#993399">0</font> <font color="#FF0000">{</font> - a<font color="#990000">[</font>i<font color="#990000">]</font> <font color="#990000">=</font> <b><font color="#000000">V</font></b><font color="#990000">(</font>rand<font color="#990000">.</font><b><font color="#000000">Intn</font></b><font color="#990000">(</font>max<font color="#990000">))</font> - <b><font color="#0000FF">continue</font></b> - <font color="#FF0000">}</font> - a<font color="#990000">[</font>i<font color="#990000">]</font> <font color="#990000">=</font> <b><font color="#000000">V</font></b><font color="#990000">(</font>rand<font color="#990000">.</font><b><font color="#000000">Int</font></b><font color="#990000">())</font> - <font color="#FF0000">}</font> - <b><font color="#0000FF">return</font></b> a -<font color="#FF0000">}</font> - -<b><font color="#0000FF">func</font></b> NewAscendingArrayList<font color="#990000">[</font>V Number<font color="#990000">](</font>l int<font color="#990000">)</font> ArrayList<font color="#990000">[</font>V<font color="#990000">]</font> <font color="#FF0000">{</font> - a <font color="#990000">:=</font> <b><font color="#000000">make</font></b><font color="#990000">(</font>ArrayList<font color="#990000">[</font>V<font color="#990000">],</font> l<font color="#990000">)</font> - <b><font color="#0000FF">for</font></b> i <font color="#990000">:=</font> <font color="#993399">0</font><font color="#990000">;</font> i <font color="#990000"><</font> l<font color="#990000">;</font> i<font color="#990000">++</font> <font color="#FF0000">{</font> - a<font color="#990000">[</font>i<font color="#990000">]</font> <font color="#990000">=</font> <b><font color="#000000">V</font></b><font color="#990000">(</font>i<font color="#990000">)</font> - <font color="#FF0000">}</font> - <b><font color="#0000FF">return</font></b> a -<font color="#FF0000">}</font> - -<b><font color="#0000FF">func</font></b> NewDescendingArrayList<font color="#990000">[</font>V Number<font color="#990000">](</font>l int<font color="#990000">)</font> ArrayList<font color="#990000">[</font>V<font color="#990000">]</font> <font color="#FF0000">{</font> - a <font color="#990000">:=</font> <b><font color="#000000">make</font></b><font color="#990000">(</font>ArrayList<font color="#990000">[</font>V<font color="#990000">],</font> l<font color="#990000">)</font> - j <font color="#990000">:=</font> l <font color="#990000">-</font> <font color="#993399">1</font> - <b><font color="#0000FF">for</font></b> i <font color="#990000">:=</font> <font color="#993399">0</font><font color="#990000">;</font> i <font color="#990000"><</font> l<font color="#990000">;</font> i<font color="#990000">++</font> <font color="#FF0000">{</font> - a<font color="#990000">[</font>i<font color="#990000">]</font> <font color="#990000">=</font> <b><font color="#000000">V</font></b><font color="#990000">(</font>j<font color="#990000">)</font> - j<font color="#990000">--</font> - <font color="#FF0000">}</font> - <b><font color="#0000FF">return</font></b> a -<font color="#FF0000">}</font> -</pre> -<br /> -<h2 style='display: inline'>Helper methods</h2><br /> -<br /> -<span>The <span class='inlinecode'>FirstN</span> method only returns the first N elements of the <span class='inlinecode'>ArrayList</span>. This is useful for printing out only parts of the data structure:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">func</font></b> <font color="#990000">(</font>a ArrayList<font color="#990000">[</font>V<font color="#990000">])</font> <b><font color="#000000">FirstN</font></b><font color="#990000">(</font>n int<font color="#990000">)</font> <font color="#009900">string</font> <font color="#FF0000">{</font> - <b><font color="#0000FF">var</font></b> sb strings<font color="#990000">.</font>Builder - j <font color="#990000">:=</font> n - - l <font color="#990000">:=</font> <b><font color="#000000">len</font></b><font color="#990000">(</font>a<font color="#990000">)</font> - <b><font color="#0000FF">if</font></b> j <font color="#990000">></font> l <font color="#FF0000">{</font> - j <font color="#990000">=</font> l - <font color="#FF0000">}</font> - - <b><font color="#0000FF">for</font></b> i <font color="#990000">:=</font> <font color="#993399">0</font><font color="#990000">;</font> i <font color="#990000"><</font> j<font color="#990000">;</font> i<font color="#990000">++</font> <font color="#FF0000">{</font> - fmt<font color="#990000">.</font><b><font color="#000000">Fprintf</font></b><font color="#990000">(&</font>sb<font color="#990000">,</font> <font color="#FF0000">"%v "</font><font color="#990000">,</font> a<font color="#990000">[</font>i<font color="#990000">])</font> - <font color="#FF0000">}</font> - - <b><font color="#0000FF">if</font></b> j <font color="#990000"><</font> l <font color="#FF0000">{</font> - fmt<font color="#990000">.</font><b><font color="#000000">Fprintf</font></b><font color="#990000">(&</font>sb<font color="#990000">,</font> <font color="#FF0000">"... "</font><font color="#990000">)</font> - <font color="#FF0000">}</font> - - <b><font color="#0000FF">return</font></b> sb<font color="#990000">.</font><b><font color="#000000">String</font></b><font color="#990000">()</font> -<font color="#FF0000">}</font> -</pre> -<br /> -<span>The <span class='inlinecode'>Sorted</span> method checks whether the <span class='inlinecode'>ArrayList</span> is sorted. This will be used by the unit tests later on:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">func</font></b> <font color="#990000">(</font>a ArrayList<font color="#990000">[</font>V<font color="#990000">])</font> <b><font color="#000000">Sorted</font></b><font color="#990000">()</font> <font color="#009900">bool</font> <font color="#FF0000">{</font> - <b><font color="#0000FF">for</font></b> i <font color="#990000">:=</font> <b><font color="#000000">len</font></b><font color="#990000">(</font>a<font color="#990000">)</font> <font color="#990000">-</font> <font color="#993399">1</font><font color="#990000">;</font> i <font color="#990000">></font> <font color="#993399">0</font><font color="#990000">;</font> i<font color="#990000">--</font> <font color="#FF0000">{</font> - <b><font color="#0000FF">if</font></b> a<font color="#990000">[</font>i<font color="#990000">]</font> <font color="#990000"><</font> a<font color="#990000">[</font>i<font color="#990000">-</font><font color="#993399">1</font><font color="#990000">]</font> <font color="#FF0000">{</font> - <b><font color="#0000FF">return</font></b> false - <font color="#FF0000">}</font> - <font color="#FF0000">}</font> - <b><font color="#0000FF">return</font></b> true -<font color="#FF0000">}</font> -</pre> -<br /> -<span>And the last utility method used is <span class='inlinecode'>Swap</span>, which allows swapping the values of two indices in the <span class='inlinecode'>ArrayList</span>:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">func</font></b> <font color="#990000">(</font>a ArrayList<font color="#990000">[</font>V<font color="#990000">])</font> <b><font color="#000000">Swap</font></b><font color="#990000">(</font>i<font color="#990000">,</font> j int<font color="#990000">)</font> <font color="#FF0000">{</font> - aux <font color="#990000">:=</font> a<font color="#990000">[</font>i<font color="#990000">]</font> - a<font color="#990000">[</font>i<font color="#990000">]</font> <font color="#990000">=</font> a<font color="#990000">[</font>j<font color="#990000">]</font> - a<font color="#990000">[</font>j<font color="#990000">]</font> <font color="#990000">=</font> aux -<font color="#FF0000">}</font> - -</pre> -<br /> -<h2 style='display: inline'>Sleep sort</h2><br /> -<br /> -<span>Let's implement our first algorithm, sleep sort. Sleep sort is a non-traditional and unconventional sorting algorithm based on the idea of waiting a certain amount of time corresponding to the value of each element in the input <span class='inlinecode'>ArrayList</span>. It's more of a fun, creative concept rather than an efficient or practical sorting technique. This is not a sorting algorithm you would use in any production code. As you can imagine, it is quite an inefficient sorting algorithm (it's only listed here as a warm-up exercise). This sorting method may also return false results depending on how the Goroutines are scheduled by the Go runtime. </span><br /> -<br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">package</font></b> sort - -<b><font color="#0000FF">import</font></b> <font color="#990000">(</font> - <font color="#FF0000">"codeberg.org/snonux/algorithms/ds"</font> - <font color="#FF0000">"sync"</font> - <font color="#FF0000">"time"</font> -<font color="#990000">)</font> - -<b><font color="#0000FF">func</font></b> Sleep<font color="#990000">[</font>V ds<font color="#990000">.</font>Integer<font color="#990000">](</font>a ds<font color="#990000">.</font>ArrayList<font color="#990000">[</font>V<font color="#990000">])</font> ds<font color="#990000">.</font>ArrayList<font color="#990000">[</font>V<font color="#990000">]</font> <font color="#FF0000">{</font> - sorted <font color="#990000">:=</font> ds<font color="#990000">.</font>NewArrayList<font color="#990000">[</font>V<font color="#990000">](</font><b><font color="#000000">len</font></b><font color="#990000">(</font>a<font color="#990000">))</font> - - numCh <font color="#990000">:=</font> <b><font color="#000000">make</font></b><font color="#990000">(</font><b><font color="#0000FF">chan</font></b> V<font color="#990000">)</font> - <b><font color="#0000FF">var</font></b> wg sync<font color="#990000">.</font>WaitGroup - wg<font color="#990000">.</font><b><font color="#000000">Add</font></b><font color="#990000">(</font><b><font color="#000000">len</font></b><font color="#990000">(</font>a<font color="#990000">))</font> - - <b><font color="#0000FF">go</font></b> <b><font color="#0000FF">func</font></b><font color="#990000">()</font> <font color="#FF0000">{</font> - wg<font color="#990000">.</font><b><font color="#000000">Wait</font></b><font color="#990000">()</font> - <b><font color="#000000">close</font></b><font color="#990000">(</font>numCh<font color="#990000">)</font> - <font color="#FF0000">}</font><font color="#990000">()</font> - - <b><font color="#0000FF">for</font></b> _<font color="#990000">,</font> num <font color="#990000">:=</font> <b><font color="#0000FF">range</font></b> a <font color="#FF0000">{</font> - <b><font color="#0000FF">go</font></b> <b><font color="#0000FF">func</font></b><font color="#990000">(</font>num V<font color="#990000">)</font> <font color="#FF0000">{</font> - <b><font color="#0000FF">defer</font></b> wg<font color="#990000">.</font><b><font color="#000000">Done</font></b><font color="#990000">()</font> - time<font color="#990000">.</font><b><font color="#000000">Sleep</font></b><font color="#990000">(</font>time<font color="#990000">.</font><b><font color="#000000">Duration</font></b><font color="#990000">(</font>num<font color="#990000">)</font> <font color="#990000">*</font> time<font color="#990000">.</font>Second<font color="#990000">)</font> - numCh <font color="#990000"><-</font> num - <font color="#FF0000">}</font><font color="#990000">(</font>num<font color="#990000">)</font> - <font color="#FF0000">}</font> - - <b><font color="#0000FF">for</font></b> num <font color="#990000">:=</font> <b><font color="#0000FF">range</font></b> numCh <font color="#FF0000">{</font> - sorted <font color="#990000">=</font> <b><font color="#000000">append</font></b><font color="#990000">(</font>sorted<font color="#990000">,</font> num<font color="#990000">)</font> - <font color="#FF0000">}</font> - - <b><font color="#0000FF">return</font></b> sorted -<font color="#FF0000">}</font> -</pre> -<br /> -<span>This Go code implements the sleep sort algorithm using generics and goroutines. The main function <span class='inlinecode'>Sleep[V ds.Integer](a ds.ArrayList[V]) ds.ArrayList[V]</span> takes a generic <span class='inlinecode'>ArrayList</span> as input and returns a sorted <span class='inlinecode'>ArrayList</span>. The code creates a separate goroutine for each element in the input array, sleeps for a duration proportional to the element's value, and then sends the element to a channel. Another goroutine waits for all the sleeping goroutines to finish and then closes the channel. The sorted result <span class='inlinecode'>ArrayList</span> is constructed by appending the elements received from the channel in the order they arrive. The <span class='inlinecode'>sync.WaitGroup</span> is used to synchronize goroutines and ensure that all of them have completed before closing the channel.</span><br /> -<br /> -<h3 style='display: inline'>Testing</h3><br /> -<br /> -<span>For testing, we only allow values up to 10, as otherwise, it would take too long to finish:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">package</font></b> sort - -<b><font color="#0000FF">import</font></b> <font color="#990000">(</font> - <font color="#FF0000">"fmt"</font> - <font color="#FF0000">"testing"</font> - - <font color="#FF0000">"codeberg.org/snonux/algorithms/ds"</font> -<font color="#990000">)</font> - -<b><font color="#0000FF">func</font></b> <b><font color="#000000">TestSleepSort</font></b><font color="#990000">(</font>t <font color="#990000">*</font>testing<font color="#990000">.</font>T<font color="#990000">)</font> <font color="#FF0000">{</font> - a <font color="#990000">:=</font> ds<font color="#990000">.</font>NewRandomArrayList<font color="#990000">[</font>int<font color="#990000">](</font><font color="#993399">10</font><font color="#990000">,</font> <font color="#993399">10</font><font color="#990000">)</font> - a <font color="#990000">=</font> <b><font color="#000000">Sleep</font></b><font color="#990000">(</font>a<font color="#990000">)</font> - <b><font color="#0000FF">if</font></b> <font color="#990000">!</font>a<font color="#990000">.</font><b><font color="#000000">Sorted</font></b><font color="#990000">()</font> <font color="#FF0000">{</font> - t<font color="#990000">.</font><b><font color="#000000">Errorf</font></b><font color="#990000">(</font><font color="#FF0000">"Array not sorted: %v"</font><font color="#990000">,</font> a<font color="#990000">)</font> - <font color="#FF0000">}</font> -<font color="#FF0000">}</font> -</pre> -<br /> -<span>As you can see, it takes <span class='inlinecode'>9s</span> here for the algorithm to finish (which is the highest value in the <span class='inlinecode'>ArrayList</span>):</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre>❯ go <b><font color="#0000FF">test</font></b> <font color="#990000">.</font>/sort -v -run SleepSort -<font color="#990000">===</font> RUN TestSleepSort ---- PASS<font color="#990000">:</font> TestSleepSort <font color="#990000">(</font><font color="#993399">9</font><font color="#990000">.</font>00s<font color="#990000">)</font> -PASS -ok codeberg<font color="#990000">.</font>org/snonux/algorithms/sort <font color="#993399">9</font><font color="#990000">.</font>002s -</pre> -<br /> -<span>I won't write any benchmark for sleep sort; that will be done for the algorithms to come in this series :-).</span><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>'Never split the difference' book notes</title> - <link href="gemini://foo.zone/gemfeed/2023-04-01-never-split-the-difference-book-notes.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-04-01-never-split-the-difference-book-notes.gmi</id> - <updated>2023-04-01T20:00:17+03:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>These are my personal takeaways after reading 'Never split the difference' by Chris Voss. Note that the book contains much more knowledge wisdom and that these notes only contain points I personally found worth writing down. This is mainly for my own use, but you might find it helpful too.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>"Never split the difference" book notes</h1><br /> -<br /> -<span class='quote'>Published at 2023-04-01T20:00:17+03:00</span><br /> -<br /> -<span>These are my personal takeaways after reading "Never split the difference" by Chris Voss. Note that the book contains much more knowledge wisdom and that these notes only contain points I personally found worth writing down. This is mainly for my own use, but you might find it helpful too.</span><br /> -<br /> -<pre> - ,.......... .........., - ,..,' '.' ',.., - ,' ,' : ', ', - ,' ,' : ', ', - ,' ,' : ', ', - ,' ,'............., : ,.............', ', -,' '............ '.' ............' ', - '''''''''''''''''';''';'''''''''''''''''' - ''' -</pre> -<br /> -<h2 style='display: inline'>Tactical listening, spreading empathy</h2><br /> -<br /> -<span>Be a mirror, copy each other to be comfy with each other to build up trust. Mirroring is mainly body language. A mirror is to repeat the words the other just said. Simple but effective.</span><br /> -<br /> -<ul> -<li>A mirror needs space and silence between the words. At least 4 seconds.</li> -<li>A mirror might be awkward to be used at first, especially with a question coupled to it.</li> -<li>We fear what's different and are drawn to what is similar.</li> -</ul><br /> -<span>Mirror training is like Jedi training. Simple but effective. A mirror needs space. Be silent after "you want this?" </span><br /> -<br /> -<h2 style='display: inline'>Mindset of discovery</h2><br /> -<br /> -<span>Try to have multiple realities in your mind and use facts to distinguish between real and false.</span><br /> -<br /> -<ul> -<li>Focus on what the counterpart has to say and what he needs and wants. Understanding him makes him vulnerable.</li> -<li>Empathy understanding the other person from his perspective, but it does not mean agreeing with him.</li> -<li>Detect and label the emotions of others for your powers. </li> -<li>To be understood seems to solve all problems magically.</li> -</ul><br /> -<span>Try: to put a label on someone's emotion and then be silent. Wait for the other to reveal himself. "You seem unhappy about this?"</span><br /> -<br /> -<h3 style='display: inline'>More tips </h3><br /> -<br /> -<ul> -<li>Put on a poker face and don't show emotions.</li> -<li>Slow things down. Don't be a problem solver.</li> -<li>Smile while you are talking, even on the phone. Be easy and encouraging.</li> -<li>Being right is not the key to successful negotiation; being mindful is.</li> -<li>Be in the safe zone of empathy and acknowledge bad news.</li> -</ul><br /> -<h2 style='display: inline'>"No" starts the conversation</h2><br /> -<br /> -<span>When the opponent starts with a "no", he feels in control and comfortable. That's why he has to start with "no".</span><br /> -<br /> -<ul> -<li>"Yes" and "maybe" might be worthless, but "no" starts the conversation.</li> -<li>If someone is saying "no" to you, he will be open to what you have to say next.</li> -<li>"No" is not stopping the negotiation but will open up opportunities you were not thinking about before.</li> -<li>Start with "no". Great negotiators seek "no" because that's when the great discussions begin.</li> -<li>A "no" can be scary if you are not used to it. If your biggest fear is "no", then you can't negotiate.</li> -</ul><br /> -<span>Get a "That's right" when negotiating. Don't get a "you're right". You can summarise the opponent to get a "that's right".</span><br /> -<br /> -<h2 style='display: inline'>Win-win</h2><br /> -<br /> -<span>Win-win is a naive approach when encountering the win-lose counterpart, but always cooperate. Don't compromise, and don't split the difference. We don't compromise because it's right; we do it because it is easy. You must embrace the hard stuff; that's where the great deals are.</span><br /> -<br /> -<h2 style='display: inline'>On Deadlines</h2><br /> -<br /> -<ul> -<li>All deadlines are imaginary.</li> -<li>Most of the time, deadlines unsettle us without a good reason.</li> -<li>They push a deal to a conclusion.</li> -<li>They rush the counterpart to cause pressure and anxiety.</li> -</ul><br /> -<h2 style='display: inline'>Analyse the opponent</h2><br /> -<br /> -<ul> -<li>Understand the motivation of people behind the table as well.</li> -<li>Ask how affected they will be.</li> -<li>Determine your and the opposite negotiation style. Accommodation, analyst, assertive.</li> -<li>Treat them how they need to be treated.</li> -</ul><br /> -<span>The person on the other side is never the issue; the problem is the issue. Keep this in mind to avoid emotional issues with the person and focus on the problem, not the person. The bond is essential; never create an enemy.</span><br /> -<br /> -<h2 style='display: inline'>Use different ways of saying "no."</h2><br /> -<br /> -<span class='quote'>I had paid my rent always in time. I had positive experiences with the building and would be sad for the landlord to lose a good tenant. I am looking for a win-win agreement between us. Pulling out the research, other neighbours offer much lower prices even if your building is a better location and services. How can I effort 200 more.... </span><br /> -<br /> -<span>...then put an extreme anker.</span><br /> -<br /> -<span>You always have to embrace thoughtful confrontation for good negotiation and life. Don't avoid honest, clear conflict. It will give you the best deals. Compromises are mostly bad deals for both sides. Most people don't negotiate a win-win but a win-lose. Know the best and worst outcomes and what is acceptable for you.</span><br /> -<br /> -<h2 style='display: inline'>Calibrated question</h2><br /> -<br /> -<span>Calibrated questions. Give the opponent a sense of power. Ask open-how questions to get the opponent to solve your problem and move him in your direction. Calibrated questions are the best tools. Summarise everything, and then ask, "how I am supposed to do that?". Asking for help this way with a calibrated question is a powerful tool for joint problem solving</span><br /> -<br /> -<span>Being calm and respectful is essential. Without control of your emotions, it won't work. The counterpart will have no idea how constrained they are with your question. Avoid questions which get a yes or short answers. Use "why?".</span><br /> -<br /> -<span>Counterparts are more involved if these are their solutions. The counterpart must answer with "that's right", not "you are right". He has to own the problem. If not, then add more why questions.</span><br /> -<br /> -<ul> -<li>Tone and body language need to align with what people are saying.</li> -<li>Deal with it via a labelled question. </li> -<li>Liers tend to talk with "them" and "their" and not with "I".</li> -<li>Also, liars tend to talk in complex sentences.</li> -</ul><br /> -<span>Prepare 3 to 5 calibrated questions for your counterpart. Be curious what is really motivating the other side. You can get out the "Black Swan".</span><br /> -<br /> -<h2 style='display: inline'>The black swan </h2><br /> -<br /> -<span>What we don't know can break our deal. Uncovering it can bring us unexpected success. You get what you ask for in this world, but you must learn to ask correctly. Reveal the black swan by asking questions.</span><br /> -<br /> -<h2 style='display: inline'>More</h2><br /> -<br /> -<span>Establish a range at top places like corp. I get... (e.g. remote London on a project basis). Set a high salary range and not a number. Also, check on LinkedIn premium for the salaries.</span><br /> -<br /> -<ul> -<li>Give an unexpected gift, e.g. show them my pet project and publicity for engineering.</li> -<li>Use an odd number, which makes you seem to have thought a lot about the sum and calculated it.</li> -<li>Define success and metrics for your next raise.</li> -<li>What does it take to be successful here? Ask the question, and they will tell you and guide you.</li> -<li>Set an extreme anker. Make the counterpart the illusion of losing something.</li> -<li>Hope-based deals. Hope is not a strategy.</li> -<li>Tactical empathy, listening as a martial art. It is emotional intelligence on steroids.</li> -<li>Being right isn't the key to a successful negotiation, but having the correct mindset is.</li> -<li>Don't shop the groceries when you are hungry.</li> -</ul><br /> -<span>Slow.... it.... down....</span><br /> -<br /> -<span>Other book notes of mine are:</span><br /> -<br /> -<a class='textlink' href='./2023-03-16-the-pragmatic-programmer-book-notes.html'>2023-03-16 "The Pragmatic Programmer" book notes</a><br /> -<a class='textlink' href='./2023-04-01-never-split-the-difference-book-notes.html'>2023-04-01 "Never split the difference" book notes (You are currently reading this)</a><br /> -<a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes</a><br /> -<a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>Gemtexter 2.0.0 - Let's Gemtext again²</title> - <link href="gemini://foo.zone/gemfeed/2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.gmi</id> - <updated>2023-03-25T17:50:32+02:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>I proudly announce that I've released Gemtexter version `2.0.0`. What is Gemtexter? It's my minimalist static site generator for Gemini Gemtext, HTML and Markdown written in GNU Bash.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Gemtexter 2.0.0 - Let's Gemtext again²</h1><br /> -<br /> -<span class='quote'>Published at 2023-03-25T17:50:32+02:00</span><br /> -<br /> -<pre> --=[ typewriters ]=- 1/98 - - .-------. - _|~~ ~~ |_ .-------. - =(_|_______|_)= _|~~ ~~ |_ - |:::::::::| =(_|_______|_) - |:::::::[]| |:::::::::| - |o=======.| |:::::::[]| - jgs `"""""""""` |o=======.| - mod. by Paul Buetow `"""""""""` -</pre> -<br /> -<span>I proudly announce that I've released Gemtexter version <span class='inlinecode'>2.0.0</span>. What is Gemtexter? It's my minimalist static site generator for Gemini Gemtext, HTML and Markdown written in GNU Bash.</span><br /> -<br /> -<a class='textlink' href='https://codeberg.org/snonux/gemtexter'>https://codeberg.org/snonux/gemtexter</a><br /> -<br /> -<span>This is a new major release, so it contains a breaking change (see "Meta cache made obsolete").</span><br /> -<br /> -<span>Let's list what's new!</span><br /> -<br /> -<h2 style='display: inline'>Minimal template engine</h2><br /> -<br /> -<span>Gemtexter now supports templating, enabling dynamically generated content to <span class='inlinecode'>.gmi</span> files before converting anything to any output format like HTML and Markdown.</span><br /> -<br /> -<span>A template file name must have the suffix <span class='inlinecode'>gmi.tpl</span>. A template must be put into the same directory as the Gemtext <span class='inlinecode'>.gmi</span> file to be generated. Gemtexter will generate a Gemtext file <span class='inlinecode'>index.gmi</span> from a given template <span class='inlinecode'>index.gmi.tpl</span>. A <span class='inlinecode'><<<</span> and <span class='inlinecode'>>>></span> encloses a multiline template. All lines starting with <span class='inlinecode'><< </span> will be evaluated as a single line of Bash code and the output will be written into the resulting Gemtext file.</span><br /> -<br /> -<span>For example, the template <span class='inlinecode'>index.gmi.tpl</span>:</span><br /> -<br /> -<pre> -# Hello world - -<< echo "> This site was generated at $(date --iso-8601=seconds) by \`Gemtexter\`" - -Welcome to this capsule! - -<<< - for i in {1..10}; do - echo Multiline template line $i - done ->>> -</pre> -<br /> -<span>... results into the following <span class='inlinecode'>index.gmi</span> after running <span class='inlinecode'>./gemtexter --generate</span> (or <span class='inlinecode'>./gemtexter --template</span>, which instructs to do only template processing and nothing else):</span><br /> -<br /> -<pre> -# Hello world - -> This site was generated at 2023-03-15T19:07:59+02:00 by `Gemtexter` - -Welcome to this capsule! - -Multiline template line 1 -Multiline template line 2 -Multiline template line 3 -Multiline template line 4 -Multiline template line 5 -Multiline template line 6 -Multiline template line 7 -Multiline template line 8 -Multiline template line 9 -Multiline template line 10 -</pre> -<br /> -<span>Another thing you can do is insert an index with links to similar blog posts. E.g.:</span><br /> -<br /> -<pre> -See more entries about DTail and Golang: - -<< template::inline::index dtail golang - -Blablabla... -</pre> -<br /> -<span>... scans all other post entries with <span class='inlinecode'>dtail</span> and <span class='inlinecode'>golang</span> in the file name and generates a link list like this:</span><br /> -<br /> -<pre> -See more entries about DTail and Golang: - -=> ./2022-10-30-installing-dtail-on-openbsd.gmi 2022-10-30 Installing DTail on OpenBSD -=> ./2022-04-22-programming-golang.gmi 2022-04-22 The Golang Programming language -=> ./2022-03-06-the-release-of-dtail-4.0.0.gmi 2022-03-06 The release of DTail 4.0.0 -=> ./2021-04-22-dtail-the-distributed-log-tail-program.gmi 2021-04-22 DTail - The distributed log tail program (You are currently reading this) - -Blablabla... -</pre> -<br /> -<h2 style='display: inline'>Added hooks</h2><br /> -<br /> -<span>You can configure <span class='inlinecode'>PRE_GENERATE_HOOK</span> and <span class='inlinecode'>POST_PUBLISH_HOOK</span> to point to scripts to be executed before running <span class='inlinecode'>--generate</span>, or after running <span class='inlinecode'>--publish</span>. E.g. you could populate some of the content by an external script before letting Gemtexter do its thing or you could automatically deploy the site after running <span class='inlinecode'>--publish</span>.</span><br /> -<br /> -<span>The sample config file <span class='inlinecode'>gemtexter.conf</span> includes this as an example now; these scripts will only be executed when they actually exist:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><b><font color="#0000FF">declare</font></b> -xr <font color="#009900">PRE_GENERATE_HOOK</font><font color="#990000">=.</font>/pre_generate_hook<font color="#990000">.</font>sh -<b><font color="#0000FF">declare</font></b> -xr <font color="#009900">POST_PUBLISH_HOOK</font><font color="#990000">=.</font>/post_publish_hook<font color="#990000">.</font>sh -</pre> -<br /> -<h2 style='display: inline'>Use of safer Bash options</h2><br /> -<br /> -<span>Gemtexter now does <span class='inlinecode'>set -euf -o pipefile</span>, which helps to eliminate bugs and to catch scripting errors sooner. Previous versions only <span class='inlinecode'>set -e</span>.</span><br /> -<br /> -<h2 style='display: inline'>Meta cache made obsolete</h2><br /> -<br /> -<span>Here is the breaking change to older versions of Gemtexter. The <span class='inlinecode'>$BASE_CONTENT_DIR/meta</span> directory was made obsolete. <span class='inlinecode'>meta</span> was used to store various information about all the blog post entries to make generating an Atom feed in Bash easier. Especially the publishing dates of each post were stored there. Instead, the publishing date is now encoded in the <span class='inlinecode'>.gmi</span> file. And if it is missing, Gemtexter will set it to the current date and time at first run.</span><br /> -<br /> -<span>An example blog post without any publishing date looks like this:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><font color="#990000">%</font> cat gemfeed<font color="#990000">/</font><font color="#993399">2023</font>-<font color="#993399">02</font>-<font color="#993399">26</font>-title-here<font color="#990000">.</font>gmi -<i><font color="#9A1900"># Title here</font></i> - -The remaining content of the Gemtext file<font color="#990000">...</font> -</pre> -<br /> -<span>Gemtexter will add a line starting with <span class='inlinecode'>> Published at ...</span> now. Any subsequent Atom feed generation will then use that date.</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><font color="#990000">%</font> cat gemfeed<font color="#990000">/</font><font color="#993399">2023</font>-<font color="#993399">02</font>-<font color="#993399">26</font>-title-here<font color="#990000">.</font>gmi -<i><font color="#9A1900"># Title here</font></i> - -<font color="#990000">></font> Published at <font color="#993399">2023</font>-<font color="#993399">02</font>-26T21<font color="#990000">:</font><font color="#993399">43</font><font color="#990000">:</font><font color="#993399">51</font><font color="#990000">+</font><font color="#993399">01</font><font color="#990000">:</font><font color="#993399">00</font> - -The remaining content of the Gemtext file<font color="#990000">...</font> -</pre> -<br /> -<h2 style='display: inline'>XMLLint support</h2><br /> -<br /> -<span>Optionally, when the <span class='inlinecode'>xmllint</span> binary is installed, Gemtexter will perform a simple XML lint check against the Atom feed generated. This is a double-check of whether the Atom feed is a valid XML.</span><br /> -<br /> -<h2 style='display: inline'>More</h2><br /> -<br /> -<span>Additionally, there were a couple of bug fixes, refactorings and overall improvements in the documentation made. </span><br /> -<br /> -<span>Other related posts are:</span><br /> -<br /> -<a class='textlink' href='./2021-04-24-welcome-to-the-geminispace.html'>2021-04-24 Welcome to the Geminispace</a><br /> -<a class='textlink' href='./2021-06-05-gemtexter-one-bash-script-to-rule-it-all.html'>2021-06-05 Gemtexter - One Bash script to rule it all</a><br /> -<a class='textlink' href='./2022-08-27-gemtexter-1.1.0-lets-gemtext-again.html'>2022-08-27 Gemtexter 1.1.0 - Let's Gemtext again</a><br /> -<a class='textlink' href='./2023-03-25-gemtexter-2.0.0-lets-gemtext-again-2.html'>2023-03-25 Gemtexter 2.0.0 - Let's Gemtext again² (You are currently reading this)</a><br /> -<a class='textlink' href='./2023-07-21-gemtexter-2.1.0-lets-gemtext-again-3.html'>2023-07-21 Gemtexter 2.1.0 - Let's Gemtext again³</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>'The Pragmatic Programmer' book notes</title> - <link href="gemini://foo.zone/gemfeed/2023-03-16-the-pragmatic-programmer-book-notes.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-03-16-the-pragmatic-programmer-book-notes.gmi</id> - <updated>2023-03-16T00:55:20+02:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>These are my personal takeaways after reading 'The Pragmatic Programmer' by David Thomas and Andrew Hunt. Note that the book contains much more knowledge wisdom and that these notes only contain points I personally found worth writing down. This is mainly for my own use, but you might find it helpful too.</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>"The Pragmatic Programmer" book notes</h1><br /> -<br /> -<span class='quote'>Published at 2023-03-16T00:55:20+02:00</span><br /> -<br /> -<span>These are my personal takeaways after reading "The Pragmatic Programmer" by David Thomas and Andrew Hunt. Note that the book contains much more knowledge wisdom and that these notes only contain points I personally found worth writing down. This is mainly for my own use, but you might find it helpful too.</span><br /> -<br /> -<pre> - ,.......... .........., - ,..,' '.' ',.., - ,' ,' : ', ', - ,' ,' : ', ', - ,' ,' : ', ', - ,' ,'............., : ,.............', ', -,' '............ '.' ............' ', - '''''''''''''''''';''';'''''''''''''''''' - ''' -</pre> -<br /> -<span>Think about your work while doing it - every day on every project. Have a feeling of continuous improvement. </span><br /> -<br /> -<ul> -<li>Be a realist.</li> -<li>Smell challenges.</li> -<li>Care about your craft.</li> -<li>Code can always be flawed, but it can meet the requirements.</li> -<li>You should be proud of your code, though.</li> -</ul><br /> -<span>No one writes perfect code, including you. However:</span><br /> -<br /> -<ul> -<li>Paranoia is good thinking.</li> -<li>Practice defensive programming and crash early.</li> -<li>Crashing is often the best thing you can do. </li> -<li>Changes should be reversible.</li> -</ul><br /> -<span>Erlang: Defensive programming is a waste of time. Let it crash. "This can never happen" - don't practise that kind of self-deception when programming. </span><br /> -<br /> -<span>Leave assertions in the code, even in production. Only leave out the assertions causing the performance issues.</span><br /> -<br /> -<span>Take small steps, always. Get feedback, too, for each of the steps the code does. Avoid fortune telling. If you have to involve in it, then the step is too large.</span><br /> -<br /> -<span>Decouple the code (e.g. OOP or functional programming). Prefer interfaces for types and mixins for a class extension over class inheritance.</span><br /> -<br /> -<ul> -<li>Refactor now and not later.</li> -<li>Later, it will be even more painful.</li> -</ul><br /> -<span>Don't think outside the box. Find the box. The box is more extensive than you think. Think about the hard problem at hand. Do you have to do it a certain way, or do you have to do it at all?</span><br /> -<br /> -<span>Do what works and not what's fashionable. E.g. does SCRUM make sense? The goal is to deliver deliverables and not to "become" agile.</span><br /> -<br /> -<h2 style='display: inline'>Continuous learning</h2><br /> -<br /> -<span>Add new tools to your repertoire every day and keep the momentum up. Learning new things is your most crucial aspect. Invest regularly in your knowledge portfolio. The learning process extends your thinking. It does not matter if you will never use it.</span><br /> -<br /> -<ul> -<li>Learn a new programming language every year.</li> -<li>Read a technical book every month.</li> -<li>Take courses.</li> -</ul><br /> -<span>Think critically about everything you learn. Use paper for your notes. There is something special about it.</span><br /> -<br /> -<h2 style='display: inline'>Stay connected</h2><br /> -<br /> -<span>It's your life, and you own it. Bruce Lee once said: </span><br /> -<br /> -<span class='quote'>"I am not on the world to life after your expectations, neither are you to life after mine."</span><br /> -<br /> -<ul> -<li>Go to meet-ups and actively engage.</li> -<li>Stay current.</li> -<li>Dealing with computers is hard. Dealing with people is harder. </li> -</ul><br /> -<span>It's your life. Share it, celebrate it, be proud and have fun.</span><br /> -<br /> -<h2 style='display: inline'>The story of stone soup</h2><br /> -<br /> -<span>How to motivate others to contribute something (e.g. ideas to a startup):</span><br /> -<br /> -<span class='quote'>A kindly, old stranger was walking through the land when he came upon a village. As he entered, the villagers moved towards their homes, locking doors and windows. The stranger smiled and asked, why are you all so frightened. I am a simple traveler, looking for a soft place to stay for the night and a warm place for a meal. "There's not a bite to eat in the whole province," he was told. "We are weak and our children are starving. Better keep moving on." "Oh, I have everything I need," he said. "In fact, I was thinking of making some stone soup to share with all of you." He pulled an iron cauldron from his cloak, filled it with water, and began to build a fire under it. Then, with great ceremony, he drew an ordinary-looking stone from a silken bag and dropped it into the water. By now, hearing the rumor of food, most of the villagers had come out of their homes or watched from their windows. As the stranger sniffed the "broth" and licked his lips in anticipation, hunger began to overcome their fear. "Ahh," the stranger said to himself rather loudly, "I do like a tasty stone soup. Of course, stone soup with cabbage -- that's hard to beat." Soon a villager approached hesitantly, holding a small cabbage he'd retrieved from its hiding place, and added it to the pot. "Wonderful!!" cried the stranger. "You know, I once had stone soup with cabbage and a bit of salt beef as well, and it was fit for a king." The village butcher managed to find some salt beef . . . And so it went, through potatoes, onions, carrots, mushrooms, and so on, until there was indeed a delicious meal for everyone in the village to share. The village elder offered the stranger a great deal of money for the magic stone, but he refused to sell it and traveled on the next day. As he left, the stranger came upon a group of village children standing near the road. He gave the silken bag containing the stone to the youngest child, whispering to a group, "It was not the stone, but the villagers that had performed the magic." </span><br /> -<br /> -<span>By working together, everyone contributes what they can, achieving a greater good together.</span><br /> -<br /> -<span>Other book notes of mine are:</span><br /> -<br /> -<a class='textlink' href='./2023-03-16-the-pragmatic-programmer-book-notes.html'>2023-03-16 "The Pragmatic Programmer" book notes (You are currently reading this)</a><br /> -<a class='textlink' href='./2023-04-01-never-split-the-difference-book-notes.html'>2023-04-01 "Never split the difference" book notes</a><br /> -<a class='textlink' href='./2023-05-06-the-obstacle-is-the-way-book-notes.html'>2023-05-06 "The Obstacle is the Way" book notes</a><br /> -<a class='textlink' href='./2023-07-17-career-guide-and-soft-skills-book-notes.html'>2023-07-17 "Software Developmers Career Guide and Soft Skills" book notes</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../resources.html'>More books and other resources I found useful.</a><br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>How to shut down after work</title> - <link href="gemini://foo.zone/gemfeed/2023-02-26-how-to-shut-down-after-work.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-02-26-how-to-shut-down-after-work.gmi</id> - <updated>2023-02-26T23:48:01+02:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>Do you need help fully discharging from work in the evenings or for the weekend? Shutting down from work won't just improve your work-life balance; it will also significantly improve the quality of your personal life and work. After a restful weekend, you will be much more energized and productive the next working day. So it should not just be in your own, but also your employers' interest that you fully relax and shut down after work. </summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>How to shut down after work</h1><br /> -<br /> -<span class='quote'>Published at 2023-02-26T23:48:01+02:00</span><br /> -<br /> -<pre> - |\ "Music should be heard not only with the ears, but also the soul." -|---|--\-----------------------|-----------------------------------------| -| | |\ | |@ |\ | -|---|---|--\-------------------|-------------/|----|------|--\----|------| -| @| | |\ |O | 3 / | |@ | | | -|---|--@|---|--\--------|------|---------/----|----|------|-------|------| -| @| @| \ |O | / | | |@ @| @|. | -|-----------|-----|-----|------|-----/---|---@|----|--------------|------| -| @| | |O | | | | @|. | -|-----------|----@|-----|------|----|---@|------------------------|------| - @| | | Larry Komro @|. - -@- [kom...@uwec.edu] -</pre> -<br /> -<span>Do you need help fully discharging from work in the evenings or for the weekend? Shutting down from work won't just improve your work-life balance; it will also significantly improve the quality of your personal life and work. After a restful weekend, you will be much more energized and productive the next working day. So it should not just be in your own, but also your employers' interest that you fully relax and shut down after work. </span><br /> -<br /> -<h2 style='display: inline'>Have a shutdown routine</h2><br /> -<br /> -<span>Have a routine. Try to finish work around the same time every day. Write any outstanding tasks down for the next day, so you are sure you will remember them. Writing them down brings wonders as you can remove them from your mind for the remainder of the day (or the upcoming weekend) as you know you will surely pick them up the next working day. Tidying up your workplace could also count toward your daily shutdown routine. </span><br /> -<br /> -<span>A commute home from the office also greatly helps, as it disconnects your work from your personal life. Don't work on your commute home, though! If you don't commute but work from home, then it helps to walk around the block or in a nearby park to disconnect from work. </span><br /> -<br /> -<h2 style='display: inline'>Don't work when you officially don't work</h2><br /> -<br /> -<span>Unless you are self-employed, you have likely signed an N-hour per week contract with your employer, and your regular working times are from X o'clock in the morning to Y o'clock in the evening (with M minutes lunch break in the middle). And there might be some flexibility in your working times, too. But that kind of flexibility (e.g. extending the lunch break so that there is time to pick up a family member from the airport) will be agreed upon, and you will counteract it, for example, by starting working earlier the next day or working late, that one exception. But overall, your weekly working time will stay N hours. </span><br /> -<br /> -<span>Another exception would be when you are on an on-call schedule and are expected to watch your work notifications out-of-office times. But that is usually only a few days per month and, therefore, not the norm. And it should also be compensated accordingly. </span><br /> -<br /> -<span>There might be some maintenance work you must carry out, which can only be done over the weekend, but it should be explicitly agreed upon and compensated for. Also, there might be a scenario that a production incident comes up shortly before the end of the work day, requiring you (and your colleagues) to stay a bit longer. But this should be an exceptional case.</span><br /> -<br /> -<span>Other than that, there is no reason why you should work out-of-office hours. I know many people who suffer "the fear of missing out", so slack messages and E-Mails are checked until late in the evening, during weekends or holidays. I have been improving here personally a lot over the last couple of months, but still, I fall into this trap occasionally. </span><br /> -<br /> -<span>Also, when you respond to slack messages and E-Mails, your colleagues can think that you have nothing better to do. They also will take it for granted and keep slacking and messaging you out of regular office times. </span><br /> -<br /> -<span>Checking for your messages constantly outside of regular office times makes it impossible to shut down and relax from work altogether. </span><br /> -<br /> -<h2 style='display: inline'>Distract your mind</h2><br /> -<br /> -<span>Often, your mind goes back to work-related stuff even after work. That's normal as you concentrated highly on your work throughout the day. The brain unconsciously continues to work and will automatically present you with random work-related thoughts. You can counteract this by focusing on non-work stuff, which may include:</span><br /> -<br /> -<ul> -<li>Exercise. A half an hour workout or yoga session, followed by some stretching, helps to calm your mind after work. </li> -<li>Play (with your family, pets, friends, or video game)</li> -<li>Mindfully listen to music. When have you ever "really" listened to music? I mean, not just as a background stimulation but really paid attention to the melody, rhythm, voice and lyrics? That requires focused attention and distracts you from other thoughts. </li> -<li>Think of or work on that fun passion project. I currently, for example, like to learn and code a bit in Rakulang. </li> -<li>Read. Nothing beats reading a good Science Fiction Novel (or whatever you prefer) before falling asleep.</li> -</ul><br /> -<span>Some of these can be habit-stacked: Exercise could be combined with watching videos about your passion project (e.g. watching lectures about that new programming language you are currently learning for fun). With walking, for example, you could combine listening to an Audiobook or music, or you could also think about your passion project during that walk. </span><br /> -<br /> -<h2 style='display: inline'>Get a pet</h2><br /> -<br /> -<span>Even if you have children, it helps wonders to get a pet. My cat, for example, will remind me a few times daily to take a few minute's breaks to pet, play or give food. So my cat not only helps me after work but throughout the day.</span><br /> -<br /> -<span>My neighbour also works from home, and he has dogs, which he regularly has to take out to the park.</span><br /> -<br /> -<h2 style='display: inline'>Journal your day</h2><br /> -<br /> -<span>If you are upset about something, making it impossible to shut down from work, write down everything (e.g., with a pen in a paper journal). Writing things down helps you to "get rid" of the negative. Especially after conflicts with colleagues or company decisions, you don't agree on. This kind of self-therapy is excellent. Brainstorm all your emotions and (even if opinionated) opinions so you have everything on paper. Once done, you don't think about it so much anymore, as you know you can access that information if required. But stopping ruminating about it will be much easier now. You will likely never access that information again, though. But at least writing the thoughts down saved your day. </span><br /> -<br /> -<span>Write down three things which went well for the day. This helps you to appreciate the day. </span><br /> -<br /> -<h2 style='display: inline'>Don't stress about what your employer expects from you</h2><br /> -<br /> -<span>Think about what's fun and motivates you. Maybe the next promotion to Principal or a Manager role isn't for you. Many fall into the trap of stressing themselves out to satisfy the employer so that the next upgrade will happen and think about it constantly, even after work. But it is more important that you enjoy your craftsmanship. Work on what you expect from yourself. Ideally, your goals should be aligned with your employer. I am not saying you should abandon everything what your manager is asking you to do, but it is, after all, your life. And you have to decide where and on what you want to work. But don't sell yourself short. Keep track of your accomplishments.</span><br /> -<br /> -<h2 style='display: inline'>Call it a day</h2><br /> -<br /> -<span>Every day you gave your best was good; the day's outcome doesn't matter. What matters is that you know you gave your best and are closer to your goals than the previous day. This gives you a sense of progress and accomplishment.</span><br /> -<br /> -<span>There are some days at work you feel drained afterwards and think you didn't progress towards your goals at all. It's more challenging to shut down from work after such a day. A quick hack is to work on a quick win before the end of the day, giving you a sense of accomplishment after all. Another way is to make progress on your fun passion project after work. It must not be work-related, but a sense of accomplishment will still be there.</span><br /> -<span> </span><br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>Why GrapheneOS rox</title> - <link href="gemini://foo.zone/gemfeed/2023-01-23-why-grapheneos-rox.gmi" /> - <id>gemini://foo.zone/gemfeed/2023-01-23-why-grapheneos-rox.gmi</id> - <updated>2023-01-23T15:31:52+02:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>Art by Joan Stark</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Why GrapheneOS rox</h1><br /> -<br /> -<span class='quote'>Published at 2023-01-23T15:31:52+02:00</span><br /> -<br /> -<pre> -Art by Joan Stark - _.===========================._ - .'` .- - __- - - -- --__--- -. `'. - __ / ,'` _|--|_________|--|_ `'. \ - /'--| ; _.'\ | ' ' | /'._ ; | - // | |_.-' .-'.' ___ '.'-. '-._| | - (\) \"` _.-` / .-'`_ `'-. \ `-._ `"/ - (\) `-' | .' .-'" "'-. '. | `-` - (\) | / .'(3)(2)(1)'. \ | - (\) | / / (4) .-. \ \ | - (\) | | |(5) ( )'==,J | | - (\) | \ \ (6) '-' (0) / / | - (\) | \ '.(7)(8)(9).' / | - (\) ___| '. '-.._..-' .' | - (\) /.--| '-._____.-' | - (\) (\) |\_ _ __ _ __ __/| - (\) (\) | | -(\)_._._.__(\) | | - (\\\\jgs\\\) '.___________________.' - '-'-'-'--' -</pre> -<br /> -<span>In 2021 I wrote "On Being Pedantic about Open-Source", and there was a section "What about mobile?" where I expressed the dilemma about the necessity of using proprietary mobile operating systems. With GrapheneOS, I found my perfect solution for personal mobile phone use. </span><br /> -<br /> -<a class='textlink' href='./2021-08-01-on-being-pedantic-about-open-source.html'>On Being Pedantic about Open-Source</a><br /> -<br /> -<span>What is GrapheneOS?</span><br /> -<br /> -<span class='quote'>GrapheneOS is a privacy and security-focused mobile OS with Android app compatibility developed as a non-profit open-source project. It's focused on the research and development of privacy and security technologies, including substantial improvements to sandboxing, exploits mitigations and the permission model.</span><br /> -<br /> -<span>GrapheneOS is an independent Android distribution based on the Android Open Source Project (AOSP) but hardened in multiple ways. Other independent Android distributions, like LineageOS, are also based on AOSP, but GrapheneOS takes it further so that it can be my daily driver on my phone.</span><br /> -<br /> -<a class='textlink' href='https://GrapheneOS.org'>https://GrapheneOS.org</a><br /> -<a class='textlink' href='https://LineageOS.org'>https://LineageOS.org</a><br /> -<br /> -<h2 style='display: inline'>User Profiles</h2><br /> -<br /> -<span>GrapheneOS allows configuring up to 32 user profiles (including a guest profile) on a single phone. A profile is a completely different environment within the phone, and it is possible to switch between them instantly. Sessions of a profile can continue running in the background or be fully terminated. Each profile can have completely different settings and different applications installed.</span><br /> -<br /> -<span>I use my default profile with primarily open-source applications installed, which I trust. I use another profile for banking (PayPal, various proprietary bank apps, Amazon store app, etc.) and another profile for various Google services (which I try to avoid, but I have to use once in a while). Furthermore, I have configured a profile for Social Media use (that one isn't in my default profile, as otherwise I am tempted to scroll social media all the time, which I try to avoid and only want to do intentionally when switching to the corresponding profile!).</span><br /> -<br /> -<span>The neat thing about the profiles is that some can run a sandboxed version of Google Play (see later in this post), while others don't. So some profiles can entirely operate without any Google Play, and only some profiles (to which I rarely switch) have Google Play enabled. </span><br /> -<br /> -<span>You notice how much longer (multiple days) your phone can be on a single charge when Google Play Services isn't running in the background. This tells a lot about the background activities and indicates that using Google Play shouldn't be the norm.</span><br /> -<br /> -<h2 style='display: inline'>Proxying some of the Google offerings </h2><br /> -<br /> -<span>There's also the case that I am using an app from the Google Play store (as the app isn't available from F-Droid), which doesn't require Google Play Services to run in the background. Here's where I use the Aurora Android store. The Aurora store can be installed through F-Droid. Aurora acts as an anonymous proxy from your phone to the Google Play Store and lets you install apps from there. No Google credentials are required for that!</span><br /> -<br /> -<a class='textlink' href='https://f-droid.org'>https://f-droid.org</a><br /> -<br /> -<span>There's a similar solution for watching videos on YouTube. You can use the NewPipe app (also from F-Droid), which acts as an anonymous proxy for watching videos from YouTube. So there isn't any need to install the official YouTube app, and there isn't any need to login to your Google account. What's so bad about the official app? You don't know which data it is sending about you to Google, so it is a privacy concern. </span><br /> -<br /> -<h2 style='display: inline'>Google Play Sandboxing </h2><br /> -<br /> -<span>Before switching to GrapheneOS, I had been using LineageOS on one of my phones for a couple of years. Still, I always had to have a secondary personal phone with all of these proprietary apps which (partially) only work with Google Play on the phone (e.g. Banking, Navigation, various travel apps from various Airlines, etc.) somewhere around as I didn't install Google Play on my LineageOS phone due to privacy concerns and only installed apps from the F-Droid store on it. When travelling, I always had to carry around a second phone with Google Play on it, as without it; life would become inconvenient pretty soon. </span><br /> -<br /> -<span>With GrapheneOS, it is different. Here, I do not just have a separate user profile, "Google", for various Google apps where Google Play runs, but Google Play also runs in a sandbox!!!</span><br /> -<br /> -<span class='quote'>GrapheneOS has a compatibility layer providing the option to install and use the official releases of Google Play in the standard app sandbox. Google Play receives no special access or privileges on GrapheneOS instead of bypassing the app sandbox and receiving a massive amount of highly privileged access. Instead, the compatibility layer teaches it how to work within the full app sandbox. It also isn't used as a backend for the OS services as it would be elsewhere since GrapheneOS doesn't use Google Play even when it's installed.</span><br /> -<br /> -<span>When I need to access Google Play, I can switch to the "Google" profile. Even there, Google is sandboxed to the absolute minimum permissions required to be operational, which gives additional privacy protection.</span><br /> -<br /> -<span>The sad truth is that Google Maps is still the best navigation app. When driving unknown routes, I can switch to my Google profile to use Google Maps. I don't need to do that when going streets I know about, but it is crucial (for me) to have Google Maps around when driving to a new destination.</span><br /> -<br /> -<span>Also, Google Translate and Google Lens are still the best translation apps I know. I just recently relocated to another country, where I am still learning the language, so Google Lens has been proven very helpful on various occasions by ad-hoc translating text into English or German for me.</span><br /> -<br /> -<span>The same applies to banking. Many banking apps require Google Play to be available (It might be even more secure to only use banking apps from the Google Play store due to official support and security updates). I rarely need to access my mobile banking app, but once in a while, I need to. As you have guessed by now, I can switch to my banking profile (with Google Play enabled), do what I need to do, and then terminate the session and go back to my default profile, and then my life can go on :-). </span><br /> -<br /> -<span>It is great to have the flexibility to use any proprietary Android app when needed. That only applies to around 1% of my phone usage time, but you often don't always know when you need "that one app now". So it's perfect that it's covered with the phone you always have with you. </span><br /> -<br /> -<h2 style='display: inline'>The camera and the cloud </h2><br /> -<br /> -<span>I really want my phone to shoot good looking pictures, so that I can later upload them to the Irregular Ninja:</span><br /> -<br /> -<a class='textlink' href='https://irregular.ninja'>https://irregular.ninja</a><br /> -<br /> -<span>The stock camera app of the OASP could be better. Photos usually look washed out, and the app lacks features. With GrapheneOS, there are two options:</span><br /> -<br /> -<ul> -<li>Use the official Google camera app with sandboxed Google Play Services running. You will get the full Google experience here.</li> -<li>Or, just use the default GrapheneOS camera app.</li> -</ul><br /> -<span>The GrapheneOS camera app is much better than the stock OASP camera app. I have been comparing the photo quality of my Pixel phone under LineageOS and GrapheneOS, and the differences are pronounced. I didn't compare the quality with the official Google camera app, but I have seen some comparison videos and the differences seem like they aren't groundbreaking. </span><br /> -<br /> -<span>For automatic backups of my photos, I am relying on a self-hosted instance of NextCloud (with a client app available via F-Droid). So there isn't any need to rely on any Google apps and services (Google Play Photos or Google Camera app) anymore, and that's great!</span><br /> -<br /> -<a class='textlink' href='https://nextcloud.com'>https://nextcloud.com</a><br /> -<br /> -<span>I also use NextCloud to synchronize my notes (NextCloud Notes), my RSS news feeds (NextCloud News) and contacts (DAVx5). All apps required are available in the F-Droid store.</span><br /> -<br /> -<h2 style='display: inline'>Fine granular permissions</h2><br /> -<br /> -<span>Another great thing about GrapheneOS is that, besides putting your apps into different profiles, you can also restrict network access and configure storage scopes per app individually.</span><br /> -<br /> -<span>For example, let's say you are installing that one proprietary app from the Google Play Store through the Aurora store, and then you want to ensure that the app doesn't send data "home" through the internet. Nothing is easier to do than that. Just remove network access permissions from that only app.</span><br /> -<br /> -<span>The app also wants to store and read some data from your phone (e.g. it could be a proprietary app for enhancing photos, and therefore storage access to a photo folder would be required). In GrapheneOS, you can configure a storage scope for that particular app, e.g. only read and write from one folder but still forbid access to all other folders on your phone.</span><br /> -<br /> -<h2 style='display: inline'>Termux</h2><br /> -<br /> -<span>Termux can be installed on any Android phone through F-Droid, so it doesn't need to be a GrapheneOS phone. But I have to mention Termux here as it significantly adds value to my phone experience. </span><br /> -<br /> -<span class='quote'>Termux is an Android terminal emulator and Linux environment app that works directly with no rooting or setup required. A minimal base system is installed automatically - additional packages are available using the APT package manager.</span><br /> -<br /> -<a class='textlink' href='https://termux.dev'>https://termux.dev</a><br /> -<br /> -<span>In short, Termux is an entire Linux environment running on your Android phone. Just pair your phone with a Bluetooth keyboard, and you will have the whole Linux experience. I am only using terminal Linux applications with Termux, though. What makes it especially great is that I could write on a new blog post (in Neovim through Termux on my phone) or do some coding whilst travelling (e.g. during a flight), or look up my passwords or some other personal documents (through my terminal-based password manager). All changes I commit to Git can be synced to the server with a simple <span class='inlinecode'>git push</span> once online (e.g. after the plane landed) again.</span><br /> -<br /> -<span>There are Pixel phones with a screen size of 6", and that's decent enough for occasional use like that, and everything (the phone, the BT keyboard, maybe an external battery pack) all fit nicely in a small travel pocket.</span><br /> -<br /> -<h2 style='display: inline'>So, why not use a pure Linux phone?</h2><br /> -<br /> -<span>Strictly speaking, an Android phone is a Linux phone, but it's heavily modified and customized. For me, a "pure" Linux phone is a more streamlined Linux kernel running in a distribution like Ubuntu Touch or Mobian. </span><br /> -<br /> -<span>A pure Linux phone, e.g. with Ubuntu Touch installed, e.g. on a PinePhone, Fairphone, the Librem 5 or the Volla phone, is very appealing to me. And they would also provide an even better Linux experience than Termux does. Some support running LineageOS within an Anbox, enabling you to run various proprietary Android apps occasionally within Linux.</span><br /> -<br /> -<a class='textlink' href='https://ubuntu-touch.io/'>Ubuntu Touch</a><br /> -<a class='textlink' href='https://en.wikipedia.org/wiki/Linux_for_mobile_devices'>More Linux distributions for mobile devices </a><br /> -<br /> -<span>But here, Google Play would not be sandboxed; you could not configure individual network permissions and storage scopes like in GrapheneOS. Pure Linux-compatible phones usually come with a crappy camera, and the battery life is generally pretty bad (only a few hours). Also, no big tech company pushes the development of Linux phones. Everything relies on hobbyists, whereas multiple big tech companies put a lot of effort into the Android project, and a lot of code also goes into the Android Open-Source project. </span><br /> -<br /> -<span>Currently, pure Linux phones are only a nice toy to tinker with but are still not ready (will they ever?) to be the daily driver. SailfishOS may be an exception; I played around with it in the past. It is pretty usable, but it's not an option for me as it is partial a proprietary operating system.</span><br /> -<br /> -<a class='textlink' href='https://sailfishos.org'>SailfishOS</a><br /> -<br /> -<h2 style='display: inline'>Small GrapheneOS downsides </h2><br /> -<br /> -<span>Sometimes, switching a profile to use a different app is annoying, and you can't copy and paste from the system clipboard from one profile to another. But that's a small price I am willing to pay!</span><br /> -<br /> -<span>Another thing is that GrapheneOS can only run on Google Pixel phones, whereas LineageOS can be installed on a much larger variety of hardware. But on the other hand, GrapheneOS works very well on Pixel phones. The GrapheneOS team can concentrate their development efforts on a smaller set of hardware which then improves the software's quality (best example: The camera app).</span><br /> -<br /> -<span>And, of course, GrapheneOS is an open-source project. This is a good thing; however, on the other side, nobody can guarantee that the OS will not break or will not damage your phone. You have to trust the GrapheneOS project and donate to the project so they can keep up with the great work. But I rather trust the GrapheneOS team than big tech. </span><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>(Re)learning Java - My takeaways</title> - <link href="gemini://foo.zone/gemfeed/2022-12-24-ultrarelearning-java-my-takeaways.gmi" /> - <id>gemini://foo.zone/gemfeed/2022-12-24-ultrarelearning-java-my-takeaways.gmi</id> - <updated>2022-12-24T23:18:40+02:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>As a regular participant in the annual Pet Project competition at work, I always try to find a project where I can learn something new. In this post, I would like to share my takeaways after revisiting Java. You can read about my motivations in my 'Creative universe' post:</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>(Re)learning Java - My takeaways</h1><br /> -<br /> -<span class='quote'>Published at 2022-12-24T23:18:40+02:00</span><br /> -<br /> -<a href='./2022-12-24-ultrarelearning-java-my-takeaways/learnjava.jpg'><img src='./2022-12-24-ultrarelearning-java-my-takeaways/learnjava.jpg' /></a><br /> -<br /> -<span>As a regular participant in the annual Pet Project competition at work, I always try to find a project where I can learn something new. In this post, I would like to share my takeaways after revisiting Java. You can read about my motivations in my "Creative universe" post:</span><br /> -<br /> -<a class='textlink' href='./2022-04-10-creative-universe.html'>Creative universe</a><br /> -<br /> -<span>I have been programming in Java back in the days as a university student, and even my Diploma Thesis I implemented in Java (it would require some overhaul so that it is fully compatible with a recent version of Java, though - It still compiles and runs, but with a lot of warnings, though!):</span><br /> -<br /> -<a class='textlink' href='https://codeberg.org/snonux/vs-sim'>VS-Sim: Distributed systems simulator</a><br /> -<br /> -<span>However, after that, I became a Linux Sysadmin and mainly continued programming in Perl, Puppet, bash, and a little Python. For personal use, I also programmed a bit in Haskell and C. After my Sysadmin role, I moved to London and became a Site Reliability Engineer (SRE), where I mainly programmed in Ruby, bash, Puppet and Golang and a little bit of C. </span><br /> -<br /> -<span>At my workplace, as an SRE, I don't do Java a lot. I have been reading Java code to understand the software better so I can apply and suggest workarounds or fixes to existing issues and bugs. However, most of our stack is in Java, and our Software Engineers use Java as their primary programming language.</span><br /> -<br /> -<h2 style='display: inline'>Stuck at Java 1.4</h2><br /> -<br /> -<span>Over time, I had been missing out on many new features that were added to the language since Java 1.4, so I decided to implement my next Pet Project in Java and learn every further aspect of the language as my main goal. Of course, I still liked the idea of winning a Pet Project Prize, but my main objective was to level up my Java skills.</span><br /> -<br /> -<h2 style='display: inline'>(Re)learning & upskilling to Java 18</h2><br /> -<br /> -<h3 style='display: inline'>Effective Java</h3><br /> -<br /> -<span>This book was recommended by my brother and also by at least another colleague at work to be one of the best, if not the best, book about Java programming. I read the whole book from the beginning to the end and immersed myself in it. I fully agree; this is a great book. Every Java developer or Java software engineer should read it!</span><br /> -<br /> -<a href='./2022-12-24-ultrarelearning-java-my-takeaways/effective-java.jpg'><img src='./2022-12-24-ultrarelearning-java-my-takeaways/effective-java.jpg' /></a><br /> -<br /> -<span>I recommend reading the 90-part effective Java Series on <span class='inlinecode'>dev.to</span>. It's a perfect companion to the book as it explains all the chapters again but from a slightly different perspective and helps you to really understand the content.</span><br /> -<br /> -<a class='textlink' href='https://dev.to/kylec32/series/2292'>Kyle Carter's 90-part Effective Java Series </a><br /> -<br /> -<h3 style='display: inline'>Java Pub House</h3><br /> -<br /> -<span>During my lunch breaks, I usually have a walk around the block or in a nearby park. I used that time to listen to the Java Pub House podcast. I listened to *every* episode and learned tons of new stuff. I can highly recommend this podcast. Especially GraalVM, a high-performance JDK distribution written for Java and other JVM languages, captured my attention. GraalVM can compile Java code into native binaries, improving performance and easing the distribution of Java programs. Because of the latter, I should release a VS-Sim GraalVM edition one day through a Linux AppImage ;-).</span><br /> -<br /> -<a class='textlink' href='https://www.javapubhouse.com'>https://www.javapubhouse.com</a><br /> -<a class='textlink' href='https://www.graalvm.org'>https://www.graalvm.org</a><br /> -<br /> -<h3 style='display: inline'>Java Concurrency course</h3><br /> -<br /> -<span>I also watched a course on O'Reilly Safari Books online about Java Concurrency. That gave an excellent refresher on how the Java thread pools work and what were the concurrency primitives available in the standard library.</span><br /> -<br /> -<h3 style='display: inline'>Read a lot of Java code</h3><br /> -<br /> -<span>First, the source code is often the best documentation (if programmed nicely), and second, it helps to get the hang of the language and standard practices. I started to read more and more Java code at work. I did that whenever I had to understand how something, in particular, worked (e.g. while troubleshooting and debugging an issue). </span><br /> -<br /> -<h3 style='display: inline'>Observed Java code reviews</h3><br /> -<br /> -<span>Another great way to get the hang of Java again was to sneak into the code reviews of the Software Engineer colleagues. They are the expert on the matter and are a great source to copy knowledge. It's OK to stay passive and only follow the reviews. Sometimes, it's OK to step up and take ownership of the review. The developers will also always be happy to answer any naive questions which come up.</span><br /> -<br /> -<h3 style='display: inline'>Took ownership of a roadmap-Java project</h3><br /> -<br /> -<span>Besides my Pet Project, I also took ownership of a regular roadmap Java project at work, making an internal Java service capable of running in Kubernetes. This was a bunch of minor changes and adding a bunch of classes and unit tests dealing with the statelessness and a persistent job queue in Redis. The job also involved reading and understanding a lot of already existing Java code. It wasn't part of my job description, but it was fun, and I learned a lot. The service runs smoothly in production now. Of course, all of my code got reviewed by my Software Engineering colleagues.</span><br /> -<br /> -<h2 style='display: inline'>The good</h2><br /> -<br /> -<span>From the new language features and syntaxes, there are many personal takeaways, and I can't possibly list them all, but here are some of my personal highlights:</span><br /> -<br /> -<ul> -<li>Static factory methods and public constructors both have their uses, and it pays to understand their relative merits. Often static factories are preferable (cleaner and easier to read), so avoid the reflex to provide public constructors without first considering static factories.</li> -<li>Java streams were utterly new to me. I love how they can help to produce more compact code. But it's challenging to set the line of when enough is enough. Overusing streams can have the opposite effect: Code becomes more complex and challenging to understand. And it is so easy to parallelize the computation of streams by "just" marking the stream as <span class='inlinecode'>.parallel()</span> (more on that later in this post).</li> -<li>Overall, object-oriented languages tend to include more and more functional paradigms. The functional interfaces, which Java provides now, are fantastic. Their full powers shine in combination with the use of streams. An entire book can be written about Java functional interfaces, so I leave it to you to do any further digging.</li> -<li>Local type inference help to reduce even more boilerplate code. E.g. instead of <span class='inlinecode'>Hash<String,Hash<String,String>> foo = new Hash<String,Hash<String,String>>();</span> it's possible to just write <span class='inlinecode'>var foo = new Hash<String,Hash<String,String>>();</span></li> -<li>Class inheritance isn't the preferred way anymore to structure reusable code. Now, it's composition over inheritance. E.g. use dependency injection (inject one object to another object through its constructor) or prefer interfaces (which now also support default implementations of methods) over class inheritance. This makes sense to me as I do that already when I program in Ruby. </li> -<li>I learned the <span class='inlinecode'>try-with-resources</span> pattern. Very useful in ensuring closing resources again correctly. No need anymore for complicated and nested <span class='inlinecode'>finally</span>-blocks, which used to be almost impossible to get right previously in case of an error condition (e.g. I/O error somewhere deeply nested in an input or output stream).</li> -<li>Optimize only when required. It's considered to be cleaner to prefer immutable variables (declaring them as <span class='inlinecode'>final</span>). I knew that already, but for Java, it always seemed to be a waste of resources (creating entirely new objects whenever states change), but apparently, it's okay. Java also does many internal tricks for performance optimization here, e.g. interning strings.</li> -<li>I learned about the concept of static member classes and the difference between non-static member classes (also sometimes known as inner classes). Non-static member classes have full access to all members of their outer class (think of closure). In contrast, static member classes act like completely separate classes without such access but provide the benefit of a nested name that can help group functionality in the code.</li> -<li>I learned about the existence of thread-local variables. These are only available to the current thread and aren't shared with other threads.</li> -<li>I learned about the concept of Java modules, which help to structure larger code bases better. The traditional Java packages are different. </li> -<li>I learned to love the new <span class='inlinecode'>Optional</span> type. I already knew the concept from Haskell, where <span class='inlinecode'>Maybe</span> would be the corresponding type. <span class='inlinecode'>Optional</span> helps to avoid <span class='inlinecode'>null</span>-pointers but comes with some (minimal) performance penalty. So, in the end, you end up with both <span class='inlinecode'>Optional</span> types and <span class='inlinecode'>null</span>-pointers in your code (depending on the requirements). But I like to prefer <span class='inlinecode'>Optional</span> over <span class='inlinecode'>null</span>-pointer when "no result" is a valid return value from a method.</li> -<li>The <span class='inlinecode'>enum</span> type is way more powerful than I thought. Initially, I felt an <span class='inlinecode'>enum</span> could only be used to define a list of constants and then to compare an instance to another instance of the same. An <span class='inlinecode'>enum</span> is still there to define a list of constants, but it's also almost like a <span class='inlinecode'>class</span> (you can implement constructors, and methods, inherit from other enums). There are quite a lot of possible use cases.</li> -<li>A small but almost the most helpful thing I learned is always to use the <span class='inlinecode'>@Override</span> annotation when overriding a method from a parent class. If done, Java helps to detect any typos or type errors when overriding methods. That's useful and spares a lot of time debugging where a method was mistakenly overloaded but not overridden.</li> -<li>Lambdas are much cleaner, shorter and easier to read than anonymous classes. Many Java libraries require passing instances of (anonymous) classes (e.g. in Swing) to other objects. Lambdas are so lovely because they are primarily compatible with the passing of anonymous classes, so they are a 1:1 replacement in many instances. Lambdas also play very nicely together with the Java functional interfaces, as each Lambda got a type, and the type can be an already existing functional interface (or, if you got a particular case, you could define your custom functional interface for your own set of Lambdas, of course).</li> -<li>I love the concept of Java records. You can think of a record as an immutable object holding some data (as members). They are ideal for pipe and stream processing. They are much easier to define (with much less boilerplate) and come with write protection out of the box.</li> -</ul><br /> -<h2 style='display: inline'>The bad and the ugly</h2><br /> -<br /> -<span>There are also many ugly corners in Java. Many are doomed to stay there forever due to historical decisions and ensuring backward compatibility with older versions of the Java language and the Java standard library. </span><br /> -<br /> -<ul> -<li>Finalizers and cleaners seem obsolete, fragile and still, you can use them.</li> -<li>In many cases, extreme caution needs to be taken to minimize the accessibility of class members. You might think that Java provides the best "out-of-the-box" solution for proper encapsulation, but the language has many loopholes.</li> -<li>In the early days, Java didn't support generics yet. So what you would use is to cast everything to <span class='inlinecode'>Object</span>. Java now fully supports generics (for a while already), but you can still cast everything to <span class='inlinecode'>Object</span> and back to whatever type you want. That can lead to nasty runtime errors. Also, there's a particular case to convert between an Array of Object to an Array of String or from an Array of String to a List of String. Java can't convert between these types automatically, and extreme caution needs to be taken when enforcing so (e.g. through explicit type casts). In many of these cases, Java would print out warnings that need to be manually suppressed via annotations. Programming that way, converting data between old and new best practices, is clunky.</li> -<li>If you don't know what you do, Java streams can be all wrong. Side effects in functions used in streams can be nasty to debug. Also, don't just blindly add a <span class='inlinecode'>.parallel()</span> to your stream. You need to understand what the stream does and how it exactly works; otherwise, parallelizing a stream can impact the performance drastically (in a negative way). There need to be language constructs preventing you from doing the wrong things. That's so much easier to do it right in a purely functional programming language like Haskell.</li> -<li>Java is a pretty old language (already), so there are many obstacles to consider. There are too many exceptions and different outcomes of how Java code can behave. In most cases, when you write an API, every method you program needs to be documented so the user won't encounter any surprises using your code. Writing and reading a lot of documentation seems to be quite the overhead when the method name is already descriptive.</li> -<li>Java serialization is broken. It works, and the language still supports it, but you better not use Java's native way of object serialization and deserialization. Unbelievable how much can get wrong here, especially regarding security (injecting arbitrary code).</li> -<li>Being a bit spoiled by Golang's Goroutines, I was shocked about the limitations of the Java threads. They are resource hungry, and you can't just spin up millions of them as you would with Goroutines. I knew this limitation of threads already (as it's not a problem of the language but of how threads work in the OS), but still, I was pretty shocked when I got reminded of them again. Of course, there's a workaround: Use asynchronous sockets so that you don't waste a whole thread on a single I/O operation (in my case, waiting for a network response). Golang's runtime does that automatically for you: An OS thread will be re-used for other tasks until the network socket unblocks. Every modern programming language should support lightweight threads or Coroutines like Go's Goroutines. </li> -</ul><br /> -<br /> -<h2 style='display: inline'>Conclusion</h2><br /> -<br /> -<span>While (re)learning Java, I felt like a student again and was quite enthusiastic about it initially. I invested around half a year, immersing myself intensively in Java (again). The last time I did that was many years ago as a university student. I even won a Silver Prize at work, implementing a project this year (2022 as of writing this). I feel confident now with understanding, debugging and patching Java code at work, which boosted my debugging and troubleshooting skills. </span><br /> -<br /> -<span>I don't hate Java, but I don't love programming in it, either. I will, I guess, always see Java as the necessary to get stuff done (reading code to understand how the service works, adding a tiny feature to make my life easier, adding a quick bug fix to overcome an obstacle...).</span><br /> -<br /> -<span>Although Java has significantly improved since 1.4, its code still tends to be more boilerplate. Not mainly because due to lines of code (Golang code tends to be quite repetitive, primarily when no generics are used), but due to the levels of abstractions it uses. Class hierarchies can be ten classes or deeper, and it is challenging to understand what the code is doing. Good test coverage and much documentation can mitigate the problem partially. Big enterprises use Java, and that also reflects to the language. There are too many libraries and too many abstractions that are bundled with too many legacy abstractions and interfaces and too many exceptions in the library APIs. There's even an external library named Lombok, which aims to reduce Java boilerplate code. Why is there a need for an external library? It should be all part of Java itself.</span><br /> -<br /> -<a class='textlink' href='https://projectlombok.org/'>https://projectlombok.org/</a><br /> -<br /> -<span>Java needs a clean cut. The clean cut shall be incompatible with previous versions of Java and only promote modern best practices without all the legacy burden carried around. The same can be said for other languages, e.g. Perl, but in Perl, they already attack the problem with the use of flags which change the behaviour of the language to more modern standards. Or do it like Python, where they had a hard (incompatible) cut from version 2 to version 3. It will be painful, for sure. But that would be the only way I would enjoy using that language as one of my primary languages to code new stuff regularly. Currently, my Java will stay limited to very few projects and the more minor things already mentioned in this post. </span><br /> -<br /> -<span>Am I a Java expert now? No, by far not. But I am better now than before :-).</span><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>I tried (Doom) Emacs, but I switched back to (Neo)Vim</title> - <link href="gemini://foo.zone/gemfeed/2022-11-24-i-tried-emacs-but-i-switched-back-to-neovim.gmi" /> - <id>gemini://foo.zone/gemfeed/2022-11-24-i-tried-emacs-but-i-switched-back-to-neovim.gmi</id> - <updated>2022-11-24T11:17:15+02:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>Art by \ \_! / __!</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>I tried (Doom) Emacs, but I switched back to (Neo)Vim</h1><br /> -<br /> -<span class='quote'>Published at 2022-11-24T11:17:15+02:00; Updated at 2022-11-26</span><br /> -<br /> -<pre> - _/ \ _(\(o - / \ / _ ^^^o - / ! \/ ! '!!!v' - ! ! \ _' ( \____ - ! . \ _!\ \===^\) -Art by \ \_! / __! - Gunnar Z. \! / \ <--- Emacs is a giant dragon - (\_ _/ _\ ) - \ ^^--^^ __-^ /(__ - ^^----^^ "^--v' -</pre> -<br /> -<span>As a long-lasting user of Vim (and NeoVim), I always wondered what GNU Emacs is really about, so I decided to try it. I didn't try vanilla GNU Emacs, but Doom Emacs. I chose Doom Emacs as it is a neat distribution of Emacs with Evil mode enabled by default. Evil mode allows Vi(m) key bindings (so to speak, it's emulating Vim within Emacs), and I am pretty sure I won't be ready to give up all the muscle memory I have built over more than a decade.</span><br /> -<br /> -<a class='textlink' href='https://www.gnu.org/software/emacs/'>GNU Emacs</a><br /> -<a class='textlink' href='https://github.com/doomemacs/'>Doom Emacs</a><br /> -<br /> -<span>I used Doom Emacs for around two months. Still, ultimately I decided to switch back to NeoVim as my primary editor and IDE and Vim (usually pre-installed on Linux-based systems) and Nvi (usually pre-installed on *BSD systems) as my "always available editor" for quick edits. (It is worth mentioning that I don't have a high opinion on whether Vim or NeoVim is the better editor, I prefer NeoVim as it comes with better defaults out of the box, but there is no real blocker to use Vim instead).</span><br /> -<br /> -<a class='textlink' href='https://www.vim.org'>Vim</a><br /> -<a class='textlink' href='https://neovim.io'>NeoVim</a><br /> -<br /> -<span>So why did I switch back to the Vi-family?</span><br /> -<br /> -<h2 style='display: inline'>Emacs is a giant dragon</h2><br /> -<br /> -<span>Emacs feels like a giant dragon as it is much more than an editor or an integrated development environment. Emacs is a whole platform on its own. There's an E-Mail client, an IRC client, or even games you can run within Emacs. And you can also change Emacs within Emacs using its own Lisp dialect, Emacs Lisp (Emacs is programmed in Emacs Lisp). Therefore, Emacs is also its own programming language. You can change every aspect of Emacs within Emacs itself. People jokingly state Emacs is an operating system and that you should directly use it as the <span class='inlinecode'>init 1</span> process (if you don't know what the <span class='inlinecode'>init 1</span> process is: Under UNIX and similar operating systems, it's the very first userland processed launched. That's usually <span class='inlinecode'>systemd</span> on Linux-based systems, <span class='inlinecode'>launchd</span> on macOS, or any other init script or init system used by the OS)!</span><br /> -<br /> -<span>In many aspects, Emacs is like shooting at everything with a bazooka! However, I prefer it simple. I only wanted Emacs to be a good editor (which it is, too), but there's too much other stuff in Emacs that I don't need to care about! Vim and NeoVim do one thing excellent: Being great text editors and, when loaded with plugins, decent IDEs, too. </span><br /> -<br /> -<h2 style='display: inline'>Magit love</h2><br /> -<br /> -<span>I almost fell in love with Magit, an integrated Git client for Emacs. But I think the best way to interact with Git is to use the <span class='inlinecode'>git</span> command line directly. I don't worry about typing out all the commands, as the most commonly used commands are in my shell history. Other useful Git programs I use frequently are <span class='inlinecode'>bit</span> and <span class='inlinecode'>tig</span>. Also, get a mechanical keyboard that makes hammering whole commands into the terminal even more enjoyable.</span><br /> -<br /> -<a class='textlink' href='https://magit.vc/'>Magit</a><br /> -<a class='textlink' href='https://github.com/jonas/tig'>Tig</a><br /> -<br /> -<span>Magit is pretty neat for basic Git operations, but I found myself searching the internet for the correct sub-commands to do the things I wanted to do in Git. Mainly, the way how branches are managed is confusing. Often, I fell back to the command line to fix up the mess I produced with Magit (e.g. accidentally pushing to the wrong remote branch, so I found myself fixing things manually on the terminal with the <span class='inlinecode'>git</span> command with forced pushes....). Magit is hotkey driven, and common commands are quickly explorable through built-in hotkey menus. Still, I found it challenging to navigate to more advanced Git sub-commands that way which was much easier accomplished by using the <span class='inlinecode'>git</span> command directly.</span><br /> -<br /> -<h2 style='display: inline'>Graphical UI</h2><br /> -<br /> -<span>If there is one thing I envy about Emacs is that it's a graphical program, whereas the Vi-family of editors are purely terminal-based. I see the benefits of being a graphical program as this enables the use of multiple fonts simultaneously to embed pictures and graphs (that would be neat as a Markdown preview, for example). There's also GVim (Vim with GTK UI), but that's more of an afterthought.</span><br /> -<br /> -<span>There are now graphical front-end clients for NeoVim, but I still need to dig into them. Let me know your experience if you have one. Luckily, I don't rely on something graphical in my text editor, but it would improve how the editor looks and feels. UTF8 can already do a lot in the terminal, and terminal emulators also allow you to use TrueType fonts. Still, you will always be limited to one TTF font for the whole terminal, and it isn't possible to have, for example, a different font for headings, paragraphs, etc... you get the idea. TTF+UTF8 can't beat authentic graphics. </span><br /> -<br /> -<h2 style='display: inline'>Scripting it</h2><br /> -<br /> -<span>It is possible to customize every aspect of Emacs through Emacs Lisp. I have done some Elk Scheme programming in the past (a dialect of Lisp), but that was a long time ago, and I am not willing to dive here again to customize my environment. I would instead take the pragmatic approach and script what I need in VimScript (a terrible language, but it gets the job done!). I watched Damian Conway's VimScript course on O'Reilly Safari Books Online, which I greatly recommend. Yes, VimScript feels clunky, funky and weird and is far less elegant than Lisp, but it gets its job done - in most cases! (That reminds me that the Vim team has announced a new major version of VimScript with improvements and language changes made - I haven't gotten to it yet - but I assume that VimScript will always stay VimScript).</span><br /> -<br /> -<a class='textlink' href='https://en.wikipedia.org/wiki/Emacs_Lisp'>Emacs Lisp</a><br /> -<a class='textlink' href='http://sam.zoy.org/elk/'>Elk Scheme</a><br /> -<a class='textlink' href='http://vimscript.org/'>VimScript</a><br /> -<a class='textlink' href='https://www.oreilly.com/library/view/scripting-vim/9781491996287/'>Scripting Vim by Damian Conway</a><br /> -<br /> -<span>NeoVim is also programmable with Lua, which seems to be a step up and Vim comes with a Perl plugin API (which was removed from NeoVim, but that is a different story - why would someone remove the most potent mature text manipulation programming language from one of the most powerful text editors?).</span><br /> -<br /> -<a class='textlink' href='https://neovim.io/doc/user/lua.html'>NeoVim Lua API</a><br /> -<br /> -<span>One example is my workflow of how I compose my blog articles (e.g. this one you are currently reading): I am writing everything in NeoVim, but I also want to have every paragraph checked against Grammarly (as English is not my first language). So I write a whole paragraph, then I select the entire paragraph via visual selection with <span class='inlinecode'>SHIFT+v</span>, and then I press <span class='inlinecode'>,y</span> to yank the paragraph to the systems clipboard, then I paste the paragraph to Grammarly's browser window with <span class='inlinecode'>CTRL+v</span>, let Grammarly suggest the improvements, and then I copy the result back with <span class='inlinecode'>CTRL+c</span> to the system clipboard and in NeoVim I type <span class='inlinecode'>,i</span> to insert the result back overriding the old paragraph (which is still selected in visual mode) with the new content. That all sounds a bit complicated, but it's surprisingly natural and efficient.</span><br /> -<br /> -<span>To come back to the example, for the clipboard integration, I use this small VimScript snippet, and I didn't have to dig into any Lisp or Perl for this:</span><br /> -<br /> -<!-- Generator: GNU source-highlight 3.1.9 -by Lorenzo Bettini -http://www.lorenzobettini.it -http://www.gnu.org/software/src-highlite --> -<pre><i><font color="#9A1900">" Clipboard</font></i> -vnoremap ,<b><font color="#0000FF">y</font></b> !pbcopy<font color="#FF6600"><CR></font>ugv -vnoremap ,<b><font color="#0000FF">i</font></b> !pbpaste<font color="#FF6600"><CR></font> -nmap ,<b><font color="#0000FF">i</font></b> !wpbpaste<font color="#FF6600"><CR></font> -</pre> -<br /> -<span>That's only a very few lines and does precisely what I want. It's quick and dirty but get's the job done! If VimScript becomes too cumbersome, I can use Lua for NeoVim scripting.</span><br /> -<br /> -<h2 style='display: inline'>The famous Emacs Org mode</h2><br /> -<br /> -<span>Org-mode is an Emacs mode for keeping notes, authoring documents, computational notebooks, literate programming, maintaining to-do lists, planning projects, and more — in a fast and effective plain-text system. There's even a dedicated website for it:</span><br /> -<br /> -<a class='textlink' href='https://orgmode.org/'>https://orgmode.org/</a><br /> -<br /> -<span>In short, Org-mode is an "interactive markup language" that helps you organize everything mentioned above. I rarely touched the surface during my two-month experiment with Emacs, and I am impressed by it, so I see the benefits of having that. But it's not for me.</span><br /> -<br /> -<span>I use "Dead Tree Mode" to organize my work and notes. Dead tree? Yeah, I use an actual pen and a real paper journal (Leuchtturm or a Moleskine and a set of coloured <span class='inlinecode'>0.5</span> Muji Pens are excellent choices). That's far more immersive and flexible than a computer program can ever be. Yes, some automation and interaction with the computer (like calendar scheduling etc.) are missing. Still, an actual paper journal forces you to stay simple and focus on the actual work rather than tinkering with your computer program. (But I could not resist, and I wrote a VimScript which parses a table of contents page in Markdown format of my scanned paper journals, and NeoVim allows me to select a topic so that the corresponding PDF scan on the right journal page gets opened in an external PDF viewer (the PDF viewer is <span class='inlinecode'>zathura</span>, it uses Vi-keybindings, of course) :-). (See the appendix of this blog post for that script).</span><br /> -<br /> -<a class='textlink' href='https://pwmt.org/projects/zathura/'>Zathura</a><br /> -<br /> -<span>On the road, I also write some of my notes in Markdown format to NextCloud Notes, which is editable from my phone and via NeoVim on my computers. Markdown is much less powerful than Org-mode, but I prefer it the simple way. There's a neat terminal application, <span class='inlinecode'>ranger</span>, which I use to browse my NextCloud Notes when they are synced to a local folder on my machine. <span class='inlinecode'>ranger</span> is a file manager inspired by Vim and therefore makes use of Vim keybindings and it feels just natural to me. </span><br /> -<br /> -<a class='textlink' href='https://github.com/ranger/ranger'>Ranger - A Vim inspired file manager</a><br /> -<span>Did I mention that I also use my <span class='inlinecode'>zsh</span> (my default shell) and my <span class='inlinecode'>tmux</span> (terminal multiplexer) in Vi-mode?</span><br /> -<br /> -<a class='textlink' href='https://zsh.sourceforge.io/'>Z shell</a><br /> -<a class='textlink' href='https://github.com/tmux/tmux'>tmux terminal multiplexer</a><br /> -<br /> -<h2 style='display: inline'>Seeking simplicity</h2><br /> -<br /> -<span>I am not ready to dive deep into the whole world of Emacs. I prefer small and simple tools as opposed to complex tools. Emacs comes with many features out of the box, whereas in Vim/NeoVim, you would need to install many plugins to replicate some of the behaviour. Yes, I need to invest time managing all the Vim/NeoVim plugins I use, but I feel more in control compared to Doom Emacs, where a framework around vanilla Emacs manages all the plugins. I could use vanilla Emacs and manage all my plugins the vanilla way, but for me, it's not worth the effort to learn and dive into that as all that I want to do I can already do with Vim/NeoVim.</span><br /> -<br /> -<span>I am not saying that Vim/NeoVim are simple programs, but they are much simpler than Emacs with much smaller footprints; furthermore, they appear to be more straightforward as I am used to them. I only need Vim/NeoVim to be an editor, an IDE (through some plugins), and nothing more.</span><br /> -<br /> -<h2 style='display: inline'>Conclusion</h2><br /> -<br /> -<span>I understand the Emacs users now. Emacs is an incredibly powerful platform for almost everything, not just text editing. With Emacs, you can do nearly everything (Writing, editing, programming, calendar scheduling and note taking, Jira integration, playing games, listening to music, reading/writing emails, browsing the web, using as a calculator, generating HTML pages, configuring interactive menus, jumping around between every feature and every file within one single session, chat on IRC, surf the Gopherspace, ... the options are endless....). If you want to have one piece of software which rules it all and you are happy to invest a large part of your time in your platform: Pick Emacs, and over time Emacs will become "your" Emacs, customized to your own needs and change the way it works, which makes the Emacs users stick even more to it.</span><br /> -<br /> -<span>Vim/NeoVim also comes with a very high degree of customization options, but to a lesser extreme than Emacs (but still, a much higher degree than most other editors out there). If you want the best text editor in the world, which can also be tweaked to be a decent IDE, you are only looking for: Pick Vim or NeoVim! You would also need to invest a lot of time in learning, tweaking and customizing Vim/NeoVim, but that's a little more straightforward, and the result is much more lightweight once you get used to the "Vi way of doing things" you never would want to change back. I haven't tried the Emacs vanilla keystrokes, but they are terrible (that's probably one of the reasons why Doom Emacs uses Vim keybindings by default).</span><br /> -<br /> -<span class='quote'>Update: One reader recommended to have a look at NvChad. NvChad is a NeoVim config written in Lua aiming to provide a base configuration with very beautiful UI and blazing fast startuptime (around <span class='inlinecode'>0.02</span> secs ~ <span class='inlinecode'>0.07</span> secs). They tweak UI plugins such as telescope, nvim-tree, bufferline etc well to provide an aesthetic UI experience. That sounds interesting!</span><br /> -<br /> -<a class='textlink' href='https://github.com/NvChad/NvChad'>https://github.com/NvChad/NvChad</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> - <entry> - <title>Installing DTail on OpenBSD</title> - <link href="gemini://foo.zone/gemfeed/2022-10-30-installing-dtail-on-openbsd.gmi" /> - <id>gemini://foo.zone/gemfeed/2022-10-30-installing-dtail-on-openbsd.gmi</id> - <updated>2022-10-30T11:03:19+02:00</updated> - <author> - <name>Paul Buetow aka snonux</name> - <email>paul@dev.buetow.org</email> - </author> - <summary>This will be a quick blog post, as I am busy with my personal life now. I have relocated to a different country and am still busy arranging things. So bear with me :-)</summary> - <content type="xhtml"> - <div xmlns="http://www.w3.org/1999/xhtml"> - <h1 style='display: inline'>Installing DTail on OpenBSD</h1><br /> -<br /> -<span class='quote'>Published at 2022-10-30T11:03:19+02:00</span><br /> -<br /> -<pre> - ,_---~~~~~----._ - _,,_,*^____ _____``*g*\"*, -/ __/ /' ^. / \ ^@q f - @f | | | | 0 _/ -\`/ \~__((@/ __ \__((@/ \ - | _l__l_ I <--- The Go Gopher - } [______] I - ] | | | | - ] ~ ~ | - | | - | | - | | A ; -~~~~~~~~~~~~~~~~~~~~~~~~~~~~|~~~,--,-/ \---,-/|~~,~~~~~~~~~~~~~~~~~~~~~~~~~~~ - _|\,'. /| /| `/|-. - \`.' /| , `;. - ,'\ A A A A _ /| `.; - ,/ _ A _ / _ /| ; - /\ / \ , , A / / `/| - /_| | _ \ , , ,/ \ - // | |/ `.\ ,- , , ,/ ,/ \/ - / @| |@ / /' \ \ , > /| ,--. - |\_/ \_/ / | | , ,/ \ ./' __:.. - | __ __ | | | .--. , > > |-' / ` - ,/| / ' \ | | | \ , | / - / |<--.__,->| | | . `. > > / ( - /_,' \\ ^ / \ / / `. >-- /^\ | - \\___/ \ / / \__' \ \ \/ \ | - `. |/ , , /`\ \ ) - \ ' |/ , V \ / `-\ - OpenBSD Puffy ---> `|/ ' V V \ \.' \_ - '`-. V V \./'\ - `|/-. \ / \ /,---`\ kat - / `._____V_____V' - ' ' -</pre> -<br /> -<span>This will be a quick blog post, as I am busy with my personal life now. I have relocated to a different country and am still busy arranging things. So bear with me :-)</span><br /> -<br /> -<span> In this post, I want to give a quick overview (or how-to) about installing DTail on OpenBSD, as the official documentation only covers Red Hat and Fedora Linux! And this blog post will also be used as my reference!</span><br /> -<br /> -<a class='textlink' href='https://dtail.dev'>https://dtail.dev</a><br /> -<br /> -<span>I am using Rexify for my OpenBSD automation. Check out the following article covering my Rex setup in a little bit more detail:</span><br /> -<br /> -<a class='textlink' href='./2022-07-30-lets-encrypt-with-openbsd-and-rex.html'>Let's Encrypt with OpenBSD and Rex</a><br /> -<br /> -<span>I will also mention some relevant <span class='inlinecode'>Rexfile</span> snippets in this post!</span><br /> -<br /> -<h2 style='display: inline'>Compile it</h2><br /> -<br /> -<span>First of all, DTail needs to be downloaded and compiled. For that, <span class='inlinecode'>git</span>, <span class='inlinecode'>go</span>, and <span class='inlinecode'>gmake</span> are required:</span><br /> -<br /> -<pre> -$ doas pkg_add git go gmake -</pre> -<br /> -<span>I am happy that the Go Programming Language is readily available in the OpenBSD packaging system. Once the dependencies got installed, clone DTail and compile it:</span><br /> -<br /> -<pre> -$ mkdir git -$ cd git -$ git clone https://github.com/mimecast/dtail -$ cd dtail -$ gmake -</pre> -<br /> -<span>You can verify the version by running the following command:</span><br /> -<br /> -<pre> -$ ./dtail --version - DTail 4.1.0 Protocol 4.1 Have a lot of fun! -$ file dtail - dtail: ELF 64-bit LSB executable, x86-64, version 1 -</pre> -<br /> -<span>Now, there isn't any need anymore to keep <span class='inlinecode'>git</span>, <span class='inlinecode'>go</span> and <span class='inlinecode'>gmake</span>, so they can be deinstalled now:</span><br /> -<br /> -<pre> -$ doas pkg_delete git go gmake -</pre> -<br /> -<span>One day I shall create an official OpenBSD port for DTail.</span><br /> -<br /> -<h2 style='display: inline'>Install it</h2><br /> -<br /> -<span>Installing the binaries is now just a matter of copying them to <span class='inlinecode'>/usr/local/bin</span> as follows:</span><br /> -<br /> -<pre> -$ for bin in dserver dcat dgrep dmap dtail dtailhealth; do - doas cp -p $bin /usr/local/bin/$bin - doas chown root:wheel /usr/local/bin/$bin -done -</pre> -<br /> -<span>Also, we will be creating the <span class='inlinecode'>_dserver</span> service user:</span><br /> -<br /> -<pre> -$ doas adduser -class nologin -group _dserver -batch _dserver -$ doas usermod -d /var/run/dserver/ _dserver -</pre> -<br /> -<span>The OpenBSD init script is created from scratch (not part of the official DTail project). Run the following to install the bespoke script:</span><br /> -<br /> -<pre> -$ cat <<'END' | doas tee /etc/rc.d/dserver -#!/bin/ksh - -daemon="/usr/local/bin/dserver" -daemon_flags="-cfg /etc/dserver/dtail.json" -daemon_user="_dserver" - -. /etc/rc.d/rc.subr - -rc_reload=NO - -rc_pre() { - install -d -o _dserver /var/log/dserver - install -d -o _dserver /var/run/dserver/cache -} - -rc_cmd $1 & -END -$ doas chmod 755 /etc/rc.d/dserver -</pre> -<br /> -<h3 style='display: inline'>Rexification</h3><br /> -<br /> -<span>This is the task for setting it up via Rex. Note the <span class='inlinecode'>. . . .</span>, that's a placeholder which we will fill up more and more during this blog post:</span><br /> -<br /> -<pre> -desc 'Setup DTail'; -task 'dtail', group => 'frontends', - sub { - my $restart = FALSE; - - file '/etc/rc.d/dserver': - content => template('./etc/rc.d/dserver.tpl'), - owner => 'root', - group => 'wheel', - mode => '755', - on_change => sub { $restart = TRUE }; - - . - . - . - . - - service 'dserver' => 'restart' if $restart; - service 'dserver', ensure => 'started'; - }; -</pre> -<br /> -<h2 style='display: inline'>Configure it</h2><br /> -<br /> -<span>Now, DTail is fully installed but still needs to be configured. Grab the default config file from GitHub ...</span><br /> -<br /> -<pre> -$ doas mkdir /etc/dserver -$ curl https://raw.githubusercontent.com/mimecast/dtail/master/examples/dtail.json.examples | - doas tee /etc/dserver/dtail.json -</pre> -<br /> -<span>... and then edit it and adjust <span class='inlinecode'>LogDir</span> in the <span class='inlinecode'>Common</span> section to <span class='inlinecode'>/var/log/dserver</span>. The result will look like this:</span><br /> -<br /> -<pre> - "Common": { - "LogDir": "/var/log/dserver", - "Logger": "Fout", - "LogRotation": "Daily", - "CacheDir": "cache", - "SSHPort": 2222, - "LogLevel": "Info" - } -</pre> -<br /> -<h3 style='display: inline'>Rexification</h3><br /> -<br /> -<span>That's as simple as adding the following to the Rex task:</span><br /> -<br /> -<pre> -file '/etc/dserver', - ensure => 'directory'; - -file '/etc/dserver/dtail.json', - content => template('./etc/dserver/dtail.json.tpl'), - owner => 'root', - group => 'wheel', - mode => '755', - on_change => sub { $restart = TRUE }; -</pre> -<br /> -<h2 style='display: inline'>Update the key cache for it</h2><br /> -<br /> -<span>DTail relies on SSH for secure authentication and communication. However, the system user <span class='inlinecode'>_dserver</span> has no permission to read the SSH public keys from the user's home directories, so the DTail server also checks for available public keys in an alternative path <span class='inlinecode'>/var/run/dserver/cache</span>. </span><br /> -<br /> -<span>The following script, populating the DTail server key cache, can be run periodically via <span class='inlinecode'>CRON</span>:</span><br /> -<br /> -<pre> -$ cat <<'END' | doas tee /usr/local/bin/dserver-update-key-cache.sh -#!/bin/ksh - -CACHEDIR=/var/run/dserver/cache -DSERVER_USER=_dserver -DSERVER_GROUP=_dserver - -echo 'Updating SSH key cache' - -ls /home/ | while read remoteuser; do - keysfile=/home/$remoteuser/.ssh/authorized_keys - - if [ -f $keysfile ]; then - cachefile=$CACHEDIR/$remoteuser.authorized_keys - echo "Caching $keysfile -> $cachefile" - - cp $keysfile $cachefile - chown $DSERVER_USER:$DSERVER_GROUP $cachefile - chmod 600 $cachefile - fi -done - -# Cleanup obsolete public SSH keys -find $CACHEDIR -name \*.authorized_keys -type f | -while read cachefile; do - remoteuser=$(basename $cachefile | cut -d. -f1) - keysfile=/home/$remoteuser/.ssh/authorized_keys - - if [ ! -f $keysfile ]; then - echo 'Deleting obsolete cache file $cachefile' - rm $cachefile - fi -done - -echo 'All set...' -END -$ doas chmod 500 /usr/local/bin/dserver-update-key-cache.sh -</pre> -<br /> -<span>Note that the script above is a slight variation of the official DTail script. The official DTail one is a <span class='inlinecode'>bash</span> script, but on OpenBSD, there's <span class='inlinecode'>ksh</span>. I run it once daily by adding it to the <span class='inlinecode'>daily.local</span>:</span><br /> -<br /> -<pre> -$ echo /usr/local/bin/dserver-update-key-cache.sh | doas tee -a /etc/daily.local -/usr/local/bin/dserver-update-key-cache.sh -</pre> -<br /> -<h3 style='display: inline'>Rexification</h3><br /> -<br /> -<span>That's done by adding ...</span><br /> -<br /> -<pre> -file '/usr/local/bin/dserver-update-key-cache.sh', - content => template('./scripts/dserver-update-key-cache.sh.tpl'), - owner => 'root', - group => 'wheel', - mode => '500'; - -append_if_no_such_line '/etc/daily.local', '/usr/local/bin/dserver-update-key-cache.sh'; -</pre> -<br /> -<span>... to the Rex task!</span><br /> -<br /> -<h2 style='display: inline'>Start it</h2><br /> -<br /> -<span>Now, it's time to enable and start the DTail server:</span><br /> -<br /> -<pre> -$ sudo rcctl enable dserver -$ sudo rcctl start dserver -$ tail -f /var/log/dserver/*.log -INFO|1022-090634|Starting scheduled job runner after 2s -INFO|1022-090634|Starting continuous job runner after 2s -INFO|1022-090644|24204|stats.go:53|2|11|7|||MAPREDUCE:STATS|currentConnections=0|lifetimeConnections=0 -INFO|1022-090654|24204|stats.go:53|2|11|7|||MAPREDUCE:STATS|currentConnections=0|lifetimeConnections=0 -INFO|1022-090719|Starting server|DTail 4.1.0 Protocol 4.1 Have a lot of fun! -INFO|1022-090719|Generating private server RSA host key -INFO|1022-090719|Starting server -INFO|1022-090719|Binding server|0.0.0.0:2222 -INFO|1022-090719|Starting scheduled job runner after 2s -INFO|1022-090719|Starting continuous job runner after 2s -INFO|1022-090729|86050|stats.go:53|2|11|7|||MAPREDUCE:STATS|currentConnections=0|lifetimeConnections=0 -INFO|1022-090739|86050|stats.go:53|2|11|7|||MAPREDUCE:STATS|currentConnections=0|lifetimeConnect -. -. -. -Ctr+C -</pre> -<br /> -<span>As we don't want to wait until tomorrow, let's populate the key cache manually:</span><br /> -<br /> -<pre> -$ doas /usr/local/bin/dserver-update-key-cache.sh -Updating SSH key cache -Caching /home/_dserver/.ssh/authorized_keys -> /var/cache/dserver/_dserver.authorized_keys -Caching /home/admin/.ssh/authorized_keys -> /var/cache/dserver/admin.authorized_keys -Caching /home/failunderd/.ssh/authorized_keys -> /var/cache/dserver/failunderd.authorized_keys -Caching /home/git/.ssh/authorized_keys -> /var/cache/dserver/git.authorized_keys -Caching /home/paul/.ssh/authorized_keys -> /var/cache/dserver/paul.authorized_keys -Caching /home/rex/.ssh/authorized_keys -> /var/cache/dserver/rex.authorized_keys -All set... -</pre> -<br /> -<h2 style='display: inline'>Use it</h2><br /> -<br /> -<span>The DTail server is now ready to serve connections. You can use any DTail commands, such as <span class='inlinecode'>dtail</span>, <span class='inlinecode'>dgrep</span>, <span class='inlinecode'>dmap</span>, <span class='inlinecode'>dcat</span>, <span class='inlinecode'>dtailhealth</span>, to do so. Checkout out all the usage examples on the official DTail page.</span><br /> -<br /> -<span>I have installed DTail server this way on my personal OpenBSD frontends <span class='inlinecode'>blowfish</span>, and <span class='inlinecode'>fishfinger</span>, and the following command connects as user <span class='inlinecode'>rex</span> to both machines and greps the file <span class='inlinecode'>/etc/fstab</span> for the string <span class='inlinecode'>local</span>:</span><br /> -<br /> -<pre> -❯ ./dgrep -user rex -servers blowfish.buetow.org,fishfinger.buetow.org --regex local /etc/fstab -CLIENT|earth|WARN|Encountered unknown host|{blowfish.buetow.org:2222 0xc0000a00f0 0xc0000a61e0 [blowfish.buetow.org]:2222 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9ZnF/LAk14SgqCzk38yENVTNfqibcluMTuKx1u53cKSp2xwHWzy0Ni5smFPpJDIQQljQEJl14ZdXvhhjp1kKHxJ79ubqRtIXBlC0PhlnP8Kd+mVLLHYpH9VO4rnaSfHE1kBjWkI7U6lLc6ks4flgAgGTS5Bb7pLAjwdWg794GWcnRh6kSUEQd3SftANqQLgCunDcP2Vc4KR9R78zBmEzXH/OPzl/ANgNA6wWO2OoKKy2VrjwVAab6FW15h3Lr6rYIw3KztpG+UMmEj5ReexIjXi/jUptdnUFWspvAmzIl6kwzzF8ExVyT9D75JRuHvmxXKKjyJRxqb8UnSh2JD4JN [23.88.35.144]:2222 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9ZnF/LAk14SgqCzk38yENVTNfqibcluMTuKx1u53cKSp2xwHWzy0Ni5smFPpJDIQQljQEJl14ZdXvhhjp1kKHxJ79ubqRtIXBlC0PhlnP8Kd+mVLLHYpH9VO4rnaSfHE1kBjWkI7U6lLc6ks4flgAgGTS5Bb7pLAjwdWg794GWcnRh6kSUEQd3SftANqQLgCunDcP2Vc4KR9R78zBmEzXH/OPzl/ANgNA6wWO2OoKKy2VrjwVAab6FW15h3Lr6rYIw3KztpG+UMmEj5ReexIjXi/jUptdnUFWspvAmzIl6kwzzF8ExVyT9D75JRuHvmxXKKjyJRxqb8UnSh2JD4JN 0xc0000a2180} -CLIENT|earth|WARN|Encountered unknown host|{fishfinger.buetow.org:2222 0xc0000a0150 0xc000460110 [fishfinger.buetow.org]:2222 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNiikdL7+tWSN0rCaw1tOd9aQgeUFgb830V9ejkyJ5h93PKLCWZSMMCtiabc1aUeUZR//rZjcPHFLuLq/YC+Y3naYtGd6j8qVrcfG8jy3gCbs4tV9SZ9qd5E24mtYqYdGlee6JN6kEWhJxFkEwPfNlG+YAr3KC8lvEAE2JdWvaZavqsqMvHZtAX3b25WCBf2HGkyLZ+d9cnimRUOt+/+353BQFCEct/2mhMVlkr4I23CY6Tsufx0vtxx25nbFdZias6wmhxaE9p3LiWXygPWGU5iZ4RSQSImQz4zyOc9rnJeP1rwGk0OWDJhdKNXuf0kIPdzMfwxv2otgY32/DJj6L [46.23.94.99]:2222 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNiikdL7+tWSN0rCaw1tOd9aQgeUFgb830V9ejkyJ5h93PKLCWZSMMCtiabc1aUeUZR//rZjcPHFLuLq/YC+Y3naYtGd6j8qVrcfG8jy3gCbs4tV9SZ9qd5E24mtYqYdGlee6JN6kEWhJxFkEwPfNlG+YAr3KC8lvEAE2JdWvaZavqsqMvHZtAX3b25WCBf2HGkyLZ+d9cnimRUOt+/+353BQFCEct/2mhMVlkr4I23CY6Tsufx0vtxx25nbFdZias6wmhxaE9p3LiWXygPWGU5iZ4RSQSImQz4zyOc9rnJeP1rwGk0OWDJhdKNXuf0kIPdzMfwxv2otgY32/DJj6L 0xc0000a2240} -Encountered 2 unknown hosts: 'blowfish.buetow.org:2222,fishfinger.buetow.org:2222' -Do you want to trust these hosts?? (y=yes,a=all,n=no,d=details): a -CLIENT|earth|INFO|STATS:STATS|cgocalls=11|cpu=8|connected=2|servers=2|connected%=100|new=2|throttle=0|goroutines=19 -CLIENT|earth|INFO|Added hosts to known hosts file|/home/paul/.ssh/known_hosts -REMOTE|blowfish|100|7|fstab|31bfd9d9a6788844.h /usr/local ffs rw,wxallowed,nodev 1 2 -REMOTE|fishfinger|100|7|fstab|093f510ec5c0f512.h /usr/local ffs rw,wxallowed,nodev 1 2 -</pre> -<br /> -<span>Running it the second time, and given that you trusted the keys the first time, it won't prompt you for the host keys anymore:</span><br /> -<br /> -<pre> -❯ ./dgrep -user rex -servers blowfish.buetow.org,fishfinger.buetow.org --regex local /etc/fstab -REMOTE|blowfish|100|7|fstab|31bfd9d9a6788844.h /usr/local ffs rw,wxallowed,nodev 1 2 -REMOTE|fishfinger|100|7|fstab|093f510ec5c0f512.h /usr/local ffs rw,wxallowed,nodev 1 2 -</pre> -<br /> -<h2 style='display: inline'>Conclusions</h2><br /> -<br /> -<span>It's a bit of manual work, but it's ok on this small scale! I shall invest time in creating an official OpenBSD port, though. That would render most of the manual steps obsolete, as outlined in this post!</span><br /> -<br /> -<span>Check out the following for more information:</span><br /> -<br /> -<a class='textlink' href='https://dtail.dev'>https://dtail.dev</a><br /> -<a class='textlink' href='https://github.com/mimecast/dtail'>https://github.com/mimecast/dtail</a><br /> -<a class='textlink' href='https://www.rexify.org'>https://www.rexify.org</a><br /> -<br /> -<span>Other related posts are:</span><br /> -<br /> -<a class='textlink' href='./2021-04-22-dtail-the-distributed-log-tail-program.html'>2021-04-22 DTail - The distributed log tail program</a><br /> -<a class='textlink' href='./2022-03-06-the-release-of-dtail-4.0.0.html'>2022-03-06 The release of DTail 4.0.0</a><br /> -<a class='textlink' href='./2022-10-30-installing-dtail-on-openbsd.html'>2022-10-30 Installing DTail on OpenBSD (You are currently reading this)</a><br /> -<br /> -<span>E-Mail your comments to <span class='inlinecode'>foo@paul.cyou</span> :-)</span><br /> -<br /> -<a class='textlink' href='../'>Back to the main site</a><br /> - </div> - </content> - </entry> diff --git a/gemfeed/dtail-usage-examples/dcat.gif b/gemfeed/dtail-usage-examples/dcat.gif Binary files differnew file mode 100644 index 00000000..a5b9369d --- /dev/null +++ b/gemfeed/dtail-usage-examples/dcat.gif diff --git a/gemfeed/dtail-usage-examples/dgrep.gif b/gemfeed/dtail-usage-examples/dgrep.gif Binary files differnew file mode 100644 index 00000000..e5314604 --- /dev/null +++ b/gemfeed/dtail-usage-examples/dgrep.gif diff --git a/gemfeed/dtail-usage-examples/dmap.gif b/gemfeed/dtail-usage-examples/dmap.gif Binary files differnew file mode 100644 index 00000000..d2701038 --- /dev/null +++ b/gemfeed/dtail-usage-examples/dmap.gif diff --git a/gemfeed/dtail-usage-examples/dtail-map.gif b/gemfeed/dtail-usage-examples/dtail-map.gif Binary files differnew file mode 100644 index 00000000..0bcfb156 --- /dev/null +++ b/gemfeed/dtail-usage-examples/dtail-map.gif diff --git a/gemfeed/dtail-usage-examples/dtail-map2.gif b/gemfeed/dtail-usage-examples/dtail-map2.gif Binary files differnew file mode 100644 index 00000000..1220b732 --- /dev/null +++ b/gemfeed/dtail-usage-examples/dtail-map2.gif diff --git a/gemfeed/dtail-usage-examples/dtail.gif b/gemfeed/dtail-usage-examples/dtail.gif Binary files differnew file mode 100644 index 00000000..24a3eb35 --- /dev/null +++ b/gemfeed/dtail-usage-examples/dtail.gif diff --git a/gemfeed/dtail-usage-examples/testing.gif b/gemfeed/dtail-usage-examples/testing.gif Binary files differnew file mode 100644 index 00000000..696921d2 --- /dev/null +++ b/gemfeed/dtail-usage-examples/testing.gif diff --git a/gemfeed/index.gmi b/gemfeed/index.gmi index 3831178a..c9d31aeb 100644 --- a/gemfeed/index.gmi +++ b/gemfeed/index.gmi @@ -2,6 +2,7 @@ ## To be in the .zone! +=> ./2023-09-25-dtail-usage-examples.gmi 2023-09-25 - DTail usage examples => ./2023-08-20-site-reliability-engineering-part-3.gmi 2023-08-20 - Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect => ./2023-08-19-site-reliability-engineering-part-2.gmi 2023-08-19 - Site Reliability Engineering - Part 2: Operational Balance in SRE => ./2023-08-18-site-reliability-engineering-part-1.gmi 2023-08-18 - Site Reliability Engineering - Part 1: SRE and Organizational Culture @@ -1,6 +1,6 @@ # foo.zone -> This site was generated at 2023-08-27T13:59:36+03:00 by `Gemtexter` +> This site was generated at 2023-09-25T15:14:22+03:00 by `Gemtexter` ``` |\---/| @@ -33,6 +33,7 @@ If you reach this site via the modern web, please read this: ### Posts +=> ./gemfeed/2023-09-25-dtail-usage-examples.gmi 2023-09-25 - DTail usage examples => ./gemfeed/2023-08-20-site-reliability-engineering-part-3.gmi 2023-08-20 - Site Reliability Engineering - Part 3: On-Call Culture and the Human Aspect => ./gemfeed/2023-08-19-site-reliability-engineering-part-2.gmi 2023-08-19 - Site Reliability Engineering - Part 2: Operational Balance in SRE => ./gemfeed/2023-08-18-site-reliability-engineering-part-1.gmi 2023-08-18 - Site Reliability Engineering - Part 1: SRE and Organizational Culture diff --git a/uptime-stats.gmi b/uptime-stats.gmi index 901610af..1f474b08 100644 --- a/uptime-stats.gmi +++ b/uptime-stats.gmi @@ -1,6 +1,6 @@ # My machine uptime stats -> This site was last updated at 2023-08-27T13:59:36+03:00 +> This site was last updated at 2023-09-25T15:14:22+03:00 The following stats were collected via `uptimed` on all of my personal computers over many years and the output was generated by `guprecords`, the global uptime records stats analyser of mine. @@ -27,14 +27,14 @@ Boots is the total number of host boots over the entire lifespan. | 4. | callisto | 153 | | 5. | dionysus | 136 | | 6. | tauceti-e | 120 | -| 7. | *earth | 106 | +| 7. | *earth | 107 | | 8. | pluto | 51 | | 9. | *mega15289 | 50 | | 10. | makemake | 50 | -| 11. | phobos | 40 | -| 12. | mega8477 | 40 | -| 13. | sun | 33 | -| 14. | *t450 | 30 | +| 11. | *t450 | 45 | +| 12. | phobos | 40 | +| 13. | mega8477 | 40 | +| 14. | sun | 33 | | 15. | vulcan | 19 | | 16. | *blowfish | 19 | | 17. | tauceti | 16 | @@ -59,15 +59,15 @@ Uptime is the total uptime of a host over the entire lifespan. | 5. | deltavega | 3 years, 1 months, 21 days | | 6. | pluto | 2 years, 10 months, 29 days | | 7. | tauceti | 2 years, 3 months, 19 days | -| 8. | *mega15289 | 1 years, 10 months, 11 days | +| 8. | *mega15289 | 1 years, 10 months, 14 days | | 9. | tauceti-f | 1 years, 9 months, 18 days | -| 10. | *earth | 1 years, 9 months, 9 days | -| 11. | *blowfish | 1 years, 8 months, 30 days | +| 10. | *earth | 1 years, 9 months, 16 days | +| 11. | *blowfish | 1 years, 9 months, 6 days | | 12. | mega8477 | 1 years, 3 months, 25 days | | 13. | host0 | 1 years, 3 months, 9 days | | 14. | tauceti-e | 1 years, 2 months, 20 days | | 15. | makemake | 1 years, 1 months, 25 days | -| 16. | *fishfinger | 0 years, 12 months, 27 days | +| 16. | *fishfinger | 1 years, 1 months, 3 days | | 17. | callisto | 0 years, 10 months, 31 days | | 18. | alphacentauri | 0 years, 10 months, 28 days | | 19. | london | 0 years, 9 months, 16 days | @@ -93,16 +93,16 @@ Score is calculated by combining all other metrics. | 8. | dionysus | 156 | | 9. | tauceti | 141 | | 10. | *mega15289 | 130 | -| 11. | *earth | 124 | -| 12. | *blowfish | 109 | +| 11. | *earth | 125 | +| 12. | *blowfish | 110 | | 13. | tauceti-f | 108 | | 14. | tauceti-e | 96 | | 15. | makemake | 90 | | 16. | callisto | 86 | | 17. | mega8477 | 80 | | 18. | host0 | 76 | -| 19. | mars | 67 | -| 20. | *fishfinger | 65 | +| 19. | *fishfinger | 67 | +| 20. | mars | 67 | +-----+----------------+-------+ ``` @@ -122,8 +122,8 @@ Downtime is the total downtime of a host over the entire lifespan. | 6. | mars | 1 years, 2 months, 10 days | | 7. | tauceti-e | 0 years, 12 months, 9 days | | 8. | *mega15289 | 0 years, 9 months, 23 days | -| 9. | sirius | 0 years, 8 months, 20 days | -| 10. | *t450 | 0 years, 6 months, 18 days | +| 9. | *t450 | 0 years, 9 months, 17 days | +| 10. | sirius | 0 years, 8 months, 20 days | | 11. | *earth | 0 years, 5 months, 24 days | | 12. | deimos | 0 years, 5 months, 15 days | | 13. | joghurt | 0 years, 2 months, 9 days | @@ -153,14 +153,14 @@ Lifespan is the total uptime + the total downtime of a host. | 6. | uugrn | 3 years, 5 months, 5 days | | 7. | deltavega | 3 years, 1 months, 21 days | | 8. | pluto | 2 years, 10 months, 30 days | -| 9. | *mega15289 | 2 years, 7 months, 2 days | +| 9. | *mega15289 | 2 years, 7 months, 5 days | | 10. | makemake | 2 years, 4 months, 19 days | | 11. | tauceti | 2 years, 3 months, 22 days | | 12. | callisto | 2 years, 3 months, 13 days | -| 13. | *earth | 2 years, 1 months, 31 days | +| 13. | *earth | 2 years, 2 months, 7 days | | 14. | tauceti-e | 2 years, 1 months, 29 days | | 15. | tauceti-f | 1 years, 9 months, 20 days | -| 16. | *blowfish | 1 years, 8 months, 31 days | +| 16. | *blowfish | 1 years, 9 months, 7 days | | 17. | mars | 1 years, 8 months, 19 days | | 18. | host0 | 1 years, 4 months, 10 days | | 19. | mega8477 | 1 years, 4 months, 1 days | @@ -181,8 +181,8 @@ Boots is the total number of host boots over the entire lifespan. | 3. | *Linux 5... | 249 | | 4. | Linux 4... | 164 | | 5. | FreeBSD 11... | 153 | -| 6. | *FreeBSD 13... | 128 | -| 7. | *Linux 6... | 50 | +| 6. | *FreeBSD 13... | 143 | +| 7. | *Linux 6... | 51 | | 8. | Darwin 13... | 40 | | 9. | *OpenBSD 7... | 40 | | 10. | FreeBSD 5... | 25 | @@ -209,21 +209,21 @@ Uptime is the total uptime of a host over the entire lifespan. +-----+----------------+------------------------------+ | 1. | Linux 3... | 15 years, 10 months, 25 days | | 2. | FreeBSD 10... | 5 years, 9 months, 9 days | -| 3. | *Linux 5... | 4 years, 4 months, 21 days | -| 4. | *OpenBSD 7... | 3 years, 4 months, 24 days | +| 3. | *Linux 5... | 4 years, 4 months, 28 days | +| 4. | *OpenBSD 7... | 3 years, 5 months, 8 days | | 5. | Linux 4... | 2 years, 8 months, 9 days | | 6. | FreeBSD 11... | 2 years, 4 months, 28 days | | 7. | Linux 2... | 1 years, 11 months, 21 days | | 8. | Darwin 13... | 1 years, 3 months, 25 days | | 9. | FreeBSD 6... | 1 years, 3 months, 9 days | -| 10. | *Linux 6... | 0 years, 10 months, 24 days | +| 10. | *Linux 6... | 0 years, 10 months, 31 days | | 11. | OpenBSD 4... | 0 years, 8 months, 12 days | | 12. | Darwin 21... | 0 years, 8 months, 9 days | | 13. | Darwin 18... | 0 years, 7 months, 18 days | | 14. | Darwin 15... | 0 years, 6 months, 15 days | -| 15. | *Darwin 22... | 0 years, 5 months, 22 days | +| 15. | *Darwin 22... | 0 years, 5 months, 25 days | | 16. | FreeBSD 5... | 0 years, 5 months, 18 days | -| 17. | *FreeBSD 13... | 0 years, 4 months, 3 days | +| 17. | *FreeBSD 13... | 0 years, 4 months, 7 days | | 18. | Darwin 20... | 0 years, 3 months, 21 days | | 19. | FreeBSD 7... | 0 years, 2 months, 5 days | | 20. | Darwin 19... | 0 years, 1 months, 9 days | @@ -240,19 +240,19 @@ Score is calculated by combining all other metrics. +-----+----------------+-------+ | 1. | Linux 3... | 1045 | | 2. | FreeBSD 10... | 406 | -| 3. | *Linux 5... | 295 | -| 4. | *OpenBSD 7... | 215 | +| 3. | *Linux 5... | 296 | +| 4. | *OpenBSD 7... | 217 | | 5. | Linux 4... | 178 | | 6. | FreeBSD 11... | 159 | | 7. | Linux 2... | 121 | | 8. | Darwin 13... | 80 | | 9. | FreeBSD 6... | 75 | -| 10. | *Linux 6... | 58 | -| 11. | OpenBSD 4... | 39 | -| 12. | Darwin 21... | 39 | +| 10. | *Linux 6... | 59 | +| 11. | Darwin 21... | 39 | +| 12. | OpenBSD 4... | 39 | | 13. | Darwin 18... | 35 | -| 14. | Darwin 15... | 29 | -| 15. | *FreeBSD 13... | 29 | +| 14. | *FreeBSD 13... | 31 | +| 15. | Darwin 15... | 29 | | 16. | *Darwin 22... | 28 | | 17. | FreeBSD 5... | 25 | | 18. | Darwin 20... | 14 | @@ -269,8 +269,8 @@ Boots is the total number of host boots over the entire lifespan. +-----+------------+-------+ | Pos | KernelName | Boots | +-----+------------+-------+ -| 1. | *Linux | 1035 | -| 2. | *FreeBSD | 877 | +| 1. | *Linux | 1036 | +| 2. | *FreeBSD | 892 | | 3. | *Darwin | 105 | | 4. | *OpenBSD | 50 | +-----+------------+-------+ @@ -281,14 +281,14 @@ Boots is the total number of host boots over the entire lifespan. Uptime is the total uptime of a host over the entire lifespan. ``` -+-----+------------+----------------------------+ -| Pos | KernelName | Uptime | -+-----+------------+----------------------------+ -| 1. | *Linux | 25 years, 6 months, 4 days | -| 2. | *FreeBSD | 9 years, 12 months, 5 days | -| 3. | *OpenBSD | 3 years, 12 months, 3 days | -| 4. | *Darwin | 3 years, 6 months, 16 days | -+-----+------------+----------------------------+ ++-----+------------+-----------------------------+ +| Pos | KernelName | Uptime | ++-----+------------+-----------------------------+ +| 1. | *Linux | 25 years, 6 months, 18 days | +| 2. | *FreeBSD | 9 years, 12 months, 8 days | +| 3. | *OpenBSD | 3 years, 12 months, 17 days | +| 4. | *Darwin | 3 years, 6 months, 19 days | ++-----+------------+-----------------------------+ ``` ## Top 20 Score's by KernelName @@ -299,10 +299,10 @@ Score is calculated by combining all other metrics. +-----+------------+-------+ | Pos | KernelName | Score | +-----+------------+-------+ -| 1. | *Linux | 1696 | -| 2. | *FreeBSD | 704 | -| 3. | *OpenBSD | 254 | -| 4. | *Darwin | 229 | +| 1. | *Linux | 1699 | +| 2. | *FreeBSD | 706 | +| 3. | *OpenBSD | 256 | +| 4. | *Darwin | 230 | +-----+------------+-------+ ``` |
