From 72e414b8da38e0284a9556b72a991f7b4621375d Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Fri, 30 Apr 2021 07:13:08 +0100 Subject: fix list --- content/gemtext/gemfeed/atom.xml | 6 +++--- content/html/contact-information.html | 1 + .../gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html | 3 +++ content/html/gemfeed/2021-04-24-welcome-to-the-geminispace.html | 1 + content/html/resources.html | 5 +++++ 5 files changed, 13 insertions(+), 3 deletions(-) (limited to 'content') diff --git a/content/gemtext/gemfeed/atom.xml b/content/gemtext/gemfeed/atom.xml index 68f1f276..845cb2bd 100644 --- a/content/gemtext/gemfeed/atom.xml +++ b/content/gemtext/gemfeed/atom.xml @@ -5,12 +5,12 @@ gemini://buetow.org - 2021-04-29T22:44:42+01:00 + 2021-04-30T07:13:01+01:00 Welcome to the Geminispace gemini://buetow.org/gemfeed/2021-04-24-welcome-to-the-geminispace.gmi - 2021-04-29T22:44:42+01:00 + 2021-04-30T07:13:01+01:00 Have you reached this article already via Gemini? You need a special client for that, web browsers such as Firefox, Chrome, Safari etc. don't support the Gemini protocol. The Gemini address of this site (or the address of this capsule as people say in Geminispace) is: Paul Buetow @@ -21,7 +21,7 @@ DTail - The distributed log tail program gemini://buetow.org/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.gmi - 2021-04-29T22:44:42+01:00 + 2021-04-30T07:13:01+01:00 This article first appeared at the Mimecast Engineering Blog but I made it available here in my personal Gemini capsule too. Paul Buetow diff --git a/content/html/contact-information.html b/content/html/contact-information.html index 0de6e0ee..ee397658 100644 --- a/content/html/contact-information.html +++ b/content/html/contact-information.html @@ -21,6 +21,7 @@ li { color: #98be65; }

Contact information

E-Mail

    +
  • Secure E-Mail: paul.buetow at protonmail dot com
  • E-Mail: paul at buetow dot org (forwards to the ProtonMail address)

Why did I just mention 2 E-Mail addresses here? The buetow.org address will always stay. It is my lifetime E-Mail address as I own the domain name. The address will stay even when I decided to change my email provider.

diff --git a/content/html/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html b/content/html/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html index 6c591fb6..99dcbd3c 100644 --- a/content/html/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html +++ b/content/html/gemfeed/2021-04-22-dtail-the-distributed-log-tail-program.html @@ -44,6 +44,7 @@ li { color: #98be65; }

The DTail family of commands

Following the UNIX philosophy, DTail includes multiple command-line commands each of them for a different purpose:

    +
  • dserver: The DTail server, the only binary required to be installed on the servers involved.
  • dtail: The distributed log tail client for following log files.
  • dcat: The distributed cat client for concatenating and displaying text files.
  • dgrep: The distributed grep client for searching text files for a regular expression pattern.
    • @@ -65,6 +66,7 @@ dtail –servers serverlist.txt –files ‘/var/log/*.log’ –regex ‘(?i:er

    Advanced features

    The features listed here are out of the scope of this blog post but are worthwhile to mention:

      +
    • Distributed map-reduce queries on stats provided in log files with dmap. dmap comes with its own SQL-like aggregation query language.
    • Stats streaming with continuous map-reduce queries. The difference to normal queries is that the stats are aggregated over a specified interval only on the newly written log lines. Thus, giving a de-facto live stat view for each interval.
    • Server-side scheduled queries on log files. The queries are configured in the DTail server configuration file and scheduled at certain time intervals. Results are written to CSV files. This is useful for generating daily stats from the log files without the need for an interactive client.
    • Server-side stats streaming with continuous map-reduce queries. This for example can be used to periodically generate stats from the logs at a configured interval, e.g., log error counts by the minute. These then can be sent to a time-series database (e.g., Graphite) and then plotted in a Grafana dashboard.
      • @@ -73,6 +75,7 @@ dtail –servers serverlist.txt –files ‘/var/log/*.log’ –regex ‘(?i:er

      For the future

      There are various features we want to see in the future.

        +
      • A spartan mode, not printing out any extra information but the raw remote log files would be a nice feature to have. This will make it easier to post-process the data produced by the DTail client with common UNIX tools. (To some degree this is possible already, just disable the ANSI terminal colour output of the client with -noColors and pipe the output to another program).
      • Tempting would be implementing the dgoawk command, a distributed version of the AWK programming language purely implemented in Go, for advanced text data stream processing capabilities. There are 3rd party libraries available implementing AWK in pure Go which could be used.
      • A more complex change would be the support of federated queries. You can connect to thousands of servers from a single client running on a laptop. But does it scale to 100k of servers? Some of the servers could be used as middleware for connecting to even more servers.
      • Another aspect is to extend the documentation. Especially the advanced features such as map-reduce query language and how to configure the server-side queries currently do require more documentation. For now, you can read the code, sample config files or just ask the author for that! But this will be certainly addressed in the future.
        • diff --git a/content/html/gemfeed/2021-04-24-welcome-to-the-geminispace.html b/content/html/gemfeed/2021-04-24-welcome-to-the-geminispace.html index 94508ee3..f95ff3af 100644 --- a/content/html/gemfeed/2021-04-24-welcome-to-the-geminispace.html +++ b/content/html/gemfeed/2021-04-24-welcome-to-the-geminispace.html @@ -38,6 +38,7 @@ li { color: #98be65; }

        As it is very easy to setup and maintain your own Gemini capsule (Gemini server + content composed via the Gemtext markup language) I decided to create my own. What I really like about Gemini is that I can just use my favourite text editor and get typing. I don't need to worry about the style and design of the presence and I also don't have to test anything in ten different web browsers. I can only focus on the content! As a matter of fact I am using the Vim editor + it's spellchecker + auto word completion functionality to write this.

        Advantages summarised

          +
        • Supports an alternative to the modern bloated web
        • Easy to operate and easy to write content
        • No need to worry about various web browser compatibilities
        • It's the client's responsibility how the content is designed+presented
          • diff --git a/content/html/resources.html b/content/html/resources.html index 6485da10..e823b9c9 100644 --- a/content/html/resources.html +++ b/content/html/resources.html @@ -25,6 +25,7 @@ li { color: #98be65; }

          You won't find any links on this site because over time the links will break. Please use your favourite search engine when you are interested in one of the resources...

          Technical books

            +
          • Effective awk programming; Arnold Robbins; O'Reilly
          • Advanced Bash-Scripting Guide; Not a book by per-se but could be
          • Java ist auch eine Insel; Christian Ullenboom;
          • Learn You a Haskell for Great Good!; Miran Lipovaca; No Starch Press
            • @@ -46,11 +47,13 @@ li { color: #98be65; }

            Technical bibles

            I didn't read them from the beginning to the end, but I am using them to look up things.

              +
            • The Linux Programming Interface; Michael Kerrisk; No Starch Press
            • Understanding the Linux Kernel; Daniel P. Bovet, Marco Cesati; O'Reilly
            • Algorithms; Robert Sedgewick, Kevin Wayne; Addison Wesley

            Self-development and soft-skills books

              +
            • Time Management for System Administrators; Thomas A. Limoncelli; O'Reilly
            • Who Moved My Cheese?; Dr. Spencer Johnson; Vermilion
            • The Bullet Journal Method; Ryder Carroll; Fourth Estate
            • Psycho-Cybernetics; Maxwell Maltz; Perigee Books
              • @@ -70,6 +73,7 @@ li { color: #98be65; }

              Technical video lectures and courses

              Some of these were in-person with exams, others were online learning lectures only.

                +
              • Linux Security and Isolation APIs Training; Michael Kerrisk; 3 day on-site training
              • The Ultimate Kubernetes Bootcamp; School of Devops; O'Reilly Online
              • Ultimate Go Programming; Bill Kennedy; O'Reilly Online
              • Scripting Vim; Damian Conway; O'Reilly Online
                • @@ -86,6 +90,7 @@ li { color: #98be65; }

                I have met many self-taught IT professionals I highly respect. In my own opinion a formal degree does not automatically qualify a person for a certain job. It is more about how you educate yourself further *after* formal education. The pragmatic way of thinking and getting things done do not require a college or university degree.

                However, I still believe a degree in Computer Science helps to achieve a good understanding of all the theory involved which you would have never learned about otherwise. Isn't it cool to understand how compiler work under the hood (automata theory) even if in your current position you are not required to hack the compiler? You could apply the same theory for other things also. This was just *one* example.

                  +
                • Student Exchange; I lived 1 year abroad and went to a US high school.
                • German School Majors (Abitur), focus areas: German and Mathematics
                • Practical working semester; I spent half a year as a C/C++ programmer abroad in Sofia, Bulgaria
                • Diplom-Inform. (FH); Aachen University of Applied Sciences
                  • -- cgit v1.2.3