diff options
Diffstat (limited to 'gemfeed/2023-09-25-dtail-usage-examples.md')
| -rw-r--r-- | gemfeed/2023-09-25-dtail-usage-examples.md | 82 |
1 files changed, 82 insertions, 0 deletions
diff --git a/gemfeed/2023-09-25-dtail-usage-examples.md b/gemfeed/2023-09-25-dtail-usage-examples.md index 8df87a93..0b289eed 100644 --- a/gemfeed/2023-09-25-dtail-usage-examples.md +++ b/gemfeed/2023-09-25-dtail-usage-examples.md @@ -80,6 +80,88 @@ To run ad-hoc map-reduce aggregations on newly written log lines you must add a Beware: For map-reduce queries to work, you have to ensure that DTail supports your log format. Check out the documentaiton of the DTail query language and the DTail log formats on the DTail homepage for more information. +# DTail usage examples + +> Published at 2023-09-25T14:57:42+03:00 + +Hey there. As I am pretty busy this month personally (I am now on Paternity Leave) and as I still want to post once monthly, the blog post of this month will only be some DTail usage examples. They're from the DTail documentation, but not all readers of my blog may be aware of those! + +DTail is a distributed DevOps tool for tailing, grepping, catting logs and other text files on many remote machines at once which I programmed in Go. + +[https://dtail.dev](https://dtail.dev) + +``` + ,_---~~~~~----._ + _,,_,*^____ _____``*g*\"*, + ____ _____ _ _ / __/ /' ^. / \ ^@q f + | _ \_ _|_ _(_) | @f | ((@| |@)) l 0 _/ + | | | || |/ _` | | | \`/ \~____ / __ \_____/ \ + | |_| || | (_| | | | | _l__l_ I + |____/ |_|\__,_|_|_| } [______] I + ] | | | | + ] ~ ~ | + | Let's tail those logs! | + | | +``` + +## Table of Contents + +* [⇢ DTail usage examples](#dtail-usage-examples) +* [⇢ ⇢ Commands](#commands) +* [⇢ ⇢ Following logs](#following-logs) +* [⇢ ⇢ Aggregating logs](#aggregating-logs) +* [⇢ ⇢ How to use `dcat`](#how-to-use-dcat) +* [⇢ ⇢ How to use `dgrep`](#how-to-use-dgrep) +* [⇢ ⇢ How to use `dmap`](#how-to-use-dmap) +* [⇢ ⇢ How to use the DTail serverless mode](#how-to-use-the-dtail-serverless-mode) +* [⇢ ⇢ ⇢ Serverless map-reduce query](#serverless-map-reduce-query) +* [⇢ ⇢ ⇢ Aggregating CSV files](#aggregating-csv-files) +* [⇢ ⇢ ⇢ Other serverless commands](#other-serverless-commands) + +## Commands + +DTail consists out of a server and several client binaries. In this post, I am showcasing their use! + +* Use `dtail` to follow logs +* Use `dtail` to aggregate logs while they are followed +* Use `dcat` to display logs and other text files already written +* Use `dgrep` to grep (search) logs and other text files already written +* Use `dmap` to aggregate logs and other text files already written +* `dserver` is the DTail server, where all the clients can connect to + +## Following logs + +The following example demonstrates how to follow logs of several servers at once. The server list is provided as a flat text file. The example filters all records containing the string `INFO`. Any other Go compatible regular expression can also be used instead of `INFO`. + +```shell +% dtail --servers serverlist.txt --grep INFO --files "/var/log/dserver/*.log" +``` + +Hint: you can also provide a comma separated server list, e.g.: `servers server1.example.org,server2.example.org:PORT,...` + +[](./dtail-usage-examples/dtail.gif) + +> Hint: You can also use the shorthand version (omitting the `--files`) + +```shell +% dtail --servers serverlist.txt --grep INFO "/var/log/dserver/*.log" +``` + +## Aggregating logs + +To run ad-hoc map-reduce aggregations on newly written log lines you must add a query. The following example follows all remote log lines and prints out every few seconds the result to standard output. + +> Hint: To run a map-reduce query across log lines written in the past, please use the `dmap` command instead. + +```shell +% dtail --servers serverlist.txt \ + --files '/var/log/dserver/*.log' \ + --query 'from STATS select sum($goroutines),sum($cgocalls), + last($time),max(lifetimeConnections)' +``` + +Beware: For map-reduce queries to work, you have to ensure that DTail supports your log format. Check out the documentaiton of the DTail query language and the DTail log formats on the DTail homepage for more information. + [](./dtail-usage-examples/dtail-map.gif) > Hint: You can also use the shorthand version: |