From 07c56086aa0c4e015c9044e333ae4001debcb28d Mon Sep 17 00:00:00 2001 From: Paul Buetow Date: Tue, 30 Apr 2024 13:14:09 +0300 Subject: Update content for html --- gemfeed/2023-09-25-dtail-usage-examples.html | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) (limited to 'gemfeed/2023-09-25-dtail-usage-examples.html') diff --git a/gemfeed/2023-09-25-dtail-usage-examples.html b/gemfeed/2023-09-25-dtail-usage-examples.html index 231cf3dc..f07fed00 100644 --- a/gemfeed/2023-09-25-dtail-usage-examples.html +++ b/gemfeed/2023-09-25-dtail-usage-examples.html @@ -8,7 +8,7 @@ -

DTail usage examples

Published at 2023-09-25T14:57:42+03:00

@@ -42,7 +42,7 @@

Use dmap to aggregate logs and other text files already written

dserver is the DTail server, where all the clients can connect to

Following logs

The following example demonstrates how to follow logs of several servers at once. The server list is provided as a flat text file. The example filters all records containing the string INFO. Any other Go compatible regular expression can also be used instead of INFO.

@@ -66,7 +66,7 @@ http://www.gnu.org/software/src-highlite -->

% dtail --servers serverlist.txt --grep INFO "/var/log/dserver/*.log"

Aggregating logs

To run ad-hoc map-reduce aggregations on newly written log lines you must add a query. The following example follows all remote log lines and prints out every few seconds the result to standard output.

@@ -123,7 +123,7 @@ http://www.gnu.org/software/src-highlite --> --query 'from STATS select ... outfile append result.csv'
-

How to use dcat

The following example demonstrates how to cat files (display the full content of the files) on several servers at once.

@@ -147,7 +147,7 @@ http://www.gnu.org/software/src-highlite -->

% dcat --servers serverlist.txt /etc/hostname

How to use dgrep

The following example demonstrates how to grep files (display only the lines which match a given regular expression) of multiple servers at once. In this example, we look after some entries in /etc/passwd. This time, we don't provide the server list via an file but rather via a comma separated list directly on the command line. We also explore the -before, -after and -max flags (see animation).

@@ -166,7 +166,7 @@ http://www.gnu.org/software/src-highlite -->
Hint: -regex is an alias for -grep.

-

How to use dmap

To run a map-reduce aggregation over logs written in the past, the dmap command can be used. The following example aggregates all map-reduce fields dmap will print interim results every few seconds. You can also write the result to an CSV file by adding outfile result.csv to the query.

@@ -184,7 +184,7 @@ http://www.gnu.org/software/src-highlite -->

How to use the DTail serverless mode

Until now, all examples so far required to have remote server(s) to connect to. That makes sense, as after all DTail is a *distributed* tool. However, there are circumstances where you don't really need to connect to a server remotely. For example, you already have a login shell open to the server an all what you want is to run some queries directly on local log files.

@@ -192,7 +192,7 @@ http://www.gnu.org/software/src-highlite -->
All commands shown so far also work in a serverless mode. All what needs to be done is to omit a server list. The DTail client then starts in serverless mode.

-

Serverless map-reduce query

The following dmap example is the same as the previously shown one, but the difference is that it operates on a local log file directly:

@@ -227,7 +227,7 @@ http://www.gnu.org/software/src-highlite --> lifetimeConnections group by $hostname order by max($cgocalls)'
-

Aggregating CSV files

In essence, this works exactly like aggregating logs. All files operated on must be valid CSV files and the first line of the CSV must be the header. E.g.:

@@ -249,7 +249,7 @@ Einstein,Albert
DMap can also be used to query and aggregate CSV files from remote servers.

-

Other serverless commands

The serverless mode works transparently with all other DTail commands. Here are some examples:

-- cgit v1.2.3