1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
|
# Operating Modes
Epimetheus has five modes. Backend support:
| Mode | Prometheus (Pushgateway) | Prometheus (Remote Write) | ClickHouse |
|-----------|--------------------------|---------------------------|------------|
| Realtime | Yes | No | No |
| Historic | No | Yes | No |
| Backfill | No | Yes | No |
| Auto | Yes (samples < 5 min) | Yes (samples ≥ 5 min) | No |
| Watch | Optional | Optional | Optional |
At least one of Prometheus or ClickHouse must be configured for watch mode.
---
## Watch Mode
Monitor CSV files and push metrics using file modification time as the timestamp. Works with any tabular CSV; numeric columns become metrics, string columns become labels.
### Watch mode data flow
```
┌─────────────────┐ poll (1s) ┌─────────────────────────────────────┐
│ CSV file(s) │ ─────────────────▶ │ Epimetheus (watch mode) │
│ │ │ • Parse tabular CSV │
│ File mtime = │ │ • Numeric columns → metrics │
│ sample time │ │ • String columns → labels │
└─────────────────┘ │ • Optional DNS resolution (IPs) │
└─────────────────────────────────────┘
│
┌────────────────────┼────────────────────┐
│ │ │
▼ ▼ │
┌───────────────┐ ┌───────────────┐ │
│ Prometheus │ │ ClickHouse │ │
│ (optional) │ │ (optional) │ │
│ │ │ │ │
│ Remote Write │ │ HTTP insert │ │
│ /api/v1/write│ │ (batched) │ │
└───────────────┘ └───────────────┘ │
│ │ │
└────────────────────┴────────────────────┘
At least one of -prometheus or -clickhouse
```
```bash
./epimetheus -mode=watch -file=mydata.csv -metric-name=myapp \
-prometheus=http://localhost:9090/api/v1/write
```
**Options:** `-file`, `-metric-name`, `-prometheus`, `-clickhouse`, `-clickhouse-table`, `-job`, `-resolve-ip-labels`. See [CLI Reference](../reference/cli.md).
**Features:** Format-agnostic CSV, automatic numeric/string detection, label name sanitization, optional DNS resolution for IP labels, timestamp from file mtime, continuous polling (1s), Remote Write (and optionally ClickHouse). See [CSV Format Flexibility](csv-format-flexibility.md) and [DNS Resolution](dns-resolution.md).
---
## Realtime Mode (default)
Push current metrics to Pushgateway with "now" timestamp.
```bash
./epimetheus -mode=realtime -continuous
```
**Options:** `-pushgateway` (default `http://localhost:9091`), `-job`, `-continuous`. Pushes every 15 seconds when `-continuous` is set.
---
## Historic Mode
Push a single historic datapoint via Remote Write.
```bash
./epimetheus -mode=historic -hours-ago=24
```
**Options:** `-prometheus` (default `http://localhost:9090/api/v1/write`), `-hours-ago` (default 24). Requires Remote Write receiver. See [Backends: Prometheus](../backends/prometheus.md).
---
## Backfill Mode
Import a range of historic data points.
```bash
./epimetheus -mode=backfill -start-hours=48 -end-hours=0 -interval=1
./epimetheus -mode=backfill -start-hours=168 -end-hours=0 -interval=6
```
**Options:** `-start-hours`, `-end-hours` (0 = now), `-interval` (hours between points). Requires Remote Write receiver.
---
## Auto Mode
Route samples by timestamp age: < 5 minutes → Pushgateway; ≥ 5 minutes → Remote Write. Use for mixed or unknown-age data.
### Auto mode data flow
```
┌─────────────────┐ ┌─────────────────────────────────────┐
│ CSV/JSON file │ ─────────────────▶ │ Epimetheus (auto mode) │
│ (per-sample │ │ • Parse file (csv or json) │
│ timestamps) │ │ • Route by sample age: │
└─────────────────┘ │ < 5 min → Pushgateway │
│ ≥ 5 min → Remote Write │
└─────────────────────────────────────┘
│
┌────────────────────┴────────────────────┐
▼ ▼ │
┌───────────────┐ ┌───────────────┐ │
│ Pushgateway │ │ Prometheus │ │
│ (realtime │ │ Remote Write │ │
│ samples) │ │ (historic │ │
└───────┬───────┘ │ samples) │ │
│ └───────────────┘ │
│ Scraped by Prometheus │
▼ │
┌───────────────┐ │
│ Prometheus │◀──────────────────────────────────┘
└───────────────┘
```
```bash
./scripts/generate-test-data.sh
./epimetheus -mode=auto -file=test-all-ages.csv
```
**Options:** `-file`, `-format` (csv or json), `-pushgateway`, `-prometheus`. See [Data Formats](data-formats.md).
|
