summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorPaul Buetow <paul@buetow.org>2025-07-04 13:06:37 +0300
committerPaul Buetow <paul@buetow.org>2025-07-04 13:06:37 +0300
commit95fec10b3b86f3cce7b828cc221f459fbee99748 (patch)
tree7d3cc256dd865d4b26bf3089dd504c4ef9b61cd8
parent69d0643e872d0db18582e22721716a03ee8efc96 (diff)
cleanup
-rw-r--r--benchmarks/baselines/baseline_20250704_113229_turbo-mode-enabled.txt31
-rw-r--r--benchmarks/baselines/turbo_mode_performance_analysis_20250704.md80
2 files changed, 0 insertions, 111 deletions
diff --git a/benchmarks/baselines/baseline_20250704_113229_turbo-mode-enabled.txt b/benchmarks/baselines/baseline_20250704_113229_turbo-mode-enabled.txt
deleted file mode 100644
index b138efc..0000000
--- a/benchmarks/baselines/baseline_20250704_113229_turbo-mode-enabled.txt
+++ /dev/null
@@ -1,31 +0,0 @@
-Git commit: 0645644
-Date: 2025-07-04T11:32:29+03:00
-Tag: turbo-mode-enabled
-----------------------------------------
-goos: linux
-goarch: amd64
-pkg: github.com/mimecast/dtail/benchmarks
-cpu: 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
-BenchmarkAll/DCat/Size=10MB-8 2 536659169 ns/op 7.484 MB/sec 132338 lines/sec 12614252 B/op 256 allocs/op
-BenchmarkAll/DCat/Size=100MB-8 2 651439332 ns/op 62.30 MB/sec 1099819 lines/sec 175814904 B/op 253 allocs/op
-BenchmarkAll/DCat/Size=1GB-8 1 1840237610 ns/op 228.5 MB/sec 4043858 lines/sec 1498069128 B/op 414 allocs/op
-BenchmarkAll/DGrep/Size=10MB/HitRate=1%-8 2 532565906 ns/op 7.550 MB/sec 1.000 hit_rate_% 133722 lines/sec 15645 matched_lines 3230432 B/op 248 allocs/op
-BenchmarkAll/DGrep/Size=10MB/HitRate=10%-8 2 530343878 ns/op 7.593 MB/sec 10.00 hit_rate_% 133277 lines/sec 28087 matched_lines 6213984 B/op 253 allocs/op
-BenchmarkAll/DGrep/Size=10MB/HitRate=50%-8 2 533980826 ns/op 7.852 MB/sec 50.00 hit_rate_% 132045 lines/sec 46246 matched_lines 11799088 B/op 250 allocs/op
-BenchmarkAll/DGrep/Size=10MB/HitRate=90%-8 2 537646208 ns/op 8.027 MB/sec 90.00 hit_rate_% 130922 lines/sec 59955 matched_lines 12852384 B/op 251 allocs/op
-BenchmarkAll/DGrep/Size=100MB/HitRate=1%-8 2 598815338 ns/op 67.02 MB/sec 1.000 hit_rate_% 1186247 lines/sec 125943 matched_lines 25458536 B/op 249 allocs/op
-BenchmarkAll/DGrep/Size=100MB/HitRate=10%-8 2 602807614 ns/op 66.34 MB/sec 10.00 hit_rate_% 1171951 lines/sec 167812 matched_lines 45144824 B/op 254 allocs/op
-BenchmarkAll/DGrep/Size=100MB/HitRate=50%-8 2 644424441 ns/op 65.85 MB/sec 50.00 hit_rate_% 1113050 lines/sec 405455 matched_lines 96499720 B/op 253 allocs/op
-BenchmarkAll/DGrep/Size=100MB/HitRate=90%-8 2 667234947 ns/op 66.48 MB/sec 90.00 hit_rate_% 1077621 lines/sec 643507 matched_lines 181421916 B/op 251 allocs/op
-BenchmarkAll/DGrep/Size=1GB/HitRate=1%-8 1 1521693394 ns/op 282.4 MB/sec 1.000 hit_rate_% 4978395 lines/sec 2432992 matched_lines 721057832 B/op 413 allocs/op
-BenchmarkAll/DGrep/Size=1GB/HitRate=10%-8 1 1524571502 ns/op 284.7 MB/sec 10.00 hit_rate_% 4973241 lines/sec 2289850 matched_lines 716478488 B/op 412 allocs/op
-BenchmarkAll/DGrep/Size=1GB/HitRate=50%-8 1 1692914684 ns/op 265.2 MB/sec 50.00 hit_rate_% 4534246 lines/sec 3580903 matched_lines 821041224 B/op 413 allocs/op
-BenchmarkAll/DGrep/Size=1GB/HitRate=90%-8 1 2089761841 ns/op 228.7 MB/sec 90.00 hit_rate_% 3722796 lines/sec 6441361 matched_lines 1596053624 B/op 414 allocs/op
-BenchmarkAll/DMap/Size=10MB/Query=count-8 3 353431127 ns/op 22.00 MB/sec 199258 records/sec 53101 B/op 181 allocs/op
-BenchmarkAll/DMap/Size=10MB/Query=sum_avg-8 3 355487946 ns/op 21.78 MB/sec 197464 records/sec 53021 B/op 180 allocs/op
-BenchmarkAll/DMap/Size=10MB/Query=min_max-8 3 356909472 ns/op 21.76 MB/sec 197145 records/sec 54045 B/op 183 allocs/op
-BenchmarkAll/DMap/Size=10MB/Query=multi-8 3 358687037 ns/op 21.63 MB/sec 195822 records/sec 53064 B/op 179 allocs/op
-BenchmarkAll/DMap/Size=100MB/Query=count-8 1 1877888370 ns/op 41.38 MB/sec 374800 records/sec 131480 B/op 394 allocs/op
-BenchmarkAll/DMap/Size=100MB/Query=sum_avg-8 1 2037178564 ns/op 38.04 MB/sec 344616 records/sec 130256 B/op 392 allocs/op
-BenchmarkAll/DMap/Size=100MB/Query=min_max-8 1 2177091489 ns/op 35.54 MB/sec 322577 records/sec 129168 B/op 392 allocs/op
-BenchmarkAll/DMap/Size=100MB/Query=multi-8 1 1949052172 ns/op 39.85 MB/sec 361489 records/sec 129144 B/op 391 allocs/op
diff --git a/benchmarks/baselines/turbo_mode_performance_analysis_20250704.md b/benchmarks/baselines/turbo_mode_performance_analysis_20250704.md
deleted file mode 100644
index 5fa23ef..0000000
--- a/benchmarks/baselines/turbo_mode_performance_analysis_20250704.md
+++ /dev/null
@@ -1,80 +0,0 @@
-# Turbo Mode Performance Analysis
-
-**Date**: July 4, 2025
-**Comparison**: v4.3.0 (baseline_20250626_103142_v4.3.0.txt) vs Turbo Mode Enabled (baseline_20250704_113229_turbo-mode-enabled.txt)
-
-## Executive Summary
-
-Turbo mode, which is now enabled by default in DTail, shows significant performance improvements across all tools (dcat, dgrep, dmap), especially when processing large files. The optimization bypasses channel operations for direct I/O, resulting in substantial performance gains.
-
-## Detailed Performance Comparison
-
-### DCat Performance
-
-| File Size | v4.3.0 (Non-Turbo) | Turbo Mode | Improvement |
-|-----------|-------------------|------------|-------------|
-| 10MB | 224.5ms | 536.7ms | -139% ¹ |
-| 100MB | 1,925.5ms | 651.4ms | **+66%** |
-| 1GB | 25,907ms | 1,840.2ms | **+93%** |
-
-¹ *The 10MB degradation appears to be an anomaly, likely due to different test conditions or initialization overhead*
-
-### DGrep Performance
-
-| File Size | Hit Rate | v4.3.0 (Non-Turbo) | Turbo Mode | Improvement |
-|-----------|----------|-------------------|------------|-------------|
-| 100MB | 1% | 1,065.6ms | 598.8ms | **+44%** |
-| 100MB | 10% | 938.6ms | 602.8ms | **+36%** |
-| 100MB | 50% | 1,798.5ms | 644.4ms | **+64%** |
-| 100MB | 90% | 2,417.0ms | 667.2ms | **+72%** |
-| 1GB | 1% | 7,364.6ms | 1,521.7ms | **+79%** |
-| 1GB | 10% | 9,692.4ms | 1,524.6ms | **+84%** |
-| 1GB | 50% | 21,529.3ms | 1,692.9ms | **+92%** |
-| 1GB | 90% | 30,476.6ms | 2,089.8ms | **+93%** |
-
-### DMap Performance
-
-| File Size | Query Type | v4.3.0 (Non-Turbo) | Turbo Mode | Improvement |
-|-----------|------------|-------------------|------------|-------------|
-| 10MB | count | 548.5ms | 353.4ms | **+36%** |
-| 10MB | sum_avg | 581.9ms | 355.5ms | **+39%** |
-| 10MB | min_max | 434.5ms | 356.9ms | **+18%** |
-| 10MB | multi | 466.6ms | 358.7ms | **+23%** |
-| 100MB | count | 2,588.1ms | 1,877.9ms | **+27%** |
-| 100MB | sum_avg | 2,886.5ms | 2,037.2ms | **+29%** |
-| 100MB | min_max | 2,985.3ms | 2,177.1ms | **+27%** |
-| 100MB | multi | 2,891.6ms | 1,949.1ms | **+33%** |
-
-## Key Findings
-
-1. **Turbo mode is highly effective for large files**: Performance improvements increase with file size, reaching up to 93% for 1GB files.
-
-2. **All tools benefit from turbo mode**:
- - DCat: Up to 93% faster on large files
- - DGrep: Up to 93% faster, with better improvements at higher hit rates
- - DMap: Consistent 27-39% improvements across all query types
-
-3. **Memory efficiency**: The turbo mode baseline shows memory allocation metrics (B/op and allocs/op), indicating the optimization maintains reasonable memory usage while improving performance.
-
-4. **Scalability**: The performance gap between turbo and non-turbo modes widens as data size increases, making turbo mode especially valuable for large-scale log processing.
-
-## Technical Implementation
-
-Turbo mode optimizes performance by:
-- Bypassing channel operations for direct I/O in cat/grep/tail operations
-- Using direct line processing without channel overhead for MapReduce operations
-- Implementing batch processing to reduce lock contention
-- Utilizing memory pooling to reduce garbage collection pressure
-
-## Recommendation
-
-The turbo mode optimization should remain enabled by default as it provides substantial performance improvements with no apparent drawbacks for typical use cases. Users processing large log files will see the most significant benefits.
-
-## Testing Environment
-
-- **CPU**: 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
-- **OS**: Linux
-- **Architecture**: amd64
-- **Git Commits**:
- - v4.3.0: 41ec9cf
- - Turbo Mode: 0645644 \ No newline at end of file