summaryrefslogtreecommitdiff
path: root/CLAUDE.md
diff options
context:
space:
mode:
authorPaul Buetow <paul@buetow.org>2025-07-04 15:35:16 +0300
committerPaul Buetow <paul@buetow.org>2025-07-04 15:35:16 +0300
commitd37f32deb6cd6a575cc169adf1a1c1fba44e53d9 (patch)
treeaaf5f6abc90066892a6a23cb619969ddd4ef5574 /CLAUDE.md
parent1249f9ec51b1355ca17f73244dcbe0acc5556516 (diff)
feat: add Profile-Guided Optimization (PGO) support
- Add comprehensive PGO module in internal/tools/pgo/ - Integrate PGO into dtail-tools command with full CLI support - Add Makefile targets for PGO workflow: - make pgo: Full PGO workflow - make pgo-quick: Quick PGO with smaller datasets - make pgo-generate: Generate profiles only - make build-pgo: Build with existing profiles - make install-pgo: Install optimized binaries - Add convenience functions to data generator for PGO - Document PGO workflow in CLAUDE.md Performance improvements observed: - DCat: 3.8-7.0% additional improvement over turbo mode - DGrep: Up to 19% improvement for low hit rates - DMap: Variable impact, up to 64% for min_max on large files Benchmarks show total performance gains (pre-turbo → turbo+PGO): - DCat: 14-21x faster - DGrep: 9-15x faster - DMap: 9-29% faster 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Diffstat (limited to 'CLAUDE.md')
-rw-r--r--CLAUDE.md76
1 files changed, 76 insertions, 0 deletions
diff --git a/CLAUDE.md b/CLAUDE.md
index c609020..845ec5b 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -28,6 +28,12 @@ DTAIL_USE_ACL=yes make build
# Enable proprietary features
DTAIL_USE_PROPRIETARY=yes make build
+
+# Build PGO-optimized binaries (requires existing profiles)
+make build-pgo
+
+# Generate PGO profiles and build optimized binaries
+make pgo
```
## Testing & Development
@@ -68,6 +74,43 @@ make benchmark-baseline
make benchmark-compare BASELINE=benchmarks/baselines/baseline_TIMESTAMP.txt
```
+## Profile-Guided Optimization (PGO)
+
+```bash
+# Full PGO workflow: generate profiles and build optimized binaries
+make pgo
+
+# Quick PGO with smaller datasets (faster)
+make pgo-quick
+
+# PGO for specific commands only
+make pgo-commands COMMANDS='dcat dgrep'
+
+# Generate PGO profiles only (without building)
+make pgo-generate
+
+# Build PGO-optimized binaries using existing profiles
+make build-pgo
+
+# Install PGO-optimized binaries to system
+make install-pgo
+
+# Clean PGO artifacts
+make pgo-clean
+
+# Show PGO help
+make pgo-help
+```
+
+### PGO Notes
+
+- PGO provides additional performance improvements on top of turbo mode
+- Typical improvements: 5-10% for DCat, up to 19% for DGrep with low hit rates
+- Profiles are saved in `pgo-profiles/` directory
+- Optimized binaries are built in `pgo-build/` directory
+- Use `make build-pgo` to rebuild optimized binaries without regenerating profiles
+- PGO profiles are workload-specific; consider custom profiles for your use case
+
## Profiling
```bash
@@ -159,6 +202,39 @@ make benchmark-mapreduce
make benchmark-ssh
```
+## Profile-Guided Optimization (PGO)
+
+```bash
+# Run PGO for all commands
+make pgo
+
+# Quick PGO with smaller datasets
+make pgo-quick
+
+# PGO for specific commands
+make pgo-commands COMMANDS='dcat dgrep'
+
+# Clean PGO artifacts
+make pgo-clean
+
+# Show PGO help
+make pgo-help
+
+# Direct usage with dtail-tools
+dtail-tools pgo # Optimize all commands
+dtail-tools pgo dcat dgrep # Optimize specific commands
+dtail-tools pgo -v -iterations 5 # Verbose with 5 iterations
+
+# After PGO, optimized binaries are in pgo-build/
+```
+
+### PGO Notes
+
+- PGO uses profile data from real workloads to optimize binary performance
+- The process involves: building baseline → generating profiles → building with PGO
+- Typical improvements range from 5-20% depending on the workload
+- Optimized binaries are placed in the `pgo-build/` directory
+
## Architecture & Code Organization
### Binary Entry Points