| Age | Commit message (Collapse) | Author |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This commit adds two major features and fixes:
1. SSH Agent Key Selection:
- Add --agentKeyIndex flag to select specific SSH agent key (0-based)
- Solves "too many authentication failures" with multiple SSH keys
- Default -1 uses all keys (backwards compatible)
- Available in dtail, dcat, dgrep, dmap commands
2. MapReduce Outfile Fixes:
- CSV files now written at every interval, not just on exit
- Proper signal handling (SIGTERM/SIGINT) with graceful shutdown
- 5-second grace period for cleanup before force exit
- Fixes issue where outfile remained as .tmp during execution
Usage:
dtail --servers host --agentKeyIndex 0 --query '...' outfile results.csv
This is particularly useful with YubiKey/hardware tokens where many
keys are loaded in the SSH agent, and for monitoring MapReduce results
in real-time as they're computed.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
|
- Add profiling support to dtail command (was missing)
- Import profiling package
- Add profile flags and profiler initialization
- Add metrics logging for startup/shutdown
- Fix PGO profile generation for dtail
- Create growing log file simulation for realistic profiling
- Add regex filtering to generate more CPU work
- Handle empty profiles gracefully
- Improve PGO test data generation
- Add growing_log file type for dtail testing
- Generate varied log levels (INFO/WARN/ERROR/DEBUG)
- Increase log generation rate for better profiling
Note: dtail and dserver may generate minimal CPU profiles as they are
primarily I/O-bound operations. PGO is most effective for CPU-intensive
operations like dgrep pattern matching and dmap data processing.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
- Add comprehensive PGO module in internal/tools/pgo/
- Integrate PGO into dtail-tools command with full CLI support
- Add Makefile targets for PGO workflow:
- make pgo: Full PGO workflow
- make pgo-quick: Quick PGO with smaller datasets
- make pgo-generate: Generate profiles only
- make build-pgo: Build with existing profiles
- make install-pgo: Install optimized binaries
- Add convenience functions to data generator for PGO
- Document PGO workflow in CLAUDE.md
Performance improvements observed:
- DCat: 3.8-7.0% additional improvement over turbo mode
- DGrep: Up to 19% improvement for low hit rates
- DMap: Variable impact, up to 64% for min_max on large files
Benchmarks show total performance gains (pre-turbo → turbo+PGO):
- DCat: 14-21x faster
- DGrep: 9-15x faster
- DMap: 9-29% faster
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
This major refactoring replaces all bash-based profiling and benchmarking
scripts with a unified Go tool (dtail-tools) that provides:
- Better cross-platform compatibility
- Improved error handling and reliability
- Structured data generation for test files
- Consistent command-line interface
- Easier maintenance and extensibility
Key changes:
- Created dtail-tools command with profile and benchmark subcommands
- Implemented common utilities for data generation and file operations
- Updated Makefile to use the new Go-based tools
- Maintained backward compatibility with existing make targets
- Fixed ParseSize to handle single-letter suffixes (10M, 1G, etc.)
The new tool supports all previous functionality:
- profile-quick, profile-all, profile-dmap
- benchmark creation, comparison, and management
- Test data generation with multiple formats
- Profile analysis and listing
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
Created a comprehensive profiling framework for dtail commands (dcat, dgrep, dmap)
to analyze CPU usage and memory allocations. The framework now prints all executed
commands to stdout for full transparency.
Key features:
- Integrated Go profiling (CPU, memory, allocations) into all three commands
- Created profile.sh bash script for analyzing pprof profiles
- Added multiple Makefile targets for different profiling scenarios
- Automated profiling scripts with command echoing
- Support for different data sizes (quick, normal, full)
- Special handling for dmap MapReduce format
All profiling commands are now echoed to stdout before execution, making it
easy to understand what the framework is doing and reproduce commands manually.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
fix a data race when reading multiple files on one server from the same session at once
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
file unless NONE is specified
|
|
logger package rewrite as dlog
|
|
|
|
|
|
all color combinations.
|
|
|
|
|
|
|
|
|