docs: finalize libbpfgo upgrade guidance

4 files changed, 122 insertions, 25 deletions

diff --git a/AGENTS.md b/AGENTS.md
index 65c9a2c..ddfdbbf 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -12,20 +12,19 @@ git -C ../libbpfgo submodule update --init --recursive
 make -C ../libbpfgo libbpfgo-static
 ```
 
-If builds/tests fail with missing libbpf headers (for example `bpf/bpf.h` not found), run `mage world` first. It bootstraps generation/dependencies and is the preferred first troubleshooting step before retrying `mage test` or `go test`.
+If builds/tests fail with missing libbpf headers (for example `bpf/bpf.h` not found), rerun the commands above and then run `env GOTOOLCHAIN=auto mage world`. Prefer Mage targets over raw `go test` for packages that import `libbpfgo`; Mage wires the required `CGO_CFLAGS`, `CGO_LDFLAGS`, and `LIBBPFGO` values.
 
 ```bash
-mage all          # Build everything (BPF objects and Go binary)
-mage test         # Run all tests
-TEST_NAME=TestEventloop mage testWithName  # Run specific test
-go test ./internal -v                      # Run tests for internal package
-mage integrationTest                       # Build + run integration tests (default parallelism is capped)
-INTEGRATION_PARALLEL=1 mage integrationTest  # Force serial integration tests
-mage generate     # Generate code (required after modifying tracepoint definitions)
-mage bench        # Run benchmarks
-mage prReview     # Run PR review baseline: world + benchProf
-mage clean        # Clean build artifacts
-mage world        # Clean + generate + test + build (recommended reset path)
+env GOTOOLCHAIN=auto mage all          # Build everything (BPF objects and Go binary)
+env GOTOOLCHAIN=auto mage test         # Run all tests
+GOTOOLCHAIN=auto TEST_NAME=TestEventloop mage testWithName  # Run specific test
+env GOTOOLCHAIN=auto mage integrationTest  # Build + run integration tests (default parallelism is capped)
+GOTOOLCHAIN=auto INTEGRATION_PARALLEL=1 mage integrationTest  # Force serial integration tests
+env GOTOOLCHAIN=auto mage generate     # Generate code (required after modifying tracepoint definitions)
+env GOTOOLCHAIN=auto mage bench        # Run benchmarks
+env GOTOOLCHAIN=auto mage prReview     # Run PR review baseline: world + benchProf
+env GOTOOLCHAIN=auto mage clean        # Clean build artifacts
+env GOTOOLCHAIN=auto mage world        # Clean + generate + test + build (recommended reset path)
 ```
 
 ## Code Generation
@@ -75,3 +74,16 @@ Generator source code:
 - BPF C code in `/internal/c/ior.bpf.c` should be minimal for verification
 - Import style: `"ior/internal/packagename"` for internal packages
 - Error handling: Return errors, don't panic except for setup validation
+
+## Rollback
+
+If `v0.9.2-libbpf-1.5.1` stops working, roll the local checkout back to commit
+`90dbffffbdab` (module version
+`v0.6.0-libbpf-1.3.0.20240111220235-90dbffffbdab`), update `go.mod`/`go.sum`
+accordingly, and rebuild:
+
+```bash
+git -C ../libbpfgo checkout 90dbffffbdab
+git -C ../libbpfgo submodule update --init --recursive
+make -C ../libbpfgo libbpfgo-static
+```
diff --git a/README.md b/README.md
index 7ee0373..a63f6a3 100644
--- a/README.md
+++ b/README.md
@@ -31,6 +31,29 @@ git -C ../libbpfgo submodule update --init --recursive
 make -C ../libbpfgo libbpfgo-static
 ```
 
+Validated commands for this pin:
+
+```shell
+env GOTOOLCHAIN=auto mage world
+env GOTOOLCHAIN=auto mage integrationTest
+```
+
+Troubleshooting and rollback:
+
+- If builds fail with `bpf/bpf.h` missing, re-run the checkout, submodule sync,
+  and `make libbpfgo-static` commands above, then retry `env GOTOOLCHAIN=auto mage world`.
+- Prefer Mage targets over raw `go test` for packages that import `libbpfgo`;
+  Mage injects the required `CGO_CFLAGS`, `CGO_LDFLAGS`, and `LIBBPFGO` values.
+- To roll back to the previous wrapper state, repin `go.mod` to
+  `github.com/aquasecurity/libbpfgo v0.6.0-libbpf-1.3.0.20240111220235-90dbffffbdab`,
+  then reset the sibling checkout and rebuild:
+
+```shell
+git -C ../libbpfgo checkout 90dbffffbdab
+git -C ../libbpfgo submodule update --init --recursive
+make -C ../libbpfgo libbpfgo-static
+```
+
 ## Timing Semantics
 
 Each reported event pair has two timing counters:
diff --git a/docs/libbpfgo-upgrade-plan.md b/docs/libbpfgo-upgrade-plan.md
index 1290479..1969015 100644
--- a/docs/libbpfgo-upgrade-plan.md
+++ b/docs/libbpfgo-upgrade-plan.md
@@ -9,15 +9,18 @@ runtime validation on that same tag.
 
 ## Current State
 
-- `go.mod` / `go.sum` are expected to pin
+- `go.mod` / `go.sum` now pin
   `github.com/aquasecurity/libbpfgo v0.9.2-libbpf-1.5.1`
 - `Magefile.go` defaults to the sibling checkout at `../libbpfgo` (local path:
-  `/home/paul/git/libbpfgo`) and should emit rebuild guidance if static
+  `/home/paul/git/libbpfgo`) and emits rebuild guidance if static
   artifacts are missing
 - The local checkout is currently ahead of the latest tag:
   `v0.9.2-libbpf-1.5.1-23-g9a319d2`
-- `README.md` / `AGENTS.md` should pin the tag, sync the `libbpf` submodule,
-  and document the static rebuild workflow explicitly
+- `README.md`, `AGENTS.md`, and `integrationtests/README.md` now pin the tag,
+  sync the `libbpf` submodule, and document the rebuild or validation workflow
+- Integration coverage now passes again after restoring the legacy
+  `-flamegraph` / `-name` compatibility path used by the harness to collect
+  `.ior.zst` artifacts
 
 ## Upgrade Target
 
@@ -31,12 +34,15 @@ runtime validation on that same tag.
 
 ## Pinned Source of Truth
 
-- `go.mod` / `go.sum` should pin `github.com/aquasecurity/libbpfgo
+- `go.mod` / `go.sum` pin `github.com/aquasecurity/libbpfgo
   v0.9.2-libbpf-1.5.1`
-- `README.md` and `AGENTS.md` should document the same checkout, tag, submodule,
-  and `make libbpfgo-static` workflow
-- `Magefile.go` should fail with explicit rebuild guidance when the local
+- `README.md`, `AGENTS.md`, and `integrationtests/README.md` document the same
+  checkout, tag, validation commands, and `make libbpfgo-static` workflow
+- `Magefile.go` fails with explicit rebuild guidance when the local
   `libbpfgo` checkout is missing the static artifacts that `ior` expects
+- `internal/ior.go` preserves the legacy `-flamegraph` / `-name` trace-output
+  path required by the integration harness while leaving TUI and `-plain`
+  behavior unchanged
 
 ## Breaking-Change Watchpoints
 
@@ -80,8 +86,8 @@ validation is still required.
      - any `probemanager` adapter code if signatures changed
 
 4. Validate behavior
-   - Run `mage world`
-   - Run root-required integration coverage
+   - Run `env GOTOOLCHAIN=auto mage world`
+   - Run root-required `env GOTOOLCHAIN=auto mage integrationTest`
    - Specifically verify:
      - embedded `ior.bpf.o` loading still works
      - tracepoint attach/detach still works
@@ -92,6 +98,41 @@ validation is still required.
    - Document the exact `libbpfgo` tag and rebuild commands
    - Mention the local checkout path used by `Magefile.go`
    - Add troubleshooting notes for submodule sync / static rebuild failures
+   - Record the rollback target: `go.mod` pseudo-version
+     `v0.6.0-libbpf-1.3.0.20240111220235-90dbffffbdab` plus local checkout
+     commit `90dbffffbdab`
+
+## Validation Result
+
+- `env GOTOOLCHAIN=auto mage world` passed after the pinning commit
+  `f28dab3`
+- `env GOTOOLCHAIN=auto mage integrationTest` passed after compatibility fix
+  commit `28338f4`
+- The embedded-object path is covered by
+  `env GOTOOLCHAIN=auto TEST_NAME=TestLoadBPFModuleUsesEmbeddedObjectByDefault mage testWithName`
+
+## Troubleshooting
+
+- Missing `bpf/bpf.h` or `libbpf` symbols usually means the sibling checkout is
+  not at `v0.9.2-libbpf-1.5.1` or was not rebuilt after a `git checkout`.
+- Raw `go test` can still fail for packages that import `libbpfgo` because it
+  does not inherit the `CGO_CFLAGS`, `CGO_LDFLAGS`, and `LIBBPFGO` values that
+  `Magefile.go` sets up. Use Mage targets for validated flows.
+- If integration tests fail immediately with unknown `-flamegraph` /
+  `-name` flags, rebuild `ior` from a checkout that includes commit `28338f4`.
+
+## Rollback
+
+If the tagged release proves insufficient, revert the `ior` side to
+`github.com/aquasecurity/libbpfgo
+v0.6.0-libbpf-1.3.0.20240111220235-90dbffffbdab`, reset the sibling checkout,
+and rebuild:
+
+```bash
+git -C /home/paul/git/libbpfgo checkout 90dbffffbdab
+git -C /home/paul/git/libbpfgo submodule update --init --recursive
+make -C /home/paul/git/libbpfgo libbpfgo-static
+```
 
 ## Validation Commands
 
diff --git a/integrationtests/README.md b/integrationtests/README.md
index be65499..0036019 100644
--- a/integrationtests/README.md
+++ b/integrationtests/README.md
@@ -8,33 +8,54 @@ harness asserts the captured `.ior.zst` output matches expectations.
 
 - Built `ior` binary (`mage all`)
 - Root privileges or `CAP_BPF` (required for BPF tracepoint attachment)
+- Local `libbpfgo` checkout pinned to `v0.9.2-libbpf-1.5.1` and rebuilt with
+  `git -C ../libbpfgo submodule update --init --recursive && make -C ../libbpfgo libbpfgo-static`
 
 The binary embeds its default BPF object. Set `IOR_BPF_OBJECT=/path/to/ior.bpf.o`
 only when you explicitly want to override the embedded object during testing.
+The harness runs `ior` in the legacy compatibility mode that writes aggregated
+`.ior.zst` output via `-flamegraph -name <scenario>`; this is kept specifically
+for integration coverage and rollback validation.
 
 ## Running
 
 ```bash
-mage integrationTest
+env GOTOOLCHAIN=auto mage integrationTest
 ```
 
 This builds everything (ior, ioworkload) and runs integration tests in parallel.
 Default parallelism is `NumCPU * 2` (minimum `1`).
 
+Validated upgrade checks:
+
+```bash
+env GOTOOLCHAIN=auto mage world
+env GOTOOLCHAIN=auto mage integrationTest
+```
+
 Tests automatically skip with `t.Skip` when not running as root.
 
 To run serially (useful for debugging/flaky triage):
 
 ```bash
-mage integrationTestSerial
+env GOTOOLCHAIN=auto INTEGRATION_PARALLEL=1 mage integrationTest
 ```
 
 Tune parallelism by setting `INTEGRATION_PARALLEL`, for example:
 
 ```bash
-INTEGRATION_PARALLEL=4 mage integrationTest
+env GOTOOLCHAIN=auto INTEGRATION_PARALLEL=4 mage integrationTest
 ```
 
+If the suite fails before tracing starts, check for these common causes:
+
+- The local `libbpfgo` checkout is not pinned to `v0.9.2-libbpf-1.5.1` or was
+  rebuilt without `git submodule update --init --recursive`.
+- `ior` was not rebuilt after pulling the compatibility fix for `-flamegraph`
+  / `-name`.
+- The run is missing root privileges, in which case the suite will skip or fail
+  before BPF attach.
+
 ## Structure
 
 - `cmd/ioworkload/` — Standalone binary performing known I/O patterns