Senpi QA
QA the omo Senpi adapter (packages/omo-senpi/) and the task engine
(packages/senpi-task/) by driving the REAL senpi binary. Unit tests never
count as live QA here: bun run test:senpi is the package gate, the drivers in
packages/omo-senpi/scripts/qa/ are the harness proof.
Golden rules
- Evidence lives at exactly one path. Every artifact goes under
.omo/evidence/omo-senpi-adapter/<slug>/. Pick it withscripts/resolve-evidence-dir.mjsand nothing else — a hand-typed path is how runs end up somewhere likelocal-ignore/qa-evidence/or a.qa-evidence/at the worktree root, which is outside the ignored root and gets committed by accident (#8703). - Evidence stays local.
.omo/evidence/is gitignored and the tracked-evidence audit test fails the build if any evidence path is tracked. Nevergit add -fan artifact; the PR body carries the summary and the decisive excerpts. - The real agent dir stays untouched. The live drivers build their own
isolated
SENPI_CODING_AGENT_DIRand deliberately IGNORE a caller-provided one, so~/.senpi/agentis never used as the sandbox. Report the driver'srealSenpiUntouched/ changed-path fields and the isolated agent-dir path; treat a whole-directory digest as supporting evidence, not proof by itself. - No binary means SKIP, not silence. When
senpiis absent the live drivers reportSKIPorFAILin their final JSON rather than degrading to the real home. ASKIPis not a pass — say so in the evidence README. - The captured JSON is the evidence. No file on disk means the QA did not happen, which means no commit and no push. The file proves the run on the machine that made it; it is not something the commit carries.
Resolve the evidence directory first
ev="$(node .agents/skills/senpi-qa/scripts/resolve-evidence-dir.mjs \
--repo-root "$(git rev-parse --show-toplevel)" --slug <YYYYMMDD>-<short-slug>)"
mkdir -p "$ev"
The resolver returns an absolute path and creates nothing, so the caller decides
when the directory appears. A slug is ONE relative segment of lowercase letters,
digits, and hyphens (20260820-senpi-qa-contract). Separators, ./..,
traversal, absolute paths, and a non-git root are rejected with a non-zero exit
and a message naming the offending slug.
Router: pick your case
| You changed… | Run | Proves |
|---|---|---|
| Any adapter code, as the fast precondition | node packages/omo-senpi/scripts/qa/drive.mjs --self-test | the driver + isolation harness itself works |
| Adapter wiring reaching a live session | node packages/omo-senpi/scripts/qa/drive.mjs | a real senpi run with the plugin loaded, isolated agent dir, and no attributed real-home changes |
| Task lifecycle (single + batch) | SENPI_BIN="$(command -v senpi)" node packages/omo-senpi/scripts/qa/task-e2e.mjs | live task start/stream/terminal states |
| Team delivery, shutdown, reclaim, restart recovery | SENPI_BIN="$(command -v senpi)" node packages/omo-senpi/scripts/qa/team-e2e.mjs | injection delivery and exactly-once recovery |
| Task RPC driver scripts | node packages/omo-senpi/scripts/qa/task-rpc-e2e.mjs --self-test | the RPC surface contract |
| Skill delivery into a task | SENPI_BIN="$(command -v senpi)" node packages/omo-senpi/scripts/qa/task-load-skills-e2e.mjs | skills reach the child |
| Continuation behavior | node packages/omo-senpi/scripts/qa/probe-continuation.mjs | turns continue as expected |
| DAG state machine / runners | bun test packages/senpi-task | unit + chaos invariants (NOT live proof) |
Point a driver's output at the resolved directory, e.g.:
TASK_E2E_OUT_DIR="$ev/live-task-dag" SENPI_BIN="$(command -v senpi)" \
node packages/omo-senpi/scripts/qa/task-e2e.mjs
Package gate
tsgo --noEmit -p packages/omo-senpi/tsconfig.json
bun run test:senpi
Write the evidence README
Every run leaves $ev/README.md a reviewer can read without rerunning anything.
The required sections are the repo-wide evidence rules in the root
AGENTS.md (what was tested / observed / why it is enough /
what was omitted). For Senpi, record the driver's changed-path/isolation fields
and sandbox agent-dir path. Some drivers report sandbox paths without removing
them; the caller must delete every task-owned sandbox and verify child PIDs are
terminal before writing the cleanup receipt.