t1k:test
| Field | Value |
|---|---|
| Module | t1k-base |
| Version | 5.15.36 |
| Effort | medium |
| Tools | Agent, Bash, Glob, Grep, Read, Task |
Keywords: compile, coverage, failing, flaky, run-tests, test, unit
How to invoke
Section titled “How to invoke”/t1k:test[context] OR compile OR coverage OR --flaky OR --diffTheOneKit Test — Test Runner
Section titled “TheOneKit Test — Test Runner”Delegate to the registered t1k-tester agent. Never ignore failing tests.
| Flag | Mode | Behavior |
|---|---|---|
| (default) | Full | Run entire test suite |
--flaky | Flaky detection | Re-run failing tests up to 3 times, report retry rate per test |
--diff | Diff-aware | Only run tests for files changed since main branch |
--coverage | Coverage | Run with coverage reporting, flag uncovered critical paths |
--quality | Suite quality | Audit, write-standard check, speed, prune: use /t1k:test-quality (no logic here) |
Flaky Test Detection (--flaky)
Section titled “Flaky Test Detection (--flaky)”- Run test suite normally
- For any failing test, re-run it up to 3 times in isolation
- If test passes on retry: mark as FLAKY (intermittent)
- If test fails all retries: mark as FAILING (genuine failure)
- Report: flaky tests with retry rate, genuine failures separately
Diff-Aware Testing (--diff)
Section titled “Diff-Aware Testing (--diff)”- Run
git diff --name-only origin/main...HEADto find changed files - Map changed source files → corresponding test files
- Run only mapped test files
- If no mapping found: run full suite (fallback)
Verification tiers (heavy runs)
Section titled “Verification tiers (heavy runs)”Filtered tests while developing, ONE full run per PR on the merged head after review, no heavy run for a docs/plans/.claude-only diff, generated files regenerated after the merge. Decide with bun .claude/scripts/t1k-verify-tier.cjs classify --base origin/main; policy and the lane procedure: references/verification-tiers.md.
CI parity and silent greens
Section titled “CI parity and silent greens”Run the shard as CI does (Node major, env, selector), count skips, keep test titles stable, classify each red as failure, flake or infra: references/ci-parity-and-silent-greens.md.
Core Principle
Section titled “Core Principle”NEVER IGNORE FAILING TESTS. Fix root causes, not symptoms.
Full-Verify Contract — What “Tests Green” Actually Means
Section titled “Full-Verify Contract — What “Tests Green” Actually Means”“Tests passed” is insufficient evidence to claim done. Three gate types must all pass before reporting complete:
| Gate | Question it answers | Typical command |
|---|---|---|
| Type-check | Does it compile? | bun run typecheck / tsc --noEmit / cargo check |
| Lint / Format | Does it meet project style? | bun run lint / biome check / eslint . / ruff check |
| Tests | Does behavior match spec? | bun test / bun test / pytest / cargo test |
These gates are orthogonal. A change can pass tests and fail lint (or vice versa). CI runs all three; claiming “done” after running only one or two is a false positive that surfaces as a red PR later.
Before claiming work complete:
- Run the kit’s type-check script — read the output
- Run the kit’s lint/format script — read the output
- Run the kit’s test command — read the output
- Only then claim “done” / “green” / “verified”
If the kit has no lint script: note this explicitly (“no lint gate in this project”) so it’s visible rather than silently skipped.
Common scripts to discover
Section titled “Common scripts to discover”Check package.json:
jq -r '.scripts | to_entries[] | select(.key | test("^(test|lint|typecheck|check|format|build)$")) | "\(.key): \(.value)"' package.jsonIf a repo has lint + typecheck + test scripts, run all three. Don’t assume one covers the others.
Invoking Watch-Capable Runners (vitest / jest) — foreground + timeout, never background
Section titled “Invoking Watch-Capable Runners (vitest / jest) — foreground + timeout, never background”Vitest and Jest default to watch mode when invoked bare and never exit — always use the one-shot form (vitest run, jest --ci).
Run the one-shot form in the foreground with a Bash timeout, not as a background job:
CI=1 timeout 300 npx vitest run path/to/suite # one-shot, exits on its own; default reporter works on every vitest version- Don’t background
vitest runyourself. Output capture is unreliable under a non-TTY background pipe. A FULL or HEAVY run is a session-owned background run:Bashwithrun_in_background: true,t1k gate run --class <gateClass> -- <heavyRun.argv>(the gate queues it), stdout and stderr redirected to a file, never a pipe. The orchestrator waits for it in-turn or with a session-owned Monitor until-loop on the exit sentinel, and never ends the turn while it is live. No detached process: t1k never starts one that outlives the session (seerules/agent-completion-discipline.md). - Force non-interactive output:
CI=1and/or--no-color. Do NOT pass--reporter=basic: vitest 5 no longer ships it and exits 1 at startup (Failed to load custom Reporter from basic, core#5321);--reporter=dotor the default reporter work on every version. - Genuine-hang fallback: re-run with
--pool=forks --no-file-parallelism; kill the stuck process rather than polling its empty log.
MANDATORY — Process-count watchdog + worker caps for suites that may spawn children
Section titled “MANDATORY — Process-count watchdog + worker caps for suites that may spawn children”A suite whose tests spawn scripts (spawnSync, bun run, a deploy/release script with a test gate) can re-enter itself and fork-bomb the host (2026-10-06: ~7,400 pids, 18 GiB, 383 OOM kills, a whole herdr session’s agents killed). Always cap workers (bun test --max-concurrency=4, vitest --maxWorkers=4, jest --maxWorkers=4, pytest -n 4) and run such a suite under a watchdog: if ps -eo args | grep -c "[n]ode --test" exceeds ~30, abort and SIGKILL the whole tree (run it under setsid, kill -KILL -- -PGID). Never use ulimit -u as the guard — it counts every process of the user. Before running, check no test spawns a script that re-runs its own suite (recursion-guard env var: rules/test-must-not-rerun-its-suite.md). Recipe: docs/test-must-not-rerun-its-suite.md.
Testing with the real user profile is allowed
Section titled “Testing with the real user profile is allowed”A test or agent MAY run against the real user profile (real HOME, browser profile, gh/cf/npm logins) when it needs real authentication; do not block on “sandbox HOME” for that. Why (user, 2026-10-22): this is an all-users machine, so working against the real profile makes development faster, and the auth a test needs already lives in the browser profile and the gh/cf/npm logins, which a sandboxed HOME cannot reproduce.
One carve-out, from an incident: a harness that deliberately WRITES or MUTATES home files (break-test mutations that write ~/.npmrc, git/npm config, tokens) still redirects HOME/USERPROFILE/NPM_CONFIG_USERCONFIG for those writes, and proves the real file is unchanged with a stat before and after (2026-10-20: a mutation overwrote the real ~/.npmrc). Reading the real profile is fine; letting a destructive test write to it is not. Never print a token, cookie or assertion read from the profile.
A sandbox HOME is not isolation when the child reads a keyring
Section titled “A sandbox HOME is not isolation when the child reads a keyring”Redirecting HOME isolates a child only as far as the child respects it. Four things it does not, each measured:
ghreads its login from the OS keyring, not fromGH_CONFIG_DIR. A sandbox with an empty gh config dir still answersgh auth tokenwith the maintainer’s real token (2026-10-23: a registry engine inside a “sandbox” minted real registry tokens this way). Put aghshim FIRST onPATHthat answersgh auth tokenonly fromGH_TOKEN/GITHUB_TOKENin the child env and fails every other call; the shim must never read the keyring. Reference implementation:.claude/skills/t1k-registry/tests/helpers/sandbox-home.cjs.- bun and node disagree about
HOME.os.homedir()honoursHOMEunder bun on Linux;os.userInfo().homedirignores it under node. SetHOMEANDUSERPROFILE, run every child under one runtime, and have a probe child report whereos.homedir()actually points before the helper creates anything. Under bun,os.homedir()resolves once and ignores a laterprocess.env.HOMEchange:hooks/__tests__/bun-os-homedir-shim.cjsis the preload that makes it follow. - The hook wrapper resets
HOMEfromos.userInfo()(hook-gateway.ts), so it is NOT a safe way to run a hook in a sandbox. Run the hook file itself. - XDG and gh read their own variables (
XDG_CONFIG_HOME,XDG_CACHE_HOME,GH_CONFIG_DIR), so set them together withHOME/USERPROFILEor a write escapes through one of them.
Prove the isolation instead of assuming it: a helper that returns a sandbox should have already shown the child’s home is inside the scratch dir, and a test should hash the real credential files before and after so it can show they did not change. Git config is a separate leak: use emptyGitConfigEnv() from hooks/__tests__/git-test-env.cjs (an empty regular file, never /dev/null, core#3945) rather than inheriting the developer’s identity.
MANDATORY — Kill vitest processes on completion (avoid OOM / IDE crash)
Section titled “MANDATORY — Kill vitest processes on completion (avoid OOM / IDE crash)”--pool=forks (and --pool=threads on older versions) leaves orphaned bun workers alive on Windows when the parent exits via timeout, Ctrl-C, or normal completion. Accumulated orphans eat RAM and have crashed the editor / Claude Code host repeatedly.
Every vitest invocation MUST end with a kill sweep — pass, fail, foreground, or background. Sweep = kill the captured PID, kill its child tree, then name-fallback against node.*vitest. Wrap in trap EXIT (Bash) / try-finally (PowerShell) when scripted. Apply after run returns, after timeout 124, and before launching the next vitest.
The sweep walks DOWN, never UP. Kill the captured PID and the processes it spawned. Never walk the parent chain to find “the tree”: a background run’s ancestry runs past your shell into the Bash tool, Claude, herdr and systemd --user, so a ps -o ppid= loop signals the user’s session manager. pkill -P <pid> / %<id> are child-of (safe); a while … ps -o ppid= loop, kill $PPID or a bare killall node is not. Never signal PID 1, -1, 0, a session manager, or any ancestor of this process. 2026-11-03: an agent cleaning up its own background run walked ppid up to systemd --user and killed it — every herdr and Claude session in that exit.target died with the desktop.
Snippets: see references/vitest-process-management.md.
MANDATORY — Cap concurrent background vitest runs by device hardware
Section titled “MANDATORY — Cap concurrent background vitest runs by device hardware”A static cap is wrong — a 4-core/8GB laptop chokes at 2, a 16-core/64GB workstation idles at 2. Compute the cap from the host:
cap = max(1, min(floor(logicalCores / 4), floor(freeMemGB / 4), 6))
cores/4 and freeMemGB/4 because one run already saturates ~25% of cores and 2-4 GB RAM (worker pool ≈ cores/2). Ceiling 6 prevents IDE/host paging. Floor 1 keeps low-end boxes functional.
Compute once per session (hardware doesn’t change). Before launch, count live node.*vitest processes — if ≥ cap, STOP (wait or kill oldest). Prefer foreground + timeout; at most cap - 1 background runs so a foreground slot stays free.
Compute snippets + reference table: see references/vitest-process-management.md.
Self-Teach on CI Gate Failures — MANDATORY
Section titled “Self-Teach on CI Gate Failures — MANDATORY”Every time a CI quality gate (lint, format, typecheck, test, build, security scan) fails on a PR, you MUST update the responsible skill to prevent the same failure in future sessions. This is the development-principles.md “Update Skills After Every Error” rule applied specifically to CI.
Failure → lesson loop:
- CI gate fails on PR
- Identify which local-verify command would have caught it (the CI step’s name + what it runs)
- If that command is NOT in the skill’s verify checklist → add it
- If it IS in the checklist but was skipped → note why, reinforce the checklist
- Commit the skill update in the SAME session as the CI fix, not “later”
If you are about to push a fix-up commit for a CI failure without updating a skill, stop and update the skill first. The fix-up without the skill update guarantees the same person (or future you) will hit the same gate again.
Bug trail (why this matters)
Section titled “Bug trail (why this matters)”PR theonekit-cli#79 (2026-04-21) had bun run typecheck clean + 433 tests green, but CI’s Lint stage failed on 3 OS matrices because biome check wasn’t in the local verify step. Cost: one extra round trip. Had a full-verify contract been followed, the formatter violations would have been caught and fixed locally before push. This skill section exists BECAUSE that happened.
Agent Routing
Section titled “Agent Routing”Follow protocol: skills/t1k-cook/references/routing-protocol.md
This command uses role: t1k-tester
Skill Activation
Section titled “Skill Activation”Follow protocol: skills/t1k-cook/references/activation-protocol.md
Workflow
Section titled “Workflow”- Compilation check (read console or build output)
- Run tests via registered t1k-tester agent
- Analyze test results for failures
- If failures → spawn registered
t1k-debuggerfor root cause - Report structured results
- Plane work-item update (see below)
Plane Work-Item Update
Section titled “Plane Work-Item Update”Comment-only — no state transition. Full contract:
skills/t1k-plane/references/workflow-enforcement.md (../t1k-plane/references/workflow-enforcement.md)
§ Stage 3. Governing rule: modules/t1k-plane/rules/plane-workitem-workflow.md.
Post the pass/fail counts to every work item bound to this session. A red suite does
NOT move the item backwards — a failing test is information, not a state regression.
If the project defines an extra started-group state named In Review / Testing /
QA, move there instead of commenting alone; never create such a state to satisfy
this. Plane enforcement now blocks, not warns, on an unbound repo or absent plane server — see the cited Stage 0 contract. Escapes: T1K_PLANE_MODE=advisory|off, --no-plane, recorded optout.
Module Context for Tester (if installedModules or modules present in metadata.json)
Section titled “Module Context for Tester (if installedModules or modules present in metadata.json)”Follow protocol: skills/t1k-cook/references/subagent-injection-protocol.md
Before spawning t1k-tester agent, inject:
- Which module’s files are being tested (from
.claude/metadata.json) - Module’s test skills if available
- Boundary: “Test files in module {name} should not test cross-module behavior”
Sub-Agent Fork Hygiene
Section titled “Sub-Agent Fork Hygiene”Sub-agent forking: see skills/t1k-architecture/references/fork-hygiene.md.