Skip to content

t1k:test-quality

FieldValue
Modulet1k-base
Version5.15.36
Efforthigh
ToolsBash, Glob, Grep, Read

Keywords: audit tests, kill-check, mutation check, mutation testing, prune test, prune tests, redundant tests, remove redundant tests, slow tests, test audit, test quality, test smells, test suite audit, test-quality, useless test, useless tests, weak assertions, write good tests

/t1k:test-quality
audit [path] | prune <audit-report> | speed [path] | standard [--diff]

Test Quality: audit, write standard, speed, evidence-gated prune

Section titled “Test Quality: audit, write standard, speed, evidence-gated prune”

t1k-test RUNS suites and t1k-review-tests mutation-tests ONE PR. This skill judges a WHOLE suite. The script emits facts (JSON, schema t1k-test-quality/1, exit codes); you decide policy and explain it. It never deletes: it proposes, and an ordinary reviewed PR lands the change.

Engine: bun .claude/skills/t1k-test-quality/scripts/t1k-test-quality.cjs <subcommand> ... (call it $TQ below). Discover the live surface with $TQ --help --json before quoting any flag. Schema, exit codes, reasons and module interfaces: references/evidence-contract.md (the only place they are defined).

You runWhat it doesProcedure
audit [path]tiers T0 to T2 over the scope, writes the reportreferences/audit-procedure.md
prune <audit-report>branch with only confirmed-delete ids, ledger, verifiedreferences/prune-pr.md
speed [path]per-file and per-test durations, slow listreferences/speed.md
standard [--diff]does new or changed test code meet the write standardreferences/write-standard.md

A plain request (“find the useless tests”, “remove useless tests”, no subcommand) is audit over the project root: pick that scope yourself and state it; never stop to ask for one (inside this repo, preflight 6 wins: audit a named subtree, never its own suite). A forked run cannot ask the user, so run the smoke sample of preflight step 5, return its report with the next command, and never widen the scope on your own. Removing tests is prune <audit-report>. Deleting or editing a test file by hand is exactly what propose-only forbids, and a flaky or exempt test is never removed, however obviously useless it reads.

standard is $TQ lint-new --base <ref> [--head <ref>] [--root <dir>] (deterministic subset, exit 1 on a high-confidence smell or a weakening) plus your judgment over the checklist items lint-new cannot see.

Preflight (every run, before the first expensive call)

Section titled “Preflight (every run, before the first expensive call)”
  1. Branch pin: git branch --show-current equals the lane’s branch; work only in the worktree the caller created. Never write the shared tree of a pinned lane session.
  2. free -g and swap. The engines refuse low memory or high swap themselves (exit 4 unsafe-environment); that is a stop, not a retry loop.
  3. Runner detection: $TQ detect <root> gives the harness split, file count, test count N, spawn-children count.
  4. Baseline green: the engines run it and answer exit 4 red-baseline otherwise. Fix the suite first.
  5. Preview first (rules/preview-first-batch.md): state an estimate (files x runs, kill-check is the expensive step) and run a smoke sample on at most 5 files; surface it; ask before the full scope.
  6. Worker caps always: bun test --max-concurrency=4; never two full suites at once; never run audit on this repo’s own suite from inside that suite’s tests. Caps, watchdog and recursion guard: t1k-test SKILL.md.
  • Propose-only. Output is an audit report and, on request, a prune branch headed for a reviewed PR. Never delete outside such a branch.
  • You do not mint evidence. confirmed-delete exists only from an evidence record the script wrote that the base-pinned verify-prune accepted. “Looks useless” is rewrite or unconfirmed.
  • Evidence ladder T0 to T3. T0 static smells flag, never delete. T1 flaky repeats and shuffles. T2 coverage + kill-check give never-kills / redundant. T3 is the deletion protocol (verify-prune). Deletable evidence kinds: target-absent, duplicate-of, kill-check only.
  • Harness model first. A runner-globals or custom-check-exit file’s unit is the FILE and its exit code. unparsed is reported, never zero-assert. Shrinking an unparsed file fails unparsed-shrink.
  • Fix over delete. A test with a purpose and weak assertions gets a strengthened-assertion proposal (rewrite).
  • Flaky is quarantine, not delete. Mixed T1 results go to the quarantine list (“fix or quarantine”); never a prune candidate. A record with t1.mixed: true fails verify-prune (t1-mixed).
  • Exempt means exempt. *_KnownGap, *_HasNoStaleEntries, pin shapes, fix/issue/PR-linked history, added in the last 30 days, pin markers: never confirmed-delete. The verifier recomputes this from BASE; you cannot override it.
  • Deletion is a protocol. One commit per cluster; ledger test-prune-ledger/<evidenceId>.json in the PR; per-PR cap tripwire (exit 1 cap-exceeded: split the PR; it does not bound aggregate deletion); parity run after.
  • Independent replay, fail closed. The reviewer lane replays a sample with a fresh seed (verify-prune --rerun --sample <s> --seed <random>). Exit 3 or 4 is “not verified”, never a pass.
  • No weakening to pass. verify-prune and lint-new check the diff (assertion count down, skip/only/xfail added, tolerance widened). Do not edit retained tests, data/runners.json or the verifier to make a prune pass (verifier-modified).
  • Refusals are results. Exit 3 (unsupported-runner / unsupported-tier: jest, vitest, pytest have no coverage or kill-check in v1) and exit 4 mean “not verified”. Say so; do not retry around them.
  • Unity v1 is static checks plus timing only: no coverage, kill-check or deletion authority.

Report counts by verdict, then per-candidate id, category, evidence kind, verdict, one-line reason. Quote script output, not memory. Every claim of “unused” or “redundant” cites an evidence id; otherwise write unconfirmed. Put the report at plans/reports/test-quality-<id>/audit.json and audit.md.

A wrong verdict (a false-confirm, a false flag, a pruned test re-added within 30 days, verify-prune --rerun exiting 1 on a record that was confirmed-delete) is a skill bug: emit [t1k:skill-bug kit="core" skill="t1k-test-quality" bug="<false-confirm|false-flag>" evidence="<evidence id>"] (existing marker; /t1k:issue files it, see docs/auto-issue-pipeline.md). Judgment errors land in references/hard-rules.md; detector or adapter errors in data/runners.json and scripts/lib/detectors.cjs, with a fixture that reproduces the miss.

Judgment (verdicts, what to prune) stays on the reasoning tier. The mechanical steps run scripts/t1k-test-quality.cjs and report its output.

StepAgent
Run detect, coverage, flaky, speed, mutate, kill-check and collect the evidence recordst1k-tester
Decide verdicts, write the prune PR, run verify-prunethe invoking session
  • references/evidence-contract.md: schema, tiers, verdicts, exit codes, trust model, module interfaces
  • references/audit-procedure.md, prune-pr.md, speed.md, write-standard.md, hard-rules.md
  • Human doc and cross-kit follow-ups: docs/test-quality-skill.md