green-that-proves-nothing
A Green That Proves Nothing — Ask What Would Have Failed
Section titled “A Green That Proves Nothing — Ask What Would Have Failed”Before trusting any passing check (doctor line, CI gate, test, dashboard tile): if the thing this guards were broken right now, would this check go red? If you cannot name the failure it would catch, it is decoration. Four shapes:
- Checks the wrong artifact
- Aggregates past the problem
- Measures the wrong population
- Rate over an empty denominator
A green that is the ONLY evidence is still open. Writing a check: break it deliberately and watch it go red, on a field a test actually reads (a seed no test reads stays green and proves nothing), pin the FAILURE states, name the population covered and excluded, separate unknown from good, state the sample size beside a rate. Pinned baselines MUST ship a companion that fails when the pin goes stale: assert by IDENTITY not count, in BOTH directions, INVERT it, never re-pin (pinned-baseline-test-companion.md, loads on test files). Adding a gate: name its runner, green on the default branch when it lands, runs on a clean checkout. Full text (§ “Gate hygiene”): docs/green-that-proves-nothing.md.