t1k:contribution-score
| Field | Value |
|---|---|
| Module | t1k-kit-feedback |
| Version | 2.14.39 |
| Effort | low |
| Tools | — |
Keywords: record contribution, score artifact
How to invoke
Section titled “How to invoke”/t1k:contribution-score--type=issue|sync-back-pr|triage-backfill --ref-url=URL --title=... --body=...t1k:contribution-score — Score a Contribution Artifact (SSOT)
Section titled “t1k:contribution-score — Score a Contribution Artifact (SSOT)”Single source of truth for AI scoring of contributions (issues, PRs).
Invoked by t1k:issue, t1k:sync-back, and t1k:triage — never user-facing
directly. Owns: rubric, endpoint resolution, POST contract, error handling.
When to use
Section titled “When to use”Call this skill from another skill after that skill has successfully created or identified a GitHub artifact (issue or PR) that should be scored as a contribution. Examples:
t1k:issue— aftergh issue createsucceeds, score the just-filed issuet1k:triage— when iterating open issues/PRs, backfill score for unscored items
Not at create time for PRs. type=sync-back-pr requires a MERGED PR (see
Gotchas), so t1k:sync-back does NOT call this skill after gh pr create — the
capture hook records the ref and t1k:contribution-flush scores it once the PR
merges. Only call with type=sync-back-pr for a PR you have confirmed merged.
This skill is fire-and-forget for callers: any failure is logged to stderr and never blocks the caller’s primary workflow (issue creation, PR opening).
Rubric — 1-to-5 generic quality scale
Section titled “Rubric — 1-to-5 generic quality scale”Apply this same rubric to issues AND PRs AND triage backfill — single rubric, no drift. Decide by checklist, not by impression: answer in order and stop at the first tier whose conditions hold.
Q0 — Is there content to score? If the title AND body are both empty or unavailable, do not score it — leave the entry out of the batch. A missing body is a capture gap, not a quality signal, and scoring it as a 2 records the gap as the author’s fault.
Two tests decide the top tiers. Judge each only from what the text actually shows — confident wording is not evidence.
- (E) Evidence — the root cause is identified AND demonstrated: a reproduction, a measurement with numbers, or a test shown failing before the change and passing after.
- (R) Reach — the insight extends beyond the single thing fixed: EITHER the change spans two or more independent components (skills, agents, hooks, modules, repos), OR it states a failure CLASS generally enough to prevent the same mistake elsewhere.
| Score | Condition |
|---|---|
| 5 | (E) AND (R) both hold. |
| 4 | Exactly one of (E) or (R) holds; OR a targeted fix/report with a specific problem, expected-vs-observed, and a clear rationale. |
| 3 | Real and correct but routine: a single fix with no demonstrated root cause, a doc or wording update, a config/data addition, plan or state bookkeeping, a version bump. |
| 2 | Adds little: vague, missing context a reader needs to act, cosmetic-only, or no behavior change and no informational value. |
| 1 | Noise: duplicate, off-topic, throwaway/test content, or reverts useful work. |
- Length and polish are not evidence. A long description of a one-component fix with no demonstrated root cause is not a 5.
- When between two tiers, pick the LOWER — it is harder to inflate scores than to deflate them, and the rubric rewards effort, not enthusiasm.
Why a checklist (measured 2026-09-19): the earlier impression-based table (“Novel & well-evidenced”, “Clear & actionable”) let two scorers given the same 68 artifacts agree on only 53% of scores (weighted kappa 0.34), almost all of it 4-vs-5; which scorer ran moved the share of 5s between 19% and 49%. With (E)/(R) the same design reached 78% (kappa 0.73), and held across a cross-model pair (76%) as well as a same-model pair (79%).
Invocation contract
Section titled “Invocation contract”Caller provides via skill arguments OR via in-context variables:
| Field | Required | Notes |
|---|---|---|
type | yes | One of issue, sync-back-pr, triage-backfill |
ref_url | yes | Canonical GitHub URL: https://github.com/<owner>/<repo>/(issues|pull)/<n> |
kit | yes | Kit short-name, e.g., theonekit-core, theonekit-unity |
repo | yes | <owner>/<repo> form, e.g., The1Studio/theonekit-core |
title | yes | Artifact title (issue title or PR title) |
body | yes | Artifact body — issue body, PR description, or PR diff summary |
evidence_excerpt | optional | Short excerpt (logs, repro snippet) supporting the score |
Workflow
Section titled “Workflow”Step 0 — T1K repo gate
Section titled “Step 0 — T1K repo gate”Skip non-T1K repos so callers (t1k:git pr, t1k:ship, t1k:cook, t1k:babysit-pr) are safe from customer apps. Uniform; no per-skill gate.
case "$REPO" in The1Studio/theonekit-*|The1Studio/t1k-*|The1Studio/unity-mcp|The1Studio/cocos-mcp-server) ;; *) echo "[contribution-score] skipped: not a T1K repo ($REPO)" >&2; exit 0 ;;esacStep 1 — AI scores the artifact
Section titled “Step 1 — AI scores the artifact”Read the rubric above. Score the provided title + body against it. Output:
score: <1..5>rationale: <one sentence explaining the score, ≤300 chars>Be conservative — when in doubt, pick the lower tier.
Step 2 — Resolve telemetry endpoint
Section titled “Step 2 — Resolve telemetry endpoint”Endpoint resolution order (first match wins):
<project>/.claude/t1k-config-core.json→telemetry.cloud.endpoint, then strip trailing/ingestif present~/.claude/t1k-config-core.json(global) — same field- Built-in studio worker default (
https://t1k-telemetry.the1studio.org/ingest) — always resolves; there is no unconfigured state left to skip.
# Reference resolver — Linux/macOS only. On Windows, run inside WSL or git-bash;# Claude Code's hook runner uses Node.js for cross-platform code (per CLAUDE.md# requirement #3), but skill bodies are instruction text and assume a POSIX# shell. If running on a non-POSIX host, port the same logic to Node.js.EP=""if [ -f .claude/t1k-config-core.json ]; then EP=$(jq -r '.telemetry.cloud.endpoint // empty' .claude/t1k-config-core.json | sed 's,/ingest$,,')fiif [ -z "$EP" ] && [ -f "$HOME/.claude/t1k-config-core.json" ]; then EP=$(jq -r '.telemetry.cloud.endpoint // empty' "$HOME/.claude/t1k-config-core.json" | sed 's,/ingest$,,')fi[ -z "$EP" ] && EP="https://t1k-telemetry.the1studio.org"Step 3 — POST to worker
Section titled “Step 3 — POST to worker”TOKEN=$(gh auth token 2>/dev/null) || { echo "[contribution-score] no gh token — skipping" >&2; exit 0}
USER=$(gh api user --jq .login 2>/dev/null) || { echo "[contribution-score] cannot resolve gh user — skipping" >&2; exit 0}
curl -sS --max-time 8 -X POST \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d "$(jq -n --arg u "$USER" --arg k "$KIT" --arg r "$REPO" --arg t "$TYPE" \ --arg url "$REF_URL" --argjson s "$SCORE" --arg ra "$RATIONALE" \ --arg ev "$EVIDENCE" \ '{user:$u, kit:$k, repo:$r, type:$t, ref_url:$url, ai_score:$s, ai_rationale:$ra} + (if $ev == "" then {} else {evidence_excerpt:$ev} end)')" \ "$EP/api/contributions" || trueStep 4 — Report outcome to caller (one line)
Section titled “Step 4 — Report outcome to caller (one line)”[contribution-score] score=<n> recorded ref=<short-url> # success[contribution-score] score=<n> already_recorded ref=<url> # idempotent[contribution-score] skipped: <reason> # missing config[contribution-score] failed: <reason> # any other errorAlways exit 0 — the caller is fire-and-forget. Errors are observability, not blockers.
POST body schema (for reference — worker enforces)
Section titled “POST body schema (for reference — worker enforces)”{ "user": "<gh-login>", // MUST equal authenticated user (worker rejects mismatch) "kit": "theonekit-core", "repo": "The1Studio/theonekit-core", "type": "issue", // issue | sync-back-pr | triage-backfill "ref_url": "https://github.com/<owner>/<repo>/(issues|pull)/<n>", "ai_score": 4, // integer 1..5 "ai_rationale": "<≤500 chars>", "evidence_excerpt": "<optional, ≤500 chars>"}Worker responses:
| HTTP | Body | Meaning |
|---|---|---|
| 201 | {status:"recorded", id, ref_url} | New contribution row |
| 200 | {status:"already_recorded", ref_url} | Idempotency — same (user, ref_url, type) already scored |
| 400 | {error:"..."} | Bad body (validation failure, score range, URL shape, user mismatch) |
| 401 | {error:"Missing Authorization header"} | No token |
| 403 | {error:"author_attribution_failed", reason:"pr_not_merged"} | sync-back-pr whose PR is not merged yet — not yet, retryable |
| 403 | {error:"author_attribution_failed", reason:"triage_requires_closed_artifact"} | triage-backfill on an open artifact — not yet, retryable |
| 403 | {error:"..."} (any other reason) | Token invalid, not in The1Studio org, or author mismatch — terminal |
Hard rules
Section titled “Hard rules”- No retries — if the POST fails (network, 5xx), DO NOT retry. The next call to the caller skill will re-score and the worker will deduplicate.
- Endpoint MUST NOT include
/ingest— the resolver strips it. The contributors API is at/api/contributions, not/ingest/api/contributions.
Security
Section titled “Security”- The skill never logs the GitHub token, the full POST body, or
ai_rationalecontents. Only short status lines go to stderr. - The worker re-runs secret-redaction on
ai_rationaleandevidence_excerptdefensively (defense-in-depth), but this skill SHOULD also avoid passing raw command output that may contain credentials.