Skip to content

t1k:contribution-score

FieldValue
Modulet1k-kit-feedback
Version2.14.39
Effortlow
Tools—

Keywords: record contribution, score artifact

/t1k:contribution-score
--type=issue|sync-back-pr|triage-backfill --ref-url=URL --title=... --body=...

t1k:contribution-score — Score a Contribution Artifact (SSOT)

Section titled “t1k:contribution-score — Score a Contribution Artifact (SSOT)”

Single source of truth for AI scoring of contributions (issues, PRs). Invoked by t1k:issue, t1k:sync-back, and t1k:triage — never user-facing directly. Owns: rubric, endpoint resolution, POST contract, error handling.

Call this skill from another skill after that skill has successfully created or identified a GitHub artifact (issue or PR) that should be scored as a contribution. Examples:

  • t1k:issue — after gh issue create succeeds, score the just-filed issue
  • t1k:triage — when iterating open issues/PRs, backfill score for unscored items

Not at create time for PRs. type=sync-back-pr requires a MERGED PR (see Gotchas), so t1k:sync-back does NOT call this skill after gh pr create — the capture hook records the ref and t1k:contribution-flush scores it once the PR merges. Only call with type=sync-back-pr for a PR you have confirmed merged.

This skill is fire-and-forget for callers: any failure is logged to stderr and never blocks the caller’s primary workflow (issue creation, PR opening).

Apply this same rubric to issues AND PRs AND triage backfill — single rubric, no drift. Decide by checklist, not by impression: answer in order and stop at the first tier whose conditions hold.

Q0 — Is there content to score? If the title AND body are both empty or unavailable, do not score it — leave the entry out of the batch. A missing body is a capture gap, not a quality signal, and scoring it as a 2 records the gap as the author’s fault.

Two tests decide the top tiers. Judge each only from what the text actually shows — confident wording is not evidence.

  • (E) Evidence — the root cause is identified AND demonstrated: a reproduction, a measurement with numbers, or a test shown failing before the change and passing after.
  • (R) Reach — the insight extends beyond the single thing fixed: EITHER the change spans two or more independent components (skills, agents, hooks, modules, repos), OR it states a failure CLASS generally enough to prevent the same mistake elsewhere.
ScoreCondition
5(E) AND (R) both hold.
4Exactly one of (E) or (R) holds; OR a targeted fix/report with a specific problem, expected-vs-observed, and a clear rationale.
3Real and correct but routine: a single fix with no demonstrated root cause, a doc or wording update, a config/data addition, plan or state bookkeeping, a version bump.
2Adds little: vague, missing context a reader needs to act, cosmetic-only, or no behavior change and no informational value.
1Noise: duplicate, off-topic, throwaway/test content, or reverts useful work.
  • Length and polish are not evidence. A long description of a one-component fix with no demonstrated root cause is not a 5.
  • When between two tiers, pick the LOWER — it is harder to inflate scores than to deflate them, and the rubric rewards effort, not enthusiasm.

Why a checklist (measured 2026-09-19): the earlier impression-based table (“Novel & well-evidenced”, “Clear & actionable”) let two scorers given the same 68 artifacts agree on only 53% of scores (weighted kappa 0.34), almost all of it 4-vs-5; which scorer ran moved the share of 5s between 19% and 49%. With (E)/(R) the same design reached 78% (kappa 0.73), and held across a cross-model pair (76%) as well as a same-model pair (79%).

Caller provides via skill arguments OR via in-context variables:

FieldRequiredNotes
typeyesOne of issue, sync-back-pr, triage-backfill
ref_urlyesCanonical GitHub URL: https://github.com/<owner>/<repo>/(issues|pull)/<n>
kityesKit short-name, e.g., theonekit-core, theonekit-unity
repoyes<owner>/<repo> form, e.g., The1Studio/theonekit-core
titleyesArtifact title (issue title or PR title)
bodyyesArtifact body — issue body, PR description, or PR diff summary
evidence_excerptoptionalShort excerpt (logs, repro snippet) supporting the score

Skip non-T1K repos so callers (t1k:git pr, t1k:ship, t1k:cook, t1k:babysit-pr) are safe from customer apps. Uniform; no per-skill gate.

Terminal window
case "$REPO" in
The1Studio/theonekit-*|The1Studio/t1k-*|The1Studio/unity-mcp|The1Studio/cocos-mcp-server) ;;
*) echo "[contribution-score] skipped: not a T1K repo ($REPO)" >&2; exit 0 ;;
esac

Read the rubric above. Score the provided title + body against it. Output:

score: <1..5>
rationale: <one sentence explaining the score, ≤300 chars>

Be conservative — when in doubt, pick the lower tier.

Endpoint resolution order (first match wins):

  1. <project>/.claude/t1k-config-core.json → telemetry.cloud.endpoint, then strip trailing /ingest if present
  2. ~/.claude/t1k-config-core.json (global) — same field
  3. Built-in studio worker default (https://t1k-telemetry.the1studio.org/ingest) — always resolves; there is no unconfigured state left to skip.
Terminal window
# Reference resolver — Linux/macOS only. On Windows, run inside WSL or git-bash;
# Claude Code's hook runner uses Node.js for cross-platform code (per CLAUDE.md
# requirement #3), but skill bodies are instruction text and assume a POSIX
# shell. If running on a non-POSIX host, port the same logic to Node.js.
EP=""
if [ -f .claude/t1k-config-core.json ]; then
EP=$(jq -r '.telemetry.cloud.endpoint // empty' .claude/t1k-config-core.json | sed 's,/ingest$,,')
fi
if [ -z "$EP" ] && [ -f "$HOME/.claude/t1k-config-core.json" ]; then
EP=$(jq -r '.telemetry.cloud.endpoint // empty' "$HOME/.claude/t1k-config-core.json" | sed 's,/ingest$,,')
fi
[ -z "$EP" ] && EP="https://t1k-telemetry.the1studio.org"
Terminal window
TOKEN=$(gh auth token 2>/dev/null) || {
echo "[contribution-score] no gh token — skipping" >&2; exit 0
}
USER=$(gh api user --jq .login 2>/dev/null) || {
echo "[contribution-score] cannot resolve gh user — skipping" >&2; exit 0
}
curl -sS --max-time 8 -X POST \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "$(jq -n --arg u "$USER" --arg k "$KIT" --arg r "$REPO" --arg t "$TYPE" \
--arg url "$REF_URL" --argjson s "$SCORE" --arg ra "$RATIONALE" \
--arg ev "$EVIDENCE" \
'{user:$u, kit:$k, repo:$r, type:$t, ref_url:$url, ai_score:$s,
ai_rationale:$ra} + (if $ev == "" then {} else {evidence_excerpt:$ev} end)')" \
"$EP/api/contributions" || true

Step 4 — Report outcome to caller (one line)

Section titled “Step 4 — Report outcome to caller (one line)”
[contribution-score] score=<n> recorded ref=<short-url> # success
[contribution-score] score=<n> already_recorded ref=<url> # idempotent
[contribution-score] skipped: <reason> # missing config
[contribution-score] failed: <reason> # any other error

Always exit 0 — the caller is fire-and-forget. Errors are observability, not blockers.

POST body schema (for reference — worker enforces)

Section titled “POST body schema (for reference — worker enforces)”
{
"user": "<gh-login>", // MUST equal authenticated user (worker rejects mismatch)
"kit": "theonekit-core",
"repo": "The1Studio/theonekit-core",
"type": "issue", // issue | sync-back-pr | triage-backfill
"ref_url": "https://github.com/<owner>/<repo>/(issues|pull)/<n>",
"ai_score": 4, // integer 1..5
"ai_rationale": "<≤500 chars>",
"evidence_excerpt": "<optional, ≤500 chars>"
}

Worker responses:

HTTPBodyMeaning
201{status:"recorded", id, ref_url}New contribution row
200{status:"already_recorded", ref_url}Idempotency — same (user, ref_url, type) already scored
400{error:"..."}Bad body (validation failure, score range, URL shape, user mismatch)
401{error:"Missing Authorization header"}No token
403{error:"author_attribution_failed", reason:"pr_not_merged"}sync-back-pr whose PR is not merged yet — not yet, retryable
403{error:"author_attribution_failed", reason:"triage_requires_closed_artifact"}triage-backfill on an open artifact — not yet, retryable
403{error:"..."} (any other reason)Token invalid, not in The1Studio org, or author mismatch — terminal
  • No retries — if the POST fails (network, 5xx), DO NOT retry. The next call to the caller skill will re-score and the worker will deduplicate.
  • Endpoint MUST NOT include /ingest — the resolver strips it. The contributors API is at /api/contributions, not /ingest/api/contributions.
  • The skill never logs the GitHub token, the full POST body, or ai_rationale contents. Only short status lines go to stderr.
  • The worker re-runs secret-redaction on ai_rationale and evidence_excerpt defensively (defense-in-depth), but this skill SHOULD also avoid passing raw command output that may contain credentials.