Skip to content

t1k:unity:mobile:ab-test

FieldValue
Modulemobile
Version3.65.6
Effortmedium
Tools—

Keywords: a/b test, ab-test, abtest-config, experiment, firebase a/b testing, firebase experiment, remote config experiment, variants

/t1k:unity:mobile:ab-test
[<experiment name>]

T1K — Firebase A/B Test (Remote Config experiment)

Section titled “T1K — Firebase A/B Test (Remote Config experiment)”

No API, CLI or MCP can CREATE or UPDATE an A/B experiment (firebase-tools and the REST API only have get/list/delete). Creation is Firebase Console only. This skill probes what it can with one script call, asks only what is still missing, then drives the Console to a Draft without further prompts and verifies the Draft from the Console.

  • In scope: driving experiment creation in the Console, the question round (often zero questions), Draft verification.
  • Out of scope: adding the parameters under test (use t1k-unity-mobile-remote-config-deploy), publishing templates (its step 8 gate), editing Packages/com.gdk.3rd. iOS-only projects are unsupported (the resolver requires google-services.json).
  1. Probe before asking, in ONE call. Run bun .claude/skills/t1k-unity-mobile-ab-test/scripts/ab-probe.cjs --param <key> [--param <key>] (it reuses the deploy skill’s resolver, fetches the live template once, reads RemoteConfigSetting.asset; --help lists flags and exit codes 0-5) and read its JSON facts: projectId, platforms, per params[] exists, live, repo, mismatch, conditionalValues, experiments, plus conditions, goals.toAdd and consoleUrl. In the SAME message start the browser setup of step 2 (so a signed-out profile is found BEFORE the user is asked anything). Echo the project id. Never hand-edit remoteconfig.template.json to fake an experiment: experiment-managed values are ABT-owned. A param with exists: false goes through the deploy skill first.

  2. Ask only what the prompt and the facts cannot answer (zero questions when nothing is missing): a. parameter(s) and variant value(s), only if the prompt lacks them; the BASELINE is live, never asked. b. audience: only when platforms has 2 entries and the prompt names none (the Console forces a condition; reuse an existing one from conditions if it fits). Everything else is pre-filled from references/experiment-defaults.json and printed in the summary, where the user changes any value; it is not asked. A mismatch: true (live vs repo default) is one line in the summary, never a question. Present the fully resolved summary (params, baseline vs variants, weights, audience, exposure, goals, name, project id, mismatch) and get approval through AskUserQuestion.

    Approving that summary IS the approval to Save. Save lands as a Draft, which does not affect players. Weights are immutable once started, so state them plainly in the summary.

  3. Account check. t1k-chrome-profile (chrome-profile.cjs list|launch|install-extension) opens the profile signed in to the Firebase project’s Google account; anthropic-skills:chrome-browser covers the Claude in Chrome tools. Verify the account email. Playwright MCP is a FALLBACK only (Google often blocks automated sign-in). Hand the human ONLY login / 2FA / passkey, then resume.

  4. Create autonomously, no further prompts. Open consoleUrl and use “Create Remote Config experiment” (never a list page). Fill the four panel steps with the Drive recipe in references/console-field-spec.md (refs, not coordinates). Stop at Draft; one final screenshot as evidence, not one per action.

  5. Verify the Draft from the Console, not the CLI: references/verify-and-delete.md § “Verify a Draft (Console)”. Report the experiment name and state (Draft), plus any Console warning (e.g. metric not yet in GA).

  6. Publish gate (separate). The experiment only runs once the Remote Config template is published (and possibly started). That affects live players: ask once via AskUserQuestion, then go through the deploy skill’s gated flow. Never click Publish changes / Start without that yes.

remoteconfig:experiments:delete / rollouts:delete only on explicit user instruction. Never automated.

Exercised end-to-end once (2026-10-21, theonefeature, Android, interstitial_ad_interval 90 vs 30, saved as Draft). The default metric labels were confirmed in the Goals picker on 2026-10-21 (en-dash retention ranges). NOT yet exercised against a live Console: scripts/ab-probe.cjs (offline tests only) and the Drive recipe.

  • Weights cannot change after start; a wrong split means delete and recreate.
  • A running experiment can make a template publish report success without the version advancing (deploy skill step 10 guards it): always check versionNumber.
  • Limits: 300 experiments per project, 24 running at once.
  • A Google sign-in prompt or 2FA is the human’s; never type credentials.
  • With no existing condition the Targeting dropdown offers only “Create new condition”, which also writes that condition into the template. The dialog resizes after the user-match estimate loads: re-screenshot before clicking Create condition or the click lands on empty space.
  • Console pre-adds Retention (4–7 days) and Crash-free users as secondary metrics: add only goals.toAdd from the probe (3 by default).
  • Save lands as Draft and raises “Changes are pending publish”; the Draft does not run.
  • A metric warning “never triggered / not in Google Analytics yet” is expected on low-traffic projects and does not block Save.
  • On a Console misfire or layout surprise, emit [t1k:skill-bug kit="theonekit-unity" skill="t1k-unity-mobile-ab-test" bug="<one-line>" evidence="<path-or-id>"].
  • Escape safely closes an open dropdown; the panel’s Next button sits below the fold, scroll the panel first.
  • scripts/ab-probe.cjs — step 0 facts probe (tests: tests/ab-probe.test.cjs, tests/skill-contract.test.cjs).
  • references/console-field-spec.md — every field, step order, Drive recipe, allowed values, validation errors.
  • references/experiment-defaults.json — studio defaults pre-filled without asking (goals, exposure, naming).
  • references/verify-and-delete.md — Draft verification (Console), post-publish CLI checks, the delete commands.

Main session (inline); scripts/ab-probe.cjs is the read-only probe, every other step is judgment.