Skip to content

t1k:jenkins-autotest

FieldValue
Modulet1k-devops
Version4.1.3
Effortmedium
Tools—

Keywords: AutomationTest job, autotest dashboard, autotest jenkins, autotest node image, autotest sweep job, AutoTest-Dashboard, autotestkit jenkins, level sweep jenkins, LIB_REF

/t1k:jenkins-autotest
<job-name> [--from <template-job>] [--repo <game-repo-url>] [--thread <discord-thread-id>]

The sweep job runs autoTestUnifiedPipeline(...) from the AutoTestKit Unity library against one game. It is ONE job per game for every target, selected by the TARGET parameter:

  • TARGET=editor (the default sweep): syncs the game, checks the manifest pin (version lock), runs the editor in a container, and archives report.json / report.html.
  • TARGET=phone: builds the APK on a Unity node, then sweeps a real phone over adb (see “Phone and emulator targets”).
  • TARGET=emulator: unproven; it has never run through the unified job (references/unified-job.md).

The separate device jobs were deleted on 2026-10-28; their build numbers in older notes are history, not current state.

This skill is only the AutoTest layer on top of two others, which it does not repeat:

  • creating or mutating the job config: t1k-jenkins-job (gen, create, update, verify; always read its “Never inherit a thread ID” section first);
  • everything about the kit itself (parameters, version lock, report): the package docs in com.theone.autotestkit.unity (Jenkins~/README.md) and skill t1k-unity-testing-autotestkit (kit theonekit-unity, module testing; install it to read the skill).

Every AutoTest job goes in the Jenkins folder UnityAutomationTest (https://jenkins.the1studio.org/job/UnityAutomationTest/), never the root, a game’s own folder, or a per-person folder. The folder argument of every jenkins-job.cjs call below is the literal UnityAutomationTest; the full job name is UnityAutomationTest/<Game>-AutomationTest. Existing jobs there to use as --base templates: Arrow3D-AutomationTest, VoxelColor-AutomationTest, AutoTestKit-UnitySample; the dashboard is AutoTest-Dashboard. Do not create the folder (it exists); if a call reports it missing, stop and ask.

  1. The game’s Packages/manifest.json pins com.theone.autotestkit and com.theone.autotestkit.unity to one exact version and carries scoped registries for com.theone and com.cysharp.
  2. The game repository has a Jenkinsfile/AutoTest that loads the library and calls autoTestUnifiedPipeline (its parameter list is declared by the library and dispatches on TARGET); its LIB is the release commit of that same version (see “Derive LIB_REF”). Arrow3D and VoxelColor both call it from Jenkinsfile/AutoTest.
  3. The Jenkins nodes you can land on have the Unity editor image for unityVersion.

LIB_REF is never typed or copied from another job: it is pinned from the versions of the two automation-test packages in the game’s Packages/manifest.json (com.theone.autotestkit and com.theone.autotestkit.unity), so the library the job loads is always the one the game compiles against. Resolve it, then pass that value to gen --set LIB_REF= and verify --expect LIB_REF=:

Terminal window
M=<project folder>/Packages/manifest.json; R=https://upm.the1studio.org/
V1=$(jq -r '.dependencies["com.theone.autotestkit"] // empty' "$M"); V2=$(jq -r '.dependencies["com.theone.autotestkit.unity"] // empty' "$M")
[ -n "$V1" ] && [ "$V1" = "$V2" ] || { echo "pins differ or missing: '$V1' vs '$V2'" >&2; exit 1; }
G1=$(bun view "com.theone.autotestkit@$V1" gitHead --registry $R); G2=$(bun view "com.theone.autotestkit.unity@$V2" gitHead --registry $R)
[ -n "$G1" ] && [ "$G1" = "$G2" ] || { echo "gitHead differs or missing: '$G1' vs '$G2'" >&2; exit 1; }
LIB_REF=$G1

Stop on a non-zero exit instead of falling back to a hand-picked ref: the two pins must be one exact version (no range, no git URL) and both packages must come from the same release commit. When the manifest pins move, re-derive and update the job’s LIB_REF and the Jenkinsfile’s LIB together.

Terminal window
S=~/.claude/skills/t1k-jenkins-job/scripts/jenkins-job.cjs
F=UnityAutomationTest # fixed: see "Where the job lives"
bun $S get-config $F <template-sweep-job> --out template.xml # e.g. Arrow3D-AutomationTest
bun $S gen --base template.xml --out job.xml \
--repo git@github.com:The1Studio/<Game>.git --branch '${BRANCH}' --script-path Jenkinsfile/AutoTest \
--set BRANCH=<default branch> --set TARGET=editor --set AUTOTEST_RANGE=0..49 \
--set PARAM_BUILD_FILE_NAME=<project folder without "Unity"> --set PARAM_UNITY_TOOL_NAME=<unityVersion> \
--set PARAM_DISCORD_THREAD_ID=<this game's thread, or empty> --set LIB_REF=$LIB_REF # from "Derive LIB_REF"
bun $S create $F <job-name> --from job.xml && bun $S verify $F <job-name> --expect LIB_REF=$LIB_REF

What differs from a Unity build job: the pipeline is loaded from the game’s own Jenkinsfile, and the parameter list is owned by the library, which rewrites the job’s parameters from the Jenkinsfile on every run (so a template’s list only has to be close). Clear or replace PARAM_DISCORD_THREAD_ID and set PARAM_SHOULD_NOTIFY_TO_CHAT_CHANNEL=false for a job that must not post.

  • First build after any release: pass LIB_REF=<sha> explicitly — the sha derived from the manifest pins (see “Derive LIB_REF”), never a value carried over from another job. A repointed job keeps its stored LIB_REF default, and that stored parameter wins over the Jenkinsfile’s LIB (params.LIB_REF ?: LIB), so an old default loads the OLD library.
  • Always pass BRANCH explicitly: the stored default may be an old feature branch carrying the old Jenkinsfile.
  • Smoke before a full sweep: AUTOTEST_RANGE=0..4 (about 6 to 7 minutes), then 0..49 (about an hour, or AUTOTEST_SHARDS=2).
  • Exit code 0 SUCCESS, 1 UNSTABLE (error findings), 2 FAILURE (aborted). Read the result from report.json (summary.done == summary.total, screenWidth x screenHeight), not from the colour of the build.
  • Exit 125 / Unable to find image is a node without the Unity image, not a code defect. Re-run on a node that has it with a replay (replayBuild, agentLabel: '<node label>') instead of editing the repository.
  • Every trap that costs a build — the No such DSL method 'autoTestUnifiedPipeline' failure from a stale LIB_REF, the sv-2 swap gate, judging a build by the wrong library, and a phone that is not ready — is in references/unified-job.md § “Operating traps”, with its symptom and remedy. Read it before the first run on a repointed job.

TARGET=editor|phone|emulator picks the sweep. Its parameter list is the library’s: read the Unified job section of the package’s Jenkins~/README.md before relying on a row. The operating traps, the phone target prerequisites and the “When a run fails” table are in references/unified-job.md; do not restate them here.

UnityAutomationTest/AutoTest-Dashboard renders the trend of the other AutoTest jobs in the folder. Its pipeline script is a shim that loads the same Unity library at a commit constant and reads the kit version from a one-line pin file in the game repository, so publishing a kit version means bumping that file and the constant together. Parameters: JOBS (comma-separated full job names), BUILDS_PER_JOB, LIB_REF (the ref the shim and its pin file are read from; a deleted branch here fails the job at checkout, set it to master).

TARGET=phone runs two stages: a Unity node builds the ARM64 APK (skipped when APK_SOURCE is given), then a phone node sweeps the device over adb on the NetBird mesh. Pass TARGET=phone, DEVICE_SERIAL=<host>:5555, and optionally DEVICE_NODE_LABEL.

Stage mechanics, the exit-code mapping, logcat and done.json capture, the sparse checkout around Git LFS, CI mode for untracked inputs and the historical per-device jobs are in references/device-sweep.md. Phone prerequisites are references/unified-job.md § “Phone target prerequisites”. The Unity side (bot, seams, stripping) is skill t1k-unity-testing-autotestkit.

Jenkinsfile/AutoTestDevice in the Arrow3D repository is unused now; copy Jenkinsfile/AutoTest and its TARGET, not that file. TARGET=emulator is unproven (needs deviceEnv: [ALLOW_TRANSLATED_ABI:'1']); say “unproven”, never “works”.

bun $S verify for the config; then one smoke build and a read of its report.json. Artifacts need the Jenkins credential (anonymous is 403).