Skip to content

t1k:unity:testing:autotestkit

FieldValue
Moduletesting
Version4.45.4
Effortmedium
Tools—

Keywords: autotest onboard, autotest report, autotest sweep, autotest version lock, autotestkit, AutoTestKit.Unity.Cli, com.theone.autotestkit, gameViewSize, level sweep, report.json autotest

/t1k:unity:testing:autotestkit
<adopt|run|read|upgrade> [version]

AutoTestKit sweeps a game’s levels unattended and writes report.json / report.html; a build fails on an error finding. A game takes two UPM packages from the studio registry (https://upm.the1studio.org/), both at the same exact version: com.theone.autotestkit (engine-neutral core) and com.theone.autotestkit.unity (the Unity adapter, with the Jenkins library inside). Source and history: The1Studio/AutoTestKit.

The docs ship with the package, so read them from the installed version, not from memory: Library/PackageCache/com.theone.autotestkit.unity@<version>/ holds README.md, Documentation~/adopt-in-a-unity-game.md, Documentation~/checks-and-report-reference.md and Jenkins~/README.md.

TaskReadNotes
Adopt it in a gameDocumentation~/adopt-in-a-unity-game.mdOnboard scaffolds, Doctor checks the setup; the game writes its seams (driver, ready signal), a VContainer or Zenject game has its own scope instrumentation
Run a sweep on your machineREADME.md § “Batchmode CLI”-executeMethod AutoTestKit.Unity.Cli.Run; never -quit; the project must not be open in another editor
Run it on an Android device over USBDocumentation~/adopt-in-a-unity-game.md section 11Editor-free player sweep, one device; see “Run on a USB device” below
Run it on Jenkinsskill t1k:jenkins-autotestthe job is created and operated by that skill; this skill does not repeat it
Read a resultDocumentation~/checks-and-report-reference.mdsummary.done == total and no error finding is a clean run; finding kinds are tabulated there. A tester who runs or reads the job: t1k:qa:base:autotest-job, if the QA kit is installed
Write a play bot, seams or a device buildreferences/bot-and-device-integration.mdquestions before a bot, Arrow3D map, link.xml/stripping, level-order snapshot, side-effect probes, permission grant; Jenkins/phone side is t1k:jenkins-autotest
Upgrade the versionbelowmanifest, lock and Jenkinsfile move together
Release a package version (AutoTestKit maintainers)references/release-and-publish.mdbump, pins, changelogs, publish, registry check, merge commit

Android only, one device, no Jenkins. The package’s Documentation~/adopt-in-a-unity-game.md section 11 is the reference. In The1Studio/AutoTestKit, unity/tools/run-sample-device.sh builds the sample APK (IL2CPP, ARM64, Development; 10 to 20 minutes the first time) and sweeps levels 0..4 on the device. A game calls the package’s Tools~/autotest/device_run.sh --apk --package --flags --out itself. Exit codes are in the package’s Tools~/autotest/device-contract.json: 0 clean, 1 findings, 2 aborted inside the app, 3 timeout, 4 environment (adb, device, install, pull), 5 the app died, 6 the pulled report failed its contract check.

Before a run the device needs USB debugging authorised (adb devices shows device, not unauthorized), “Install via USB” allowed, and the screen on and unlocked. Set ADB_SERIAL when more than one device is attached. The acceptance gate for a device is verify-device-sweep.sh in The1Studio/AutoTestKit, which needs ADB_SERIAL and fails on an empty coverage, a missing IAutoTestRunInstrumentation seam, a level that is not Done, or an error finding.

  1. Version lock. The manifest pin of com.theone.autotestkit.unity must equal the package.json version at the ref the Jenkinsfile loads the library from. Any other combination stops the run before a container starts.
  2. LIB_REF wins. The job parameter overrides the Jenkinsfile’s LIB; after an upgrade the first build passes LIB_REF=<release commit>.
  3. gameViewSize ceiling. On Unity 6000.3.25f1 a Game view above about 2.2 million pixels is enlarged (1080x2280 becomes 2173x4588) and the run carries a gameViewSizeNotApplied Warning. 540x1140 is exact. The size is part of a run’s identity: changing it re-baselines once (frameSizeMismatch).
  4. No graphics, no size. With snapshots off the editor runs -nographics and no size is applied.
  5. Names shown in report.html are display text. Renaming a step label changes no finding kind and no JSON field.
  6. The release commit SHA is load-bearing. It is the registry’s gitHead and the ref games load the library from. Maintainers merge the AutoTestKit PR with a merge commit, never squash or rebase, or that SHA is on no branch.
  7. A gitlink records a PR head too. A game repository that records the AutoTestKit submodule at a PR’s head breaks the same way when the PR is squashed (the SHA is orphaned). Check the submodule with git merge-base --is-ancestor <sha> origin/<default>.
  8. Publishing is irreversible. A push touching any package.json publishes on any branch, and a published version is never changed or withdrawn. Never bump a version without an explicit go from the user.
  9. Changelogs are bot-written. After a publish the bot pushes docs: update changelogs for published packages [skip ci]. git pull --ff-only before the next push, and fold the hand-kept [Unreleased] entries into the release section.
  1. Find the release commit: bun info com.theone.autotestkit.unity@<version> gitHead --registry https://upm.the1studio.org/.
  2. In the game: both pins in Packages/manifest.json, the matching entries of packages-lock.json, and LIB in the game’s AutoTest Jenkinsfile (the release commit).
  3. One Jenkins smoke (AUTOTEST_RANGE=0..4) with LIB_REF=<release commit>; read report.json.
  4. Releasing AutoTestKit itself (bumping the packages, publishing, tagging) is maintainer work in The1Studio/AutoTestKit: references/release-and-publish.md here, and the repository’s README.md § “Releasing”.