t1k:unity:testing:autotestkit
| Field | Value |
|---|---|
| Module | testing |
| Version | 4.45.4 |
| Effort | medium |
| Tools | — |
Keywords: autotest onboard, autotest report, autotest sweep, autotest version lock, autotestkit, AutoTestKit.Unity.Cli, com.theone.autotestkit, gameViewSize, level sweep, report.json autotest
How to invoke
Section titled “How to invoke”/t1k:unity:testing:autotestkit<adopt|run|read|upgrade> [version]AutoTestKit in a Unity game
Section titled “AutoTestKit in a Unity game”AutoTestKit sweeps a game’s levels unattended and writes report.json / report.html; a build fails on an error finding. A
game takes two UPM packages from the studio registry (https://upm.the1studio.org/), both at the same exact version:
com.theone.autotestkit (engine-neutral core) and com.theone.autotestkit.unity (the Unity adapter, with the Jenkins library
inside). Source and history: The1Studio/AutoTestKit.
The docs ship with the package, so read them from the installed version, not from memory:
Library/PackageCache/com.theone.autotestkit.unity@<version>/ holds README.md, Documentation~/adopt-in-a-unity-game.md,
Documentation~/checks-and-report-reference.md and Jenkins~/README.md.
Pick the task
Section titled “Pick the task”| Task | Read | Notes |
|---|---|---|
| Adopt it in a game | Documentation~/adopt-in-a-unity-game.md | Onboard scaffolds, Doctor checks the setup; the game writes its seams (driver, ready signal), a VContainer or Zenject game has its own scope instrumentation |
| Run a sweep on your machine | README.md § “Batchmode CLI” | -executeMethod AutoTestKit.Unity.Cli.Run; never -quit; the project must not be open in another editor |
| Run it on an Android device over USB | Documentation~/adopt-in-a-unity-game.md section 11 | Editor-free player sweep, one device; see “Run on a USB device” below |
| Run it on Jenkins | skill t1k:jenkins-autotest | the job is created and operated by that skill; this skill does not repeat it |
| Read a result | Documentation~/checks-and-report-reference.md | summary.done == total and no error finding is a clean run; finding kinds are tabulated there. A tester who runs or reads the job: t1k:qa:base:autotest-job, if the QA kit is installed |
| Write a play bot, seams or a device build | references/bot-and-device-integration.md | questions before a bot, Arrow3D map, link.xml/stripping, level-order snapshot, side-effect probes, permission grant; Jenkins/phone side is t1k:jenkins-autotest |
| Upgrade the version | below | manifest, lock and Jenkinsfile move together |
| Release a package version (AutoTestKit maintainers) | references/release-and-publish.md | bump, pins, changelogs, publish, registry check, merge commit |
Run on a USB device
Section titled “Run on a USB device”Android only, one device, no Jenkins. The package’s Documentation~/adopt-in-a-unity-game.md section 11 is the reference. In
The1Studio/AutoTestKit, unity/tools/run-sample-device.sh builds the sample APK (IL2CPP, ARM64, Development; 10 to 20 minutes the
first time) and sweeps levels 0..4 on the device. A game calls the package’s Tools~/autotest/device_run.sh --apk --package --flags --out
itself. Exit codes are in the package’s Tools~/autotest/device-contract.json: 0 clean, 1 findings, 2 aborted inside the app,
3 timeout, 4 environment (adb, device, install, pull), 5 the app died, 6 the pulled report failed its contract check.
Before a run the device needs USB debugging authorised (adb devices shows device, not unauthorized), “Install via USB”
allowed, and the screen on and unlocked. Set ADB_SERIAL when more than one device is attached. The acceptance gate for a device is
verify-device-sweep.sh in The1Studio/AutoTestKit, which needs ADB_SERIAL and fails on an empty coverage, a missing
IAutoTestRunInstrumentation seam, a level that is not Done, or an error finding.
- Version lock. The manifest pin of
com.theone.autotestkit.unitymust equal thepackage.jsonversion at the ref the Jenkinsfile loads the library from. Any other combination stops the run before a container starts. LIB_REFwins. The job parameter overrides the Jenkinsfile’sLIB; after an upgrade the first build passesLIB_REF=<release commit>.gameViewSizeceiling. On Unity 6000.3.25f1 a Game view above about 2.2 million pixels is enlarged (1080x2280becomes2173x4588) and the run carries agameViewSizeNotAppliedWarning.540x1140is exact. The size is part of a run’s identity: changing it re-baselines once (frameSizeMismatch).- No graphics, no size. With snapshots off the editor runs
-nographicsand no size is applied. - Names shown in
report.htmlare display text. Renaming a step label changes no finding kind and no JSON field. - The release commit SHA is load-bearing. It is the registry’s
gitHeadand the ref games load the library from. Maintainers merge the AutoTestKit PR with a merge commit, never squash or rebase, or that SHA is on no branch. - A gitlink records a PR head too. A game repository that records the AutoTestKit submodule at a PR’s head breaks the same way
when the PR is squashed (the SHA is orphaned). Check the submodule with
git merge-base --is-ancestor <sha> origin/<default>. - Publishing is irreversible. A push touching any
package.jsonpublishes on any branch, and a published version is never changed or withdrawn. Never bump a version without an explicit go from the user. - Changelogs are bot-written. After a publish the bot pushes
docs: update changelogs for published packages [skip ci].git pull --ff-onlybefore the next push, and fold the hand-kept[Unreleased]entries into the release section.
Upgrade a game to a new version
Section titled “Upgrade a game to a new version”- Find the release commit:
bun info com.theone.autotestkit.unity@<version> gitHead --registry https://upm.the1studio.org/. - In the game: both pins in
Packages/manifest.json, the matching entries ofpackages-lock.json, andLIBin the game’s AutoTest Jenkinsfile (the release commit). - One Jenkins smoke (
AUTOTEST_RANGE=0..4) withLIB_REF=<release commit>; readreport.json. - Releasing AutoTestKit itself (bumping the packages, publishing, tagging) is maintainer work in The1Studio/AutoTestKit:
references/release-and-publish.mdhere, and the repository’sREADME.md§ “Releasing”.