t1k:unity:editor:headless-capture
| Field | Value |
|---|---|
| Module | editor |
| Version | 3.50.7 |
| Effort | medium |
| Tools | — |
Keywords: capture frames, contact sheet, gameplay clip, gameplay video, headless capture, inspector screenshot, inspector window capture, screen locked capture, xvfb capture
How to invoke
Section titled “How to invoke”/t1k:unity:editor:headless-capture<what to show> [size e.g. 540x960] [fps]Headless gameplay capture (video and frame screenshots)
Section titled “Headless gameplay capture (video and frame screenshots)”Records real gameplay frames from a Unity Editor running without a display: it works while the machine is locked and never touches the user’s open Editor. Output is a PNG frame sequence, encoded to an H.264 clip, then verified with a contact sheet and a per-frame colour count before anyone sees it.
One variant needs a window instead of Xvfb: the Editor Inspector of an asset, page by page. It is a windowed
launch on the real display, because the pixels come from the window’s native host view — recipe and every gotcha in
references/inspector-capture.md.
Rendering is the stated need that justifies leaving -nographics (rules/unity-headless-by-default.md): Xvfb for a
frame sequence, the real display for an Inspector window.
When to use which
Section titled “When to use which”| Need | Use |
|---|---|
| A clip or a frame sequence of something moving (VFX, animation, a hint, a transition) | this skill |
| The real Inspector window of an asset or component, page by page (layout, group order, attribute drawers) | this skill, windowed: references/inspector-capture.md |
| One still of a scene whose content exists outside Play Mode | t1k-unity-editor-look-spec (LookSpecHeadlessShot) |
| A still from an Editor that is already open for this project | t1k unity eval with ScreenCapture (t1k-unity-editor-cli) |
| Values only, no window | log them from an -executeMethod harness; do not photograph an Inspector |
Prior art swept before this skill: a one-PNG awaited capture inside an AutoTestKit run (PuzzleGame
ScreenCapturer.CapturePngAsync) and a Game-view menu capture (DOTSCore.Editor.ScreenCaptureUtil). Neither records a
frame sequence from a headless launch. If your game already has a capture helper, call it from the harness instead of
the skeleton’s loop.
| Path | What |
|---|---|
templates/run-capture.sh | Launcher: Xvfb + -force-glcore, licensing bridge, batch gate or timeout ceiling. Copy it out, do not run it from the skill dir |
templates/InspectorCapture.cs.txt | Editor harness that photographs an asset’s Inspector page by page. Drop the .txt, copy under Assets/Editor/HeadlessCapture/, delete after use |
references/harness-skeleton.md | The Editor harness every game needs (boot, wait N frames, capture M frames, exit), as an unproven generic skeleton |
references/inspector-capture.md | The Inspector-window recipe: windowed launch, host-view grab via reflection, Odin expand rounds, the headless-display dead end, every gotcha seen in the proven run |
references/capture-walkthrough.md | Order of operations on a real capture: plan the proof, warm worktree, compile-error table, healthy log sequence, scenario mode, troubleshooting. Read before the first capture |
scripts/encode-clip.sh | Frames to H.264 yuv420p mp4 with faststart, optional top crop |
scripts/contact-sheet.py | Tile chosen frames into one PNG (ffmpeg only) |
scripts/color-count.sh | Per-frame count of pixels inside an RGB box (ffmpeg only) |
Licensing: the one thing that may need a human
Section titled “Licensing: the one thing that may need a human”A Personal license lives in the Hub’s licensing client. When Unity Hub is installed as a Flatpak, that client runs
inside the sandbox and its sockets live in the sandbox’s /tmp, so a headless Editor started outside it logs
No valid Unity Editor license found. templates/run-capture.sh finds the client by its --namedPipe argument and
bridges its two unix sockets into the host /tmp with socat, and removes them on exit. A native Hub shares /tmp;
the script detects that and skips the bridge.
Try the launch from Bash first. The Claude Code auto-mode classifier can deny an agent-started socket bridge: if it does, never edit the script or find another route around it, hand the user ONE line to paste (keep the script path short, a wrapped paste breaks it):
! nohup /tmp/run-capture.sh <project-root> <out-dir> <Namespace.Class.Method> [args...] > /tmp/run-capture.out 2>&1 &Prerequisites: Unity Hub open and signed in, the project’s Editor version installed, xvfb-run, ffmpeg, and socat
when the Hub is a Flatpak.
Plan the proof first (log the state you show, per frame): references/capture-walkthrough.md § 0.
- Pick a worktree, never the user’s open Editor (one Unity per project,
rules/unity-headless-by-default.md). The first launch in a fresh worktree imports the whole project and can take many minutes; aLibrary/copy from the main checkout (a reflink copy where the filesystem supports it) shortens it. - Add the harness from
references/harness-skeleton.mdunderAssets/Editor/HeadlessCapture/with its own Editor-only asmdef. Keep it untracked. Put the game-specific trigger (the thing you want to film) in the marked spot. - Pick content where the effect is visible: a level, board or scene that is big enough to show it.
- Launch (yourself, or the user via the
!line above):templates/run-capture.sh <project-root> <out-dir> <Namespace.Class.Method> -captureScene <Assets/...unity> .... Arm aMonitoron<out-dir>/unity.logfiltering: error CS[0-9]+, your harness log tag,Aborting batchmodeandNo valid Unity Editor license. Do not filter onException: ordinary warning stack traces contain it. - Encode:
scripts/encode-clip.sh <frames-dir> <out.mp4> [fps] [crop-top-px]. - Verify before showing anyone. A clip that looks plausible can still be wrong:
scripts/contact-sheet.py <frames-dir> <out.png> 30 59 100 145tiles chosen frames into one readable image.scripts/color-count.sh <frames-dir> <rmin> <rmax> <gmin> <gmax> <bmin> <bmax>prints one pixel count per frame for a colour box. Use it to prove a pulse, fade or width change: expect a clean periodic or monotonic series, not noise.- Read the contact sheet natively with
Read(always). When the model router’s vision role is installed, also runmr-role.cjs call --role vision --file <png>, but expect it to be exhausted (exit 4 or 5) most of the time: that is the usual state, not a failed check. Then rely on nativeReadplus crops of the regions that matter, and say which path produced each judgement. Thet1k-unity-base-visual-directoragent has no Write tool, so the caller saves its report to disk.
- Share where the project’s visual-change and status-thread rules say to, and run the human-writing skill (when installed) on the caption first.
Executed by
Section titled “Executed by”Running the launcher and the three helper scripts and reporting their output carries no judgment, so a cheap-tier agent runs them. Choosing what to film, writing the harness trigger and judging the verified frames stay with the caller.
| Step | Agent |
|---|---|
Run scripts/encode-clip.sh, scripts/contact-sheet.py, scripts/color-count.sh and report the output | t1k-fact-fetcher |
Launch templates/run-capture.sh and watch its log | caller (licensing may need the user) |
Recipe the launcher and skeleton follow (do not re-learn these)
Section titled “Recipe the launcher and skeleton follow (do not re-learn these)”| Need | Do |
|---|---|
| No display | xvfb-run -a -s "-screen 0 1280x1024x24" + -batchmode -force-glcore (llvmpipe). Not -nographics: it has no renderer, so no frames and WaitForEndOfFrame never fires |
| Deterministic frames | Time.captureFramerate = fps, then wait on frame counts, never seconds |
| Grab a frame | ScreenCapture.CaptureScreenshotAsTexture() after WaitForEndOfFrame, write frame_%05d.png |
| Exit | EditorApplication.Exit(code) from EnteredEditMode. Never -quit: it exits before the Play session runs |
| Domain reload | keep harness state in SessionState and re-subscribe from [InitializeOnLoad], or enter Play Mode the way the game’s own test tooling does |
| Ceiling | the kit batch gate (unity-batch-gate.sh, T1K_UNITY_BATCH_TIMEOUT_SEC) when present, else timeout; never both |
| Unity version | read from ProjectSettings/ProjectVersion.txt; override with UNITY_BIN |
Hard rules
Section titled “Hard rules”- Whole-frame statistics and a glance at one still both lie for thin glowing lines: the whole-frame mean moved under 1% while a line’s glow tripled. Use the colour count.
- ffmpeg
geqreadsr()/g()/b()only on planar RGB: addformat=gbrpor you get “Unknown function”. ffprobe -of csv=p=0:s=' 'does not parse; use-of csv=p=0andIFS=, read.- The helper scripts need ffmpeg only (no numpy or PIL), so they run on a bare dev machine.
- Unity churn in the capture worktree (
Packages/packages-lock.json,ProjectSettings/*, a few.metafiles) is not part of your change. Never commit it. - With the batch gate present, a loaded host (low
MemAvailable, swap aboveT1K_UNITY_BATCH_MAX_SWAP_PCT, or all slots taken) makes the launcher WAIT up toT1K_UNITY_BATCH_WAIT_SECbefore Unity starts, then exit 75. A quiet log in the first minutes is the gate queuing, not a hung Editor: read the gate’s stderr line in the launcher output. - A timed-out launch may still have written frames: check
<out-dir>/framesand the log before relaunching (rules/unity-forbidden-operations.md). - An Inspector capture is a WINDOWED launch, not a headless one. Two things bite: the kit’s Unity guard reads a raw
windowed
-executeMethodlaunch as a violation (escapeT1K_UNITY_RAW_LAUNCH_OK=1), and a headless virtual display cannot host it — undercage/wlroots Unity floats every window at ~200x200 regardless ofminSize, and a maximized docked Inspector grabs blank gray. Use the real display,-logFile, and atimeout -k 60 900ceiling (references/inspector-capture.md). EditorApplication.delayCallnever fired under a windowed-executeMethod. Drive the harness fromEditorApplication.update, ~60 frames plus!isCompiling && !isUpdating.- The first windowed launch on a project imports for minutes (~254 s observed) before
-executeMethodruns. A quiet log is not a hang: watch for: error CS, not for silence. - A windowed Editor under
cagestarted directly fails withNo valid Unity Editor license found. The licence session lives only in the Unity Hub flatpak sandbox, so launch through the Hub wrapper (see Licensing above). Observed on Linux, Unity 6000.3.15f1, withcagepresent and Xvfb absent. run-capture.shonce refused withN licensing clients are runningwhenever ANOTHER project’s Editor was open: that Editor embeds its own licensing client (.../Editor/Data/Resources/Licensing/Client, private per-version pipe). The script now ignores Editor-embedded clients and counts only Hub-side ones; a copy made before the fix still refuses, so re-copy the template.run-capture.shonce ended withecho "[capture] unity exited with $?", so it always exited 0 and a caller never saw Unity’s status (a recorder’s exit 3 for a failed capture was swallowed). It now exits with Unity’s own status (the gate’s mapping, e.g. 75 when the gate refuses). A copy made before the fix still exits 0, so re-copy the template, and read the exit code, not just the log.xvfb-runis not always installed. On a Linux machine withcageand no Xvfb, the headless recipe above does not start: use the windowed route (references/inspector-capture.md) or install Xvfb first.- A PlayMode
[UnityTest]assembly definition needs an emptyincludePlatforms. An Editor-only platform list makes the assembly an EditMode one, so-runTests PlayModefinds zero tests. - A windowed
-runTestsalso boots the project scene. The game’s own error logs then fail the test unless it setsLogAssert.ignoreFailingMessages = true. - Observed, unconfirmed: in a 200x200
cagewindow,WaitForEndOfFrameappeared never to fire for the Game view. This is inferred from a stalled probe, not isolated. In that case render a camera into aRenderTexturewith an explicitCamera.Render()and read the pixels back.