Skip to content

t1k:unity:editor:headless-capture

FieldValue
Moduleeditor
Version3.50.7
Effortmedium
Tools—

Keywords: capture frames, contact sheet, gameplay clip, gameplay video, headless capture, inspector screenshot, inspector window capture, screen locked capture, xvfb capture

/t1k:unity:editor:headless-capture
<what to show> [size e.g. 540x960] [fps]

Headless gameplay capture (video and frame screenshots)

Section titled “Headless gameplay capture (video and frame screenshots)”

Records real gameplay frames from a Unity Editor running without a display: it works while the machine is locked and never touches the user’s open Editor. Output is a PNG frame sequence, encoded to an H.264 clip, then verified with a contact sheet and a per-frame colour count before anyone sees it.

One variant needs a window instead of Xvfb: the Editor Inspector of an asset, page by page. It is a windowed launch on the real display, because the pixels come from the window’s native host view — recipe and every gotcha in references/inspector-capture.md.

Rendering is the stated need that justifies leaving -nographics (rules/unity-headless-by-default.md): Xvfb for a frame sequence, the real display for an Inspector window.

NeedUse
A clip or a frame sequence of something moving (VFX, animation, a hint, a transition)this skill
The real Inspector window of an asset or component, page by page (layout, group order, attribute drawers)this skill, windowed: references/inspector-capture.md
One still of a scene whose content exists outside Play Modet1k-unity-editor-look-spec (LookSpecHeadlessShot)
A still from an Editor that is already open for this projectt1k unity eval with ScreenCapture (t1k-unity-editor-cli)
Values only, no windowlog them from an -executeMethod harness; do not photograph an Inspector

Prior art swept before this skill: a one-PNG awaited capture inside an AutoTestKit run (PuzzleGame ScreenCapturer.CapturePngAsync) and a Game-view menu capture (DOTSCore.Editor.ScreenCaptureUtil). Neither records a frame sequence from a headless launch. If your game already has a capture helper, call it from the harness instead of the skeleton’s loop.

PathWhat
templates/run-capture.shLauncher: Xvfb + -force-glcore, licensing bridge, batch gate or timeout ceiling. Copy it out, do not run it from the skill dir
templates/InspectorCapture.cs.txtEditor harness that photographs an asset’s Inspector page by page. Drop the .txt, copy under Assets/Editor/HeadlessCapture/, delete after use
references/harness-skeleton.mdThe Editor harness every game needs (boot, wait N frames, capture M frames, exit), as an unproven generic skeleton
references/inspector-capture.mdThe Inspector-window recipe: windowed launch, host-view grab via reflection, Odin expand rounds, the headless-display dead end, every gotcha seen in the proven run
references/capture-walkthrough.mdOrder of operations on a real capture: plan the proof, warm worktree, compile-error table, healthy log sequence, scenario mode, troubleshooting. Read before the first capture
scripts/encode-clip.shFrames to H.264 yuv420p mp4 with faststart, optional top crop
scripts/contact-sheet.pyTile chosen frames into one PNG (ffmpeg only)
scripts/color-count.shPer-frame count of pixels inside an RGB box (ffmpeg only)

Licensing: the one thing that may need a human

Section titled “Licensing: the one thing that may need a human”

A Personal license lives in the Hub’s licensing client. When Unity Hub is installed as a Flatpak, that client runs inside the sandbox and its sockets live in the sandbox’s /tmp, so a headless Editor started outside it logs No valid Unity Editor license found. templates/run-capture.sh finds the client by its --namedPipe argument and bridges its two unix sockets into the host /tmp with socat, and removes them on exit. A native Hub shares /tmp; the script detects that and skips the bridge.

Try the launch from Bash first. The Claude Code auto-mode classifier can deny an agent-started socket bridge: if it does, never edit the script or find another route around it, hand the user ONE line to paste (keep the script path short, a wrapped paste breaks it):

! nohup /tmp/run-capture.sh <project-root> <out-dir> <Namespace.Class.Method> [args...] > /tmp/run-capture.out 2>&1 &

Prerequisites: Unity Hub open and signed in, the project’s Editor version installed, xvfb-run, ffmpeg, and socat when the Hub is a Flatpak.

Plan the proof first (log the state you show, per frame): references/capture-walkthrough.md § 0.

  1. Pick a worktree, never the user’s open Editor (one Unity per project, rules/unity-headless-by-default.md). The first launch in a fresh worktree imports the whole project and can take many minutes; a Library/ copy from the main checkout (a reflink copy where the filesystem supports it) shortens it.
  2. Add the harness from references/harness-skeleton.md under Assets/Editor/HeadlessCapture/ with its own Editor-only asmdef. Keep it untracked. Put the game-specific trigger (the thing you want to film) in the marked spot.
  3. Pick content where the effect is visible: a level, board or scene that is big enough to show it.
  4. Launch (yourself, or the user via the ! line above): templates/run-capture.sh <project-root> <out-dir> <Namespace.Class.Method> -captureScene <Assets/...unity> .... Arm a Monitor on <out-dir>/unity.log filtering : error CS[0-9]+, your harness log tag, Aborting batchmode and No valid Unity Editor license. Do not filter on Exception: ordinary warning stack traces contain it.
  5. Encode: scripts/encode-clip.sh <frames-dir> <out.mp4> [fps] [crop-top-px].
  6. Verify before showing anyone. A clip that looks plausible can still be wrong:
    • scripts/contact-sheet.py <frames-dir> <out.png> 30 59 100 145 tiles chosen frames into one readable image.
    • scripts/color-count.sh <frames-dir> <rmin> <rmax> <gmin> <gmax> <bmin> <bmax> prints one pixel count per frame for a colour box. Use it to prove a pulse, fade or width change: expect a clean periodic or monotonic series, not noise.
    • Read the contact sheet natively with Read (always). When the model router’s vision role is installed, also run mr-role.cjs call --role vision --file <png>, but expect it to be exhausted (exit 4 or 5) most of the time: that is the usual state, not a failed check. Then rely on native Read plus crops of the regions that matter, and say which path produced each judgement. The t1k-unity-base-visual-director agent has no Write tool, so the caller saves its report to disk.
  7. Share where the project’s visual-change and status-thread rules say to, and run the human-writing skill (when installed) on the caption first.

Running the launcher and the three helper scripts and reporting their output carries no judgment, so a cheap-tier agent runs them. Choosing what to film, writing the harness trigger and judging the verified frames stay with the caller.

StepAgent
Run scripts/encode-clip.sh, scripts/contact-sheet.py, scripts/color-count.sh and report the outputt1k-fact-fetcher
Launch templates/run-capture.sh and watch its logcaller (licensing may need the user)

Recipe the launcher and skeleton follow (do not re-learn these)

Section titled “Recipe the launcher and skeleton follow (do not re-learn these)”
NeedDo
No displayxvfb-run -a -s "-screen 0 1280x1024x24" + -batchmode -force-glcore (llvmpipe). Not -nographics: it has no renderer, so no frames and WaitForEndOfFrame never fires
Deterministic framesTime.captureFramerate = fps, then wait on frame counts, never seconds
Grab a frameScreenCapture.CaptureScreenshotAsTexture() after WaitForEndOfFrame, write frame_%05d.png
ExitEditorApplication.Exit(code) from EnteredEditMode. Never -quit: it exits before the Play session runs
Domain reloadkeep harness state in SessionState and re-subscribe from [InitializeOnLoad], or enter Play Mode the way the game’s own test tooling does
Ceilingthe kit batch gate (unity-batch-gate.sh, T1K_UNITY_BATCH_TIMEOUT_SEC) when present, else timeout; never both
Unity versionread from ProjectSettings/ProjectVersion.txt; override with UNITY_BIN
  • Whole-frame statistics and a glance at one still both lie for thin glowing lines: the whole-frame mean moved under 1% while a line’s glow tripled. Use the colour count.
  • ffmpeg geq reads r()/g()/b() only on planar RGB: add format=gbrp or you get “Unknown function”.
  • ffprobe -of csv=p=0:s=' ' does not parse; use -of csv=p=0 and IFS=, read.
  • The helper scripts need ffmpeg only (no numpy or PIL), so they run on a bare dev machine.
  • Unity churn in the capture worktree (Packages/packages-lock.json, ProjectSettings/*, a few .meta files) is not part of your change. Never commit it.
  • With the batch gate present, a loaded host (low MemAvailable, swap above T1K_UNITY_BATCH_MAX_SWAP_PCT, or all slots taken) makes the launcher WAIT up to T1K_UNITY_BATCH_WAIT_SEC before Unity starts, then exit 75. A quiet log in the first minutes is the gate queuing, not a hung Editor: read the gate’s stderr line in the launcher output.
  • A timed-out launch may still have written frames: check <out-dir>/frames and the log before relaunching (rules/unity-forbidden-operations.md).
  • An Inspector capture is a WINDOWED launch, not a headless one. Two things bite: the kit’s Unity guard reads a raw windowed -executeMethod launch as a violation (escape T1K_UNITY_RAW_LAUNCH_OK=1), and a headless virtual display cannot host it — under cage/wlroots Unity floats every window at ~200x200 regardless of minSize, and a maximized docked Inspector grabs blank gray. Use the real display, -logFile, and a timeout -k 60 900 ceiling (references/inspector-capture.md).
  • EditorApplication.delayCall never fired under a windowed -executeMethod. Drive the harness from EditorApplication.update, ~60 frames plus !isCompiling && !isUpdating.
  • The first windowed launch on a project imports for minutes (~254 s observed) before -executeMethod runs. A quiet log is not a hang: watch for : error CS, not for silence.
  • A windowed Editor under cage started directly fails with No valid Unity Editor license found. The licence session lives only in the Unity Hub flatpak sandbox, so launch through the Hub wrapper (see Licensing above). Observed on Linux, Unity 6000.3.15f1, with cage present and Xvfb absent.
  • run-capture.sh once refused with N licensing clients are running whenever ANOTHER project’s Editor was open: that Editor embeds its own licensing client (.../Editor/Data/Resources/Licensing/Client, private per-version pipe). The script now ignores Editor-embedded clients and counts only Hub-side ones; a copy made before the fix still refuses, so re-copy the template.
  • run-capture.sh once ended with echo "[capture] unity exited with $?", so it always exited 0 and a caller never saw Unity’s status (a recorder’s exit 3 for a failed capture was swallowed). It now exits with Unity’s own status (the gate’s mapping, e.g. 75 when the gate refuses). A copy made before the fix still exits 0, so re-copy the template, and read the exit code, not just the log.
  • xvfb-run is not always installed. On a Linux machine with cage and no Xvfb, the headless recipe above does not start: use the windowed route (references/inspector-capture.md) or install Xvfb first.
  • A PlayMode [UnityTest] assembly definition needs an empty includePlatforms. An Editor-only platform list makes the assembly an EditMode one, so -runTests PlayMode finds zero tests.
  • A windowed -runTests also boots the project scene. The game’s own error logs then fail the test unless it sets LogAssert.ignoreFailingMessages = true.
  • Observed, unconfirmed: in a 200x200 cage window, WaitForEndOfFrame appeared never to fire for the Game view. This is inferred from a stalled probe, not isolated. In that case render a camera into a RenderTexture with an explicit Camera.Render() and read the pixels back.