t1k:extended-multimodal
| Field | Value |
|---|---|
| Module | t1k-multimodal |
| Version | 2.10.46 |
| Effort | medium |
| Tools | Bash, Edit, Read, Write |
Keywords: gemini, hailuo, image generation, media analysis, minimax, multimodal, music generation, nano banana, OCR, speech synthesis, transcribe, veo, video generation, vision analysis
How to invoke
Section titled “How to invoke”/t1k:extended-multimodal[file-path] [prompt]AI Multimodal
Section titled “AI Multimodal”Process audio, videos, documents using Gemini. Analyse an existing local image with the model-router vision role (mr-role.cjs call --role vision ..., below). Generate images, videos, speech, music via Gemini + MiniMax. The Gemini scripts stay for generation, video and audio.
# 1. Install Python dependenciespip install -r .claude/skills/t1k-extended-multimodal/scripts/requirements.txt
# 2. Set required API keyexport GEMINI_API_KEY="your-key" # https://aistudio.google.com/apikey
# 3. Set optional MiniMax key (for image/video/speech/music generation)export MINIMAX_API_KEY="your-key" # https://platform.minimax.io/user-center/basic-information/interface-keyImage analysis (describe, OCR, compare screenshots) of an existing local image goes through the model-router’s vision role, not through this skill’s Gemini scripts:
bun <claudeDir>/scripts/mr-role.cjs call --role vision --file <path> --prompt "<question>"--file repeats (up to 4). The command ships with the model-router module; when it is absent, native Read handles the image. Exit 4 or 5 also means native Read. Rule: modules/t1k-multimodal/rules/image-analysis-routing.md. Generation, speech, music, video and audio below are unchanged.
Video, hearing, dictation and web search also go through the model-router roles, not through the Gemini scripts (command text only; the router owns the model ids):
| Need | Role | Command |
|---|---|---|
Understand a video (.mp4, .mov, .webm) | video | bun <claudeDir>/scripts/mr-role.cjs call --role video --file <path> --prompt "<question>" |
Understand an audio file (.wav) | audio | ... call --role audio --file <path> --prompt "<question>" |
| Just the words of an audio file | dictation | ... call --role dictation --file <path> |
| Live web answer | web | ... call --role web --prompt "<question>" |
Run video, audio and dictation with a Bash timeout of 300000 ms and web with 150000 ms (the Bash default of 120 s is shorter than their worst cases). A prompt naming an existing .mp4, .mov, .webm or .wav path gets a [t1k:video-role] or [t1k:audio-role] frame from the media-routing-role hook (kill switch features.mediaRoleRouting). On exit 4 or 5: audio and dictation fall back to the Gemini audio path of this skill (scripts/gemini_batch_process.py); video has no native fallback (tell the user, offer ffmpeg frame extraction and the vision role); web falls back to native WebSearch/WebFetch. Speech, music and SFX are not roles and keep their MiniMax/Gemini scripts below. Rationale: docs/image-analysis-routing.md.
Optional — OpenAI-compatible backend (LiteLLM proxy / model-router)
Section titled “Optional — OpenAI-compatible backend (LiteLLM proxy / model-router)”Set GEMINI_OPENAI_BASE_URL to route generate (image) and the script’s analyze
through an OpenAI-compatible gateway instead of the native Gemini API.
GEMINI_API_KEY is reused as the bearer token. Only analyze+generate are
supported this way — audio/video/transcribe stay on native Gemini.
# LiteLLM proxy — vision + image generationexport GEMINI_OPENAI_BASE_URL="https://litellm.athena.tools"export GEMINI_API_KEY="sk-..." # proxy keypython scripts/gemini_batch_process.py --task generate --prompt "a green square" \ --model gemini-3.1-flash-image-preview --output out.pngpython scripts/gemini_batch_process.py --files out.png --task analyze \ --prompt "what color?" --model gemini-3.1-pro-preview
# TheOneKit model-router (CCS) — vision ONLY (no image-gen route)export GEMINI_OPENAI_BASE_URL="https://ccs.the1studio.org/api/provider/kimi"export GEMINI_API_KEY="$(gh auth token)" # The1Studio org membershippython scripts/gemini_batch_process.py --files img.png --task analyze \ --prompt "describe this" --model gpt-5.4-mini # or kimi-k2.6Quick Start
Section titled “Quick Start”Verify setup: python scripts/check_setup.py
Analyze audio/video/documents: python scripts/gemini_batch_process.py --files <file> --task <analyze|transcribe|extract> --help (a local image: the vision role command above)
Generate (Gemini): python scripts/gemini_batch_process.py --task <generate|generate-video> --prompt "desc"
Generate (MiniMax): python scripts/minimax_cli.py --task <generate|generate-video|generate-speech|generate-music> --prompt "desc" --help
Models
Section titled “Models”| Provider | Type | Model | Notes |
|---|---|---|---|
| Gemini | Image gen | gemini-3.1-flash-image-preview | Nano Banana 2 — DEFAULT |
| Gemini | Image gen | gemini-3-pro-image-preview | Nano Banana Pro — production / 4K text |
| Gemini | Video gen | veo-3.1-generate-preview | 8s clips with audio |
| Gemini | Analysis | gemini-2.5-flash | Recommended |
| MiniMax | Image gen | image-01 | $0.03/image |
| MiniMax | Video gen | MiniMax-Hailuo-2.3 | 1080p |
| MiniMax | Speech/TTS | speech-2.8-hd | 300+ voices, 40+ languages |
| MiniMax | Music | music-2.5 | 4-min songs with lyrics |
Scripts
Section titled “Scripts”gemini_batch_process.py: Gemini CLI — transcribe, analyze, extract, generate, generate-videominimax_cli.py: MiniMax CLI — generate, generate-video, generate-speech, generate-musicminimax_generate.py: MiniMax generation library for programmatic useminimax_api_client.py: MiniMax HTTP client, auth, async polling, file downloadmedia_optimizer.py: ffmpeg/Pillow preflight — compress/resize/convert media before API callsdocument_converter.py: Gemini-powered PDF/image/Office to markdown convertercheck_setup.py: Setup checker for API keys and dependencies
References
Section titled “References”| Topic | File |
|---|---|
| Vision/OCR of an existing local image | the mr-role.cjs call --role vision command above; references/vision-understanding.md is the Gemini-API fallback guide |
| Image Generation | references/image-generation.md |
| Image crop/resize/rotate/mask (local, no MCP; replaces the removed vision server’s jimp/rmbg tools) | references/local-image-recipes.md |
| Video Analysis | references/video-analysis.md |
| Video Generation | references/video-generation.md |
| Audio/TTS | references/audio-processing.md |
| Music Generation | references/music-generation.md |
| MiniMax API | references/minimax-generation.md |
Limits
Section titled “Limits”Formats: Audio (WAV/MP3/AAC, 9.5h), Images (PNG/JPEG/WEBP, 3.6k), Video (MP4/MOV, 6h), PDF (1k pages)
Size: 20MB inline, 2GB File API
Transcription: Audio/video >15 min must be chunked (15 min max per chunk) to avoid truncation.
Transcription format: [HH:MM:SS -> HH:MM:SS] content per segment, with metadata header.
Outputs
Section titled “Outputs”Save outputs to plans/multimodal-outputs/<YYYYMMDD>/.