Descriptions:
Veteran AI introduces an LTX Video Sound-to-Video (S2V) workflow that generates lip-synced video driven by an audio input, offering both a low-VRAM GGUF version and a standard-model version. The core distinction from previous LTX workflows is replacing the empty placeholder audio latent with a real encoded audio file, enabling the model to derive mouth movements and facial animation directly from speech.
The pipeline uses Kijai’s dedicated LTX Video audio encoder alongside GGUF-quantized main models via ComfyUI_GGUF. The distilled Q4 model runs at 8 steps, CFG 1.0, and LCM scheduling, generating at 1280×720. On the audio side, a clipped segment (typically 5 seconds from a longer track) is encoded with the Audio encode node, a zero-value mask is applied via the set latent noise mask node, and the resulting audio latent is combined with the video latent before sampling. For music tracks with background audio, an optional vocal separation node isolates the voice before encoding. A critical model-loading caveat: only GGUF models that contain embedded metadata are compatible with Kijai’s loading nodes.
Multiple test cases across different character types and audio segments demonstrate strong lip-sync accuracy, with the tutorial suggesting that longer audio tracks be split into sequential 5-second clips for batch generation. Both workflow versions are available on RunningHub’s ComfyUI platform, and the video cross-references the prior low-VRAM LTX Two tutorial for node-level setup details.
📺 Source: Veteran AI · Published January 15, 2026
🏷️ Format: Tutorial Demo
