MOTION-FORGE HuMo 17B — Speech-to-Video (Lip-Sync)

A speech-to-video character animation workflow powered by HuMo-17B, turning a single still into a talking-head performance driven by uploaded audio. Audio is encoded with Whisper-large v3, merged with CLIP prompt conditioning and a reference image to build motion latents in WanHuMoImageToVideo. KSampler with SD3 sampling head refines lip motion and facial dynamics, and VideoHelperSuite muxes frames with the audio at 25 FPS.​

Key features

• HuMo-17B GGUF backbone with SD3 sampling head for stable face dynamics and expressive lip sync.​

• Whisper-large v3 audio encoder for timing-aware mouth shapes and speech rhythm.​

• All-in-one subgraph encapsulating text prompt, dimensions, model names, and sampler settings for clean UX.​

Inputs

• Reference image loader, audio upload (MP3/WAV), positive/negative prompts for scene, style, and camera.​

• Defaults: 448×576, ~177 frames, 25 FPS output MP4 with audio.​

Recommended settings

• Sampler: UniPC, steps 4, denoise 1.0; SD3 sampling head scale 5.​

• LoRA: LightX2V at strength 1.1 for subtle motion enhancement without overbaking identity.​

Where each model goes

• Diffusion model (GGUF): HuMo-17b-Q8_0.gguf — loaded by LoaderGGUF as base model before SD3 sampling head. Place under models used by GGUF loader.​

• VAE: pig_wan_vae_fp32-f16.gguf — loaded by VaeGGUF and used by VAEDecode and WanHuMoImageToVideo. Place under models/vae.​

• CLIP text encoder: cow-umt5xxl-q8_0.gguf — loaded by ClipLoaderGGUF with vision “wan” and projection “default”. Place under models/clip.​

• LoRA: wan_loras/lightx2v_I2V_14B_480p_cfg_step_distill_rank128_bf16.safetensors — LoraLoaderModelOnly before ModelSamplingSD3, strength 1.1. Place under models/loras/wan_loras.​

• Audio encoder: whisper_large_v3_fp16.safetensors — AudioEncoderLoader and AudioEncoderEncode for speech features. Place under models/audio or the audio encoder directory.​

Node highlights

• WanHuMoImageToVideo: fuses CLIP conditioning, VAE, audio features, and reference image into motion latents.​

• ModelSamplingSD3 + KSampler: sets SD3 sampling parameters, then denoises with prompt control for identity stability.​

• VHS_VideoCombine: assembles frames and the original audio into an H.264 MP4 at 25 FPS with low CRF.​

• Subgraph IO: exposes width, height, length, gguf_name, lora_name, clip_name, vae_name, audio_encoder_name, sampler/scheduler, and seed for quick reuse