JoyAI-Echo × LTX-2.3 Surgical Merge (e50) - bf16, fp8, INT8, GGUF (Q4_0 / Q5_0 / Q8_0) - Reference Audio Talking Head Lip Sync

**CIVITAI IS RENAMING EVERY DOWNLOADED FILE THE SAME THING. MAKE SURE YOU SAVE-AS AND NAME THE FILE SO YOU KNOW WHAT IT IS.**

Everything here is free and stays free — the format spec, the nodes, the workflows, the cartridges, the LoRAs. If it saved you a night of debugging (it contains several hundred of mine), tips keep the 5090 warm:

• ☕ Buy me a coffee on Ko-fi

• 💛 Sponsor on GitHub

• 🔁 Liberapay (recurring)

• ⚡ Or right here: the Civitai tip button on this page sends Buzz directly.

JoyAI-Echo's video branch married to LTX-2.3-distilled-1.1's audio branch — so you get Echo's identity-holding multishot behaviour with an audio stack the TalkVid ID-LoRA still works against.

The e50 builds are the current recommendation for reference-voice work — driving a character from a supplied voice clip at conversational speech rates.

This is a targeted fix, not a general lip-sync repair. Under the JoyAI-Echo pack's own production recipe — slower, deliberate delivery and memory-bank conditioning — the original merge already scores 9.5+ on sync. The deficit e50 addresses shows up specifically in the reference-audio path (LTXVAddGuide + LTXVReferenceAudio + the TalkVid ID-LoRA) at ~2.5–3 words/sec. If that is not how you drive it, the original merge is not broken and you do not need this.

Speed: turn on fp8_scaled_mm

If your renders crawl in the JoyAI-Echo pack, this is almost always why. It is off by default. It stores the DiT linears as float8_e4m3fn and computes the matmuls natively via torch._scaled_mm on RTX 40/50-series — unlike fp8_transformer there is no per-layer upcast tax.

• bf16 + toggle ON — linears downcast at load, native fp8 compute: ~22 GB resident, sequential_offload can go OFF

• INT8 ConvRot + toggle ON — reconstructs to bf16, then downcasts: identical to bf16, at ~60% of the download

• bf16 or INT8, toggle off — runs bf16: ~40 GB staged, streams over PCIe on a 32 GB card

• fp8 file, toggle off — upcasts to bf16 at load: saves download size only

• fp8 file + toggle ON — hard error. The toggle must start from bf16

Short version: feed it bf16 or INT8 and turn the toggle on. The fp8 file is for running without the toggles.

Files in this listing

Every e50 format is here — full checkpoints carry the DiT, both VAEs, the vocoder and the text projection, so they need no companion download. The GGUFs are DiT-only.

• ltx23_echoVid-ltxAud_surgical_e50_bf16.safetensors — 43.0 GB, reference precision. Pair with fp8_scaled_mm ON — this is the combination most people should run.

• ltx23_echoVid-ltxAud_surgical_e50_int8_convrot.safetensors — 25.7 GB, INT8 ConvRot. Reconstructs to bf16 at load, so with fp8_scaled_mm ON it is identical in behaviour to the bf16 file at ~60% of the download. Also runs on stock ComfyUI's native W8A8 loader.

• ltx23_echoVid-ltxAud_surgical_e50_fp8.safetensors — 23.4 GB. For stock ComfyUI, or the pack with the toggles off. Cannot be paired with fp8_scaled_mm — see the matrix above.

• LTX23-echoVid-ltxAud-Surgical-e50-DiT-Q8_0.gguf — 23.1 GB, 0.4% spot error. The GGUF to take unless VRAM forces otherwise.

• LTX23-echoVid-ltxAud-Surgical-e50-DiT-Q5_0.gguf — 15.5 GB, 3.3% spot error. 16 GB cards.

• LTX23-echoVid-ltxAud-Surgical-e50-DiT-Q4_0.gguf — 13.0 GB, 7.1% spot error. Tightest VRAM; the quality cost is visible.

Any GGUF is DiT-only — keep a full bf16 checkpoint in checkpoint_path to supply the VAEs, vocoder and connectors.

Which file do I want?

• Reference-voice talking heads at conversational pace — start here: e50_bf16 + fp8_scaled_mm ON

• Same, but 40% less to download: the INT8 ConvRot build + fp8_scaled_mm ON

• Stock ComfyUI, or the pack with toggles off: e50_fp8

• 16 GB card: e50-DiT-Q5_0.gguf (plus a full bf16 for the VAEs)

• Generated-voice multishot, or the pack's slower recipe: the original full-Echo surgical merge — it is not outclassed here

What e50 actually is

The video side is rebuilt from two independent doses rather than taken verbatim:

video          = dev + 1.0 x (distilled-1.1 - dev) + 0.5 x (JoyAI-Echo - dev)
tables/adalns  = dev + 0.3 x (distilled-1.1 - dev) + 0.5 x (JoyAI-Echo - dev)
audio + cross-modal = stock distilled-1.1   (unchanged, so TalkVid still works)

Why. JoyAI-Echo is a dev-lineage finetune — its training fingerprint matches ltx-2.3-22b-dev, not distilled-1.1 — so on the 8-step distilled sigma ladder its video stack is under-distilled. That surfaces as under-articulated mouths and smeared fine motion when the mouth is being driven hard by reference audio at speed. Adding the full distillation direction back restores articulation; holding Echo's own finetune at 0.5 recovers stock's crisper viseme formation, clearest on rounded vowels. The modulation tables take a lighter distill dose because at full strength they produce specular, over-shiny skin.

Both doses were chosen by rendering and eye, not theory: a full-strength distill correction reads rubbery, stock alone reads stiff, and 0.5 beat 0.6 and 0.7 on viseme shape.

Known trade. Echo's finetune suppresses the occasional robotic voice and drives the pack's audio memory bank. At 0.5 you keep half of it.

Shot length is a nodes issue, not a weights issue

Through an unpatched pack, lip sync visibly breaks about 9.6 s into every shot — the video RoPE clock is hardcoded to 24 fps while audio runs in true seconds, so a 25 fps render drifts ~4%/s. No checkpoint can fix it; the multishot patch can (Bug fix #0). With it, 60–105 s multishot masters hold sync.

Sampling

1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875, 0.0

cfg 1, euler — but note cfg 1 makes negative prompts inert, so raise to ~1.3 if you need one.

License

LTX-2 Community License, inherited. Non-commercial: LTX-2 itself permits commercial use below $10M ARR, but the JoyAI-Echo weights this is merged from are non-commercial, so the merge is too.

Everything else I've published

• All models on Hugging Face

• JoyAI-Echo multishot patch — nodes, releases, issues

• LTX-2.3 Multishot Lite — the core-node workflow