Skip to content

LTX-2 minor fixes#373

Open
prishajain1 wants to merge 1 commit intomainfrom
prisha/ltx2_fixes
Open

LTX-2 minor fixes#373
prishajain1 wants to merge 1 commit intomainfrom
prisha/ltx2_fixes

Conversation

@prishajain1
Copy link
Copy Markdown
Collaborator

@prishajain1 prishajain1 commented Apr 8, 2026

This PR introduces the following changes for the LTX2 pipeline:

  • Added Context parallelism replication for audio stream in transformer (this resolved some audio related issues occurring in particular sharding configurations on ironwood and seems to reduce the latency for other cases. Since audio stream is only 126 tokens, this shouldn't be an issue even if replicated)
  • Replicated VAE params and latents before call to decode method. With this change, single video latency dropped by 3x (observed no OOM in different configs caused by this change, tested on trillium and ironwood 8 chips)
  • Added flash_min_seq_length parameter to config file, now the choice of attention is controlled by this parameter. Experimentally observed that having dot attention for audio to audio self attn and video to audio cross attn and flash for audio to video provided more speed up. We had to hardcode "flash" as the attention choice for audio to video, because the way the attention mechanism is currently decided bases on flash min seq length compares the seq length for all q, k/v and flash is chosen only if all have seq length > flash min seq length. Experimentally we observed more speed ups if audio to video was kept in flash instead of dot.
  • Added JAX annotations for profiling
  • jitted the step() function in scheduling_flow_match_flax.py
  • Converted the timesteps ‘t’ from static Python integer/float to a dynamic JAX tensor (to prevent compilation of a program for each and every timestep)

Results

We performed the experiments below for dp = 2, cp = 2, tp = 2, fsdp = 1, per device batch size = 0.125

Hardware Status Time (s) Video Link
v7x-8 Current main 105.14 View Video
After fix 28.63 View Video
v6e-8 Current main 138.24 View Video
After fix 58.74 View Video

@prishajain1 prishajain1 requested a review from entrpn as a code owner April 8, 2026 03:23
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

Copy link
Copy Markdown
Collaborator

@Perseus14 Perseus14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. Few minor comments.

Please add few results comparing baseline and this branch to get a better idea of the improvements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants