NeurIPS 2026
KAIST
A shared model for dance generation and understanding.
Rhythm through periodic phase. Expressivity through dance-specific language.
01 / OVERVIEW
Musical dance is organized into periodic units. Its character goes beyond a genre label.
Generating dance that reflects many aspects of music remains a fundamental challenge in dance motion synthesis. Existing music-conditioned approaches formulate the task as beat-level matching, which captures local synchrony but misses the compositional structure of rhythm, where movement is organized into global beat units. We argue that musical dance arises from the alignment of periodicity between motion and music, not from instantaneous beat coincidence.
To address this, we propose PhaseDance, a phase-conditioned framework that represents dance as a quasi-musical signal on periodic phase manifolds, enabling unified dance generation across multiple tasks. Alongside this, we ground text descriptions in fifteen music-correlated metrics derived from Laban Movement Analysis to capture the qualitative richness of dance that coarse genre labels cannot convey. Experiments show that PhaseDance produces choreography with stronger rhythmic coherence and richer expressive understanding than existing models.
02 / SEE IT IN MOTION
Phase-conditioned generation, rhythm control,
and phase ablations in the final demo.
Camera-ready demonstration · 2 min 53 sec • Download video ↓
The video could not be loaded. Please try the download link above.
03 / METHOD
Two complementary representations bring rhythmic structure and choreographic detail into a shared encoder–decoder model.
A deterministic phase signal is extracted from motion and aligned to the music’s period. It captures the position within a rhythmic cycle and works with music-paired and music-free clips.
No separate phase-encoder training stage.
Fifteen Laban-grounded motion descriptors are paired with music features. Kinetic geometry becomes a common language for energy, texture, spatial use, groove, and style.
Genre + movement + description + texture.
Separate modality branchesA frozen T5 text encoder and a trainable motion encoder preserve their respective representations.
A shared phase-aware decoderFour FFN experts and eight output-head experts route generation by rhythmic state.
Local motion contextWindowed causal attention handles motion generation; full attention supports text coherence.
Generate motion from a dance description and a supplied phase signal.
Continue a motion prefix while controlling the rhythm of what comes next.
Generate the middle of a sequence from its beginning and ending.
Describe a motion’s movement and expressive qualities without a phase input.
04 / RHYTHMIC CONTROL
Change the supplied phase frequency to steer the rhythmic character of a dance continuation.
Phase robustness. In the paper’s inference-time ablation (Table 9), fixed phase rates and displaced beats retain comparable trajectory errors, while removing phase degrades generation. The final demo includes the corresponding phase-ablation comparison.
Adjust the frequency below to see how the conditioning signal changes. This diagram illustrates the phase representation; it does not run the dance model.
05 / EXPRESSIVE UNDERSTANDING
Dance-to-Text captures movement names and qualities that generic action descriptions miss.
PhaseDance
In this house sequence, the dancer performs a farmer with punchy quality. Alternating hops transfer weight from one foot to the other while the free knee lifts and the arms swing in opposition.
UniMuMo
The motion is that a person is dancing on the spot.
Generated captions reproduced from Table 16. Examples can be switched using the menu above.
06 / EVALUATION
Evaluate the periodicity of motion and music, and the rhythmic identity of each dance genre.
BAS–Genre Pearson correlation on AIST++ for text-to-dance with synthetic phase.
Text-conditioned generation with synthetic phase, comparable to the reported music-conditioned baselines.
Dance-to-Text with LMA-grounded descriptions, compared with 12.80 for UniMuMo.
synthesis_phase is the deployment setting and uses phase synthesized from a nominal BPM. gt_phase uses phase extracted from the ground-truth test motion and is an oracle reference. Motion quality is competitive rather than uniformly best. The same checkpoint is used for both phase modes. BPS measures tempo agreement, while BAS–Genre Correlation measures agreement with the ground-truth rhythmic profile across genres.In the paper’s perceptual study, viewers preferred a period-matched, beat-shifted clip over a period-mismatched clip with higher beat alignment by 111 to 3 among forced-choice responses. Human ratings of genre-specific beat precision correlated with the ground-truth BAS profile at Pearson r = 0.81. See Appendix G for protocols and complete results.
07 / REFERENCE
@inproceedings{kim2026phasedance,
title = {PhaseDance: Capturing Rhythm and Expressivity in Dance Modeling},
author = {Kim, Meongeun and Lee, Taehui and Park, Soomin and Lee, Sung-Hee},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026}
}