The pipeline ingests a rigged 3D model in USD format and builds two structured representations that the LLM can reason about: an Object JSON describing the skeleton at rest, and an Animation JSON capturing motion as sparse keyframes.
Animating arbitrary rigged 3D assets remains labor-intensive, while existing generative methods rarely combine broad asset support, fine-grained language control, iterative editing, and preservation of production assets. We present GenRA, an end-to-end framework for generating and refining skeletal animation directly in user-provided assets. GenRA conditions a multimodal large language model on the rig hierarchy and bind pose, multi-view renders, a natural-language instruction, and optional motion examples compactly encoded through keyframe decimation. The model produces sparse, bind-relative joint keyframes that are reconstructed into smooth animation curves and authored back into the original asset without modifying its geometry, skinning, or rig hierarchy. GenRA requires no task-specific training or retargeting and supports zero- and multi-shot generation together with localized, compositional, temporal, and quantitative language-based refinement. In 1,080 blind judgments across 40 diverse assets, GenRA achieves a 74.5% tie-aware preference rate over Puppeteer and is preferred in every evaluated category, while reducing median generation time by 83% and cost by 84%.
Click on any stage above to reveal the corresponding method details.
The pipeline ingests a rigged 3D model in USD format and builds two structured representations that the LLM can reason about: an Object JSON describing the skeleton at rest, and an Animation JSON capturing motion as sparse keyframes.
Joint names and parent-child relationships forming the rig hierarchy.
The rest pose of each joint — position, rotation, and scale.
Per-frame joint transformations decomposed into translation, Euler rotation, and scale.
Axis remapping normalizes all data to a Z-up, right-handed coordinate
system. The bind pose and hierarchy are serialized into Object JSON —
keys are hierarchical joint paths (parent/child) with
p, r, s (positions, rotations, and scales) values.
{
"root_j": {
"p": [0.0, 0.0, 0.0],
"r": [0.0, 0.0, 0.0],
"s": [1.0, 1.0, 1.0]
},
"root_j/body_main_j": {
"p": [0.25, 0.17, -2.54],
"r": [90.0, 0.0, 0.0],
"s": [1.0, 1.0, 1.0]
}
}{
"root_j/body_main_j": {
"0.00": {
"p": [0.25, 0.17, -2.54],
"r": [90.0, 0.0, 0.0]
},
"1.00": {
"p": [0.35, 0.17, -2.54],
"r": [45.0, 10.0, 0.0]
}
}
}Optionally, the pipeline renders annotated views of the model showing the skeleton overlay and coordinate axes. These images give the LLM spatial context about joint placement and orientation.
A dense source motion $\mathcal{M}_i$ is reduced to a sparse representation $\mathcal{K}_i = \mathsf{D}_{\varepsilon_0^2,\delta}(\mathcal{M}_i)$ using greedy Bézier-based decimation. Each scalar component of every joint's translation, rotation, and scale is processed independently.
For a scalar channel $x[n]$ where $n=0,\ldots,T_i-1$ represent the $n$-th frame of the motion $\mathcal{M}_i$, values are normalized by their maximum absolute magnitude to obtain $\widehat{x}[n]$. The normalized points $\mathbf{p}_n=(n,\widehat{x}[n])$, initialize a dense piecewise-linear spline represented by degenerate cubic Bézier segments. A min-heap algorithm iteratively removes the admissible knot with the smallest removal cost.
For an interior candidate knot $k$ with neighboring knots $a$ and $b$, removing $k$ replaces the affected spline portion with a cubic Bézier segment $\mathbf{B}_k$. Its endpoints and tangent directions are fixed, while the signed handle lengths $\lambda_L$ and $\lambda_R$ are fitted to the reference points by least squares.
Candidate removals are evaluated against fixed reference points uniformly sampled from the original dense curve, with $R=10$ samples per segment. For a candidate spanning retained knots $a$ and $b$, let $\{\mathbf{z}_\ell\}_{\ell\in\mathcal{L}_{ab}}$ denote the ordered reference points covered by that candidate, where $\mathcal{L}_{ab}=\{0,\ldots,R(b-a)-1\}$ indexes the points spanning $a$ to $b$. The respective parametrization, handles and cost of segment $\mathbf{B}_k$ are computed as follows.
Chord-length parameterization: $$u_\ell = \frac{\sum_{s=0}^{\ell}\|\mathbf{z}_s-\mathbf{z}_{s-1}\|_2} {\sum_{s=0}^{R(b-a)-1}\|\mathbf{z}_s-\mathbf{z}_{s-1}\|_2}, \qquad \ell\in\mathcal{L}_{ab},\quad u_\ell\in[0,1].$$
Handle fitting: $$(\lambda_L^*,\lambda_R^*) = \operatorname*{arg\,min}_{\lambda_L,\lambda_R} \sum_{\ell\in\mathcal{L}_{ab}} \left\|\mathbf{B}_k(u_\ell;\lambda_L,\lambda_R)-\mathbf{z}_\ell\right\|_2^2.$$
Removal cost: $$c_k = \max_{\ell\in\mathcal{L}_{ab}} \left\|\mathbf{B}_k(u_\ell;\lambda_L^*,\lambda_R^*)-\mathbf{z}_\ell\right\|_2^2.$$ Thus, the handle coefficients are fitted over all reference points in the affected interval, and $c_k$ is the largest squared deviation over that same set.
For a channel belonging to joint $j$, removal continues while $c_k<\varepsilon_j^2$, where $$\varepsilon_j^2 = \varepsilon_0^2(1 + 0.1h_j),$$ and $h_j$ is the joint's depth in the hierarchy. The base squared tolerance is $\varepsilon_0^2=0.01$ by default.
After scalar-channel decimation, retained frame indices are united within each joint and transform type. Nearby indices are grouped into clusters whose temporal extent does not exceed $\delta$, and each cluster is represented by its median frame; $\delta=2$ frames by default. Complete translation, rotation, or scale vectors are then read from the original motion $\mathcal{M}_i$ at the retained frames and serialized in $\mathcal{K}_i$; fitted Bézier handles are not serialized.
Edit source points, adjust the error threshold, and scrub through each removal step.
This interactive demo requires a desktop browser.
The core generation step uses a Large Language Model to produce Animation JSON from a structured prompt assembled from several components.
Task identity, Object & Animation JSON format specs, coordinate system conventions (Z-up, right-handed), keyframe authoring rules (hold keyframes, <180° rotation deltas), hierarchical transformation notes.
Natural language description of the 3D model together with the Object JSON bind pose.
Example animations pairing short motion descriptions with Animation JSON of sampled keyframes. Their presence and quantity directly improve generation quality.
Multi-view renders of the model with skeleton overlay and axis gizmo, providing spatial context.
The natural language animation description to generate.
Keyframed joint transformations — joint paths map to timestamps with
p, r, s values.
An LLM selects the most relevant few-shot examples from a library based on semantic similarity of motion descriptions, transformation ratios, and rig complexity.
Verbose joint names (e.g., mixamorig_LeftShoulder) are shortened
by an LLM to reduce token usage while preserving anatomical meaning.
User prompts are refined into clean, concise motion descriptions, stripping meta-language like “please generate” or “I want”.
In REFINE mode the LLM receives an existing animation and a modification request, enabling progressive improvement via chat.
Generated keyframes are interpolated back into dense per-frame animation. Translations and scales use cubic Bezier FCurve interpolation; rotations use quaternion SLERP.
Translation and scale are interpolated independently for each scalar channel $x$. Let $k_r$ be the frame index of the $r$-th keyframe and $x_r=x[k_r]$ its value. At an interior keyframe, the tangent slope is the arithmetic mean of the incoming and outgoing secant slopes:
$$m_r = \frac{1}{2}\left( \frac{x_r-x_{r-1}}{k_r-k_{r-1}}+ \frac{x_{r+1}-x_r}{k_{r+1}-k_r} \right).$$
The left and right Bézier handles are aligned with this tangent. Their frame-axis distances from the keyframe are $(k_r-k_{r-1})/3$ and $(k_{r+1}-k_r)/3$, respectively. Treating the intervals separately accommodates non-uniform keyframe spacing while preserving a common tangent slope.
Handles are clamped to suppress overshoot:
Boundary handles are horizontal, and values outside the first and last keyframes are held constant.
Rotations are interpolated using Spherical Linear Interpolation:
$$\operatorname{slerp}(\mathbf{q}_r,\mathbf{q}_{r+1};\tau) = \frac{\sin((1-\tau)\theta)}{\sin\theta}\mathbf{q}_r + \frac{\sin(\tau\theta)}{\sin\theta}\mathbf{q}_{r+1},$$
where $\mathbf{q}_r$ and $\mathbf{q}_{r+1}$ are unit quaternions at consecutive keyframes, $\tau=(n-k_r)/(k_{r+1}-k_r)$, and $\theta=\arccos(\mathbf{q}_r\cdot\mathbf{q}_{r+1})$. If $\mathbf{q}_r\cdot\mathbf{q}_{r+1}<0$, one quaternion is negated to select the shorter arc. Nearly coincident rotations use normalized linear interpolation for numerical stability.
Drag keys to edit, toggle handle visibility, and experiment with clamping modes.
This interactive demo requires a desktop browser.
The final stage writes the generated animation back to a USD file in a non-destructive manner. The original scene data is fully preserved through a four-step process.
The entire source file content is transferred to a new output file, preserving meshes, materials, textures, skeleton bindings, and all other scene data.
A new animation primitive is created under the skeleton, and the skeleton’s animation source is updated to reference it.
For each frame, per-joint translations, rotations (as quaternions), and scales are written as time-sampled attributes on the animation primitive.
Animation JSON, motion description, model description, generation timestamp, and library version are stored as custom data for reproducibility.
We compare against Puppeteer, the closest state-of-the-art baseline that combines text conditioning, rig-based output, and support for diverse object classes. We evaluate overall preference over prompt adherence and motion quality, since generic motion metrics typically assume canonical skeletons, paired ground truth, or pretrained motion embeddings that do not transfer reliably to this setting. The benchmark uses 40 rigged assets from Articulation-XL-2.0, spanning mechanisms, characters, humanoids, plants, household objects, animals, and vehicles or robots. For each asset, we generate 3 prompts and 3 stochastic trials per prompt, yielding 360 GenRA-Puppeteer pairs. Prompts are produced once by a VLM-assisted procedure from the asset render, category label, and joint list, then used unchanged for both methods; all results are zero-shot, with no exemplar animations provided.
We conduct a blind pairwise preference study with 27 raters. Each pair receives 3 independent judgments, for a total of 1,080 ratings. We report the tie-aware preference score, where A and B denote ratings favoring GenRA and Puppeteer, respectively, and Tie denotes ratings that prefer neither method.
Tie-aware preference rate:
$$\frac{A + 0.5 \cdot \mathrm{Tie}}{A + B + \mathrm{Tie}}$$
By asset category:
| Group | #Models | A | B | Tie | GenRA preference rate |
|---|---|---|---|---|---|
| tool/mechanism | 4 | 87 | 3 | 18 | 89% |
| humanoid/anthropomorphic | 4 | 72 | 14 | 22 | 77% |
| vehicle/robot | 6 | 116 | 29 | 17 | 77% |
| animal | 7 | 123 | 24 | 42 | 76% |
| household item/furniture | 3 | 51 | 16 | 14 | 72% |
| character/blob | 13 | 220 | 69 | 62 | 72% |
| plant | 3 | 37 | 21 | 23 | 60% |
| Overall | 40 | 706 | 176 | 198 | 74.5% |
By rig complexity:
| Group | #Models | A | B | Tie | GenRA preference rate |
|---|---|---|---|---|---|
| low complexity (≤5 joints) | 16 | 287 | 59 | 86 | 76% |
| medium complexity (6-24 joints) | 17 | 320 | 63 | 76 | 78% |
| high complexity (≥25 joints) | 7 | 99 | 54 | 36 | 62% |
| Overall | 40 | 706 | 176 | 198 | 74.5% |
Overall, GenRA attains a 74.5% tie-aware preference rate and stays above the 50% indifference point in every reported asset category and rig-complexity group. The largest gains appear on tools/mechanisms and low- to medium-complexity rigs, where direct rig-level generation better preserves asset-specific articulation and prompt semantics, and where prior-driven methods are less helped by existing dataset coverage. Puppeteer can sometimes look more natural on highly organic motions, but its single-view optimization is more prone to motion drift, unreliable joint recovery, and weaker prompt alignment. The gap narrows on plants and high-complexity rigs, where direct generation must coordinate many coupled joints across deeper hierarchies.
Efficiency-wise, GenRA is also substantially faster and cheaper at inference time. Under realistic deployment settings, GenRA reduces median generation time by 83% and median processing cost by 84% relative to Puppeteer. Its median generation time is 120 s per animation at a median cost of $0.09, using the OpenAI API with gpt-5.4-medium. By comparison, Puppeteer requires 690 s and $0.55 per animation in total, combining a KlingAI API video prior generation step (60 s, $0.35) with a subsequent optimization stage on a Vast.ai 80 GB Nvidia A100 SXM4 instance (630 s, $0.20). This gap is consistent with the design of the two pipelines: GenRA generates directly on the target rig through a single LLM-driven stack, whereas Puppeteer combines video-prior synthesis with downstream optimization.
| Approach | Generation stack | Median time / animation | Median cost / animation |
|---|---|---|---|
| GenRA | OpenAI API + gpt-5.4-medium | 120 s | $0.09 |
| Puppeteer | KlingAI API video prior + Vast.ai A100 SXM4 optimization | 690 s | $0.55 |
Evaluation source files, including all compared pairs and collected ratings, can be found here: Hugging Face dataset.
Below we show representative qualitative results beyond the benchmark: side-by-side comparisons against Puppeteer, iterative refinement examples, and few-shot or multi-shot generations with exemplar motions.
Note: visual differences in appearance between the two methods' videos are due to different rendering engines and do not reflect animation quality.
GenRA's chat-based interface supports localized, compositional revisions: modifying a short interval, adjusting a single limb, or specifying approximate angles and speeds — while leaving the rest of the motion unchanged. Edits can be applied repeatedly, enabling progressive improvement or authoring of entirely new parts of the animation through follow-up prompts.
Multi-shot generation on a proprietary character with animator-authored examples. Few-shot conditioning substantially improves motion specificity and stylistic consistency. The system is robust across prompt styles from short action descriptions to detailed motion directions; in several cases, the results reach production-usable quality.
Existing motion-generation approaches broadly fall into several categories: category-specific generators trained on domain-specific datasets, limited to particular character classes such as humans or animals (MotionGPT3, AniMo, Sketch2Anim); topology-conditioned diffusion models that operate on skeleton descriptors without preserving the original rigged asset (AnyTop, Dragon); video-prior approaches that may not preserve the rig and have limited control over the configuration and look of animation (Puppeteer, AniMimic, AnimaX); and LLM-based methods that, while flexible, have been limited to rotation-only outputs or canonical representations (Huang et al.). In production pipelines, animation is authored for concrete rigged assets whose rig hierarchy, skinning weights, and geometry must remain unchanged to maintain compatibility with existing scenes and tools, while supporting iterative and fine-grained nature of refinement and editing of the animations. GenRA uniquely operates directly on user-provided rigged USD assets, preserves all scene data (geometry, skinning, materials), supports all three transformation types (translation, rotation, scale), and enables full iterative editing through a chat-based workflow — making it suitable for direct integration into production DCC pipelines.
| Work | Method | Conditioning | Scope | Ex. | Dur. | G/R | Edit |
|---|---|---|---|---|---|---|---|
| AniMo | Motion transformer | Text + species | Animals | / | |||
| Motion Avatar | LLM planner + learned models | Text | Humans, animals | / | |||
| MotionGPT3 | LLM + diffusion | Text or motion | Humans | Part. | / | Part. | |
| Sketch2Anim | Latent diffusion | Sketches + trajectories + text | Humans | / | |||
| AnyTop | Topology-aware diffusion | Skeleton structure | Diverse skeletons | / | Part. | ||
| Dragon | SiT diffusion | Text + skeleton | Diverse creatures | Part. | / | ||
| RigMo | VAE + diffusion transformer | Mesh sequence | Diverse meshes | Part. | Part. | Part. / | Part. |
| Puppeteer | Auto-rig + video opt. | Static mesh + video | Diverse meshes | Part. | / | ||
| AniMimic | Auto-rig + video opt. + physics | Text + static mesh + video | Diverse meshes | Part. | / | ||
| AnimaX | Video-pose diffusion | Text + rigged mesh + renders | Diverse meshes | / | |||
| Huang et al. | LLM | Text + rig + examples | Diverse skeletons | Part. | / | ||
| GenRA | Multimodal LLM | Text + rig + renders | Diverse assets | / |
supported; Part. partial or task-specific; unsupported or not demonstrated. Conditioning lists the inputs supplied to each method. Ex. denotes support for motion exemplars; Dur. duration control; G/R preservation of input geometry and rig; and Edit method-level iterative refinement.
@inproceedings{yaroshevych2026genra,
title = {GenRA: End-to-end Framework for LLM-Driven Animation of Rigged 3D Models},
author = {Yaroshevych, Andrii and Kryven, Pavlo and Hirna, Mariya and Shamrai, Maksym},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV) Workshops},
year = {2026}
}