这是论文清单仓库，不含可运行代码，无需安装；论文对应的代码需到各自 GitHub 仓库单独获取；部分论文只提供 arXiv 链接，没有开源实现
Awesome-Controllable-Video-Generation
mayuelala活跃维护系统梳理可控视频生成领域研究进展的综述资源库，汇总相关论文与方法，帮研究者快速了解如何精确控制生成视频的内容、运动和风格。
[ArXiv 2025] A survey about controllable video generation: This repo is the official awesome of "Controllable video generation: A survey"
暂无版本776star45fork同步于 2026-11-07 07:33 UTC
README 里没有找到可直接复制的安装命令，请到 README 或官方文档查看安装方式。
硬件要求按 README 说明
README 未说明
可控视频生成领域的论文精选清单，按控制类型分类，方便快速定位相关研究
系统梳理可控视频生成领域研究进展的综述资源库，汇总相关论文与方法，帮研究者快速了解如何精确控制生成视频的内容、运动和风格。
核心能力
- 按结构、ID、图像、时序、音频等控制类型分目录索引
- 点击目录即可跳转到对应研究方向的论文列表
- 用 ctrl+F 搜索作者名，自动展开作者下拉列表
- 用 [M] 标签标记多控制（multi-control）论文
- 每条论文附 arXiv、代码库和项目主页链接
容易踩的坑
内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
暂无 release 数据（可能没有发版，或尚未同步）。
用户点评
登录用户的真实使用记录，可按版本和硬件筛选共 0 条
还没有人点评。第一个说出真实体验的人最有价值。
🤗 Introduction
🚀🚀🚀A curated list of papers on controllable video generation. Please join us for more comprehensive summary. If you have any additions to the list, please raise them in the issue section. 欢迎补充👏
💖 Citation
If you find this repo is useful for your research, welcome to 🌟 this repo and cite our work using the following BibTeX:
@article{ma2025controllable,
title={Controllable video generation: A survey},
author={Ma, Yue and Feng, Kunyu and Hu, Zhongyuan and Wang, Xinyu and Wang, Yucheng and Zheng, Mingzhe and He, Xuanhua and Zhu, Chenyang and Liu, Hongyu and He, Yingqing and others},
journal={arXiv preprint arXiv:2507.16869},
year={2025}
}
📋 Contents
- 🌐 Structure Control
- 🧬 ID Control
- 🖼️ Image Control
- ⏳ Temporal Control
- 🔊 Audio Control
- 🧩 Other Controls
- 🌀 Universal Control
💘 Tips
- ✅ Paper searching via catalog: directly clicking the content of the catalog to select the area of your research and browse related papers.
- ✅ Paper searching via author name: Feel free to search papers of a specific author via
ctrl + Fand then type the author name. The dropdown list of authors will automatically expand when searching. - ✅ Multicontrol indicator: The tag
indicates "multi-control" following the title of the paper.
🌐 Structure Control
🏙️ Layout-Guided
-
DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenes
🕺 Pose-Guided
-
Hand2World: Autoregressive Egocentric Interaction Generation via Free-Space Hand Gestures (10 Feb 2026) Yuxi Wang, Wenqi Ouyang, Tianyi Wei, et al. Yuxi Wang, Wenqi Ouyang, Tianyi Wei, Yi Dong, Zhiqi Shen, Xingang Pan
-
MTVCrafter: 4D Motion Tokenization for Open-World Human Image Animation
-
HyperMotion: DiT-Based Pose-Guided Human Image Animation of Complex Motions
-
FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios
-
DualReal: Joint Training for Lossless Identity-Motion Fusion in Video Customization
-
DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation
-
TASTE-Rob: Advancing Video Generation of Task-Oriented Hand-Object Interaction for Generalizable Robotic Manipulation
-
Point-to-Point Video Generation (14 Mar 2025) Tsun-Hsuan Wang, Yen-Chi Cheng, et al. Tsun-Hsuan Wang, Yen-Chi Cheng, Chieh Hubert Lin, Hwann-Tzong Chen, Min Sun
-
Pose Guided Human Video Generation (14 Mar 2025) Ceyuan Yang, Zhe Wang, et al. Ceyuan Yang, Zhe Wang, Xinge Zhu, Chen Huang, Jianping Shi, Dahua Lin
-
SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers
-
AnyCharV: Bootstrap Controllable Character Video Generation with Fine-to-Coarse Guidance
-
Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance
-
DirectorLLM for Human-Centric Video Generation (19 Dec 2024) Kunpeng Song, Tingbo Hou, et al. Kunpeng Song, Tingbo Hou, Zecheng He, Haoyu Ma, Jialiang Wang, Animesh Sinha, Sam Tsai, Yaqiao Luo, Xiaoliang Dai, Li Chen, Xide Xia, Peizhao Zhang, Peter Vajda, Ahmed Elgammal, Felix Juefei-Xu
-
Generative Inbetweening through Frame-wise Conditions-Driven Video Generation
-
DisPose: Disentangling Pose Guidance for Controllable Human Image Animation
-
StableAnimator: High-Quality Identity-Preserving Human Image Animation
-
AnimateAnywhere: Context-Controllable Human Video Generation with ID-Consistent One-shot Learning
-
ControlNeXt: Powerful and Efficient Control for Image and Video Generation
-
TCAN: Animating Human Images with Temporally Consistent Pose Guidance using Diffusion Models
-
MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance
-
Follow-Your-Pose v2: Multiple-Condition Guided Character Image Animation for Stable Pose Control
-
UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation
-
VividPose: Advancing Stable Video Diffusion for Realistic Human Image Animation
-
Disentangling Foreground and Background Motion for Enhanced Realism in Human Video Generation
-
PoseCrafter: One-Shot Personalized Video Synthesis Following Flexible Pose Control
-
Zero-shot High-fidelity and Pose-controllable Character Animation
-
Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance
-
Do You Guys Want to Dance: Zero-Shot Compositional Human Dance Generation with Multiple Persons
-
DreaMoving: A Human Video Generation Framework based on Diffusion Models
-
Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models
-
Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation
-
MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model
-
MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion
-
Dancing Avatar: Pose and Text-Guided Human Motion Videos Synthesis with Image Diffusion Model (15 Aug 2023) Bosheng Qin, Wentao Ye, et al. Bosheng Qin, Wentao Ye, Qifan Yu, Siliang Tang, Yueting Zhuang
-
DISCO: Disentangled Control for Realistic Human Dance Generation
-
DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion
-
Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos (3 Apr 2023) [AAAI 2024] Yue Ma, Yingqing He, et al. Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Ying Shan, Xiu Li, Qifeng Chen
-
vid2vid:Video-to-Video Synthesis
📏 Depth-Guided
-
Frame Guidance: Training-Free Guidance for Frame-Level Control in Video Diffusion Models
-
OmniVDiff: Omni Controllable Video Diffusion for Generation and Understanding
-
DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Poses
-
ControlNeXt: Powerful and Efficient Control for Image and Video Generation
-
Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance
-
MoonShot: Towards Controllable Video Generation and Editing with Multimodal Conditions
-
SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models (28 Nov 2023)[ECCV 2024] Yuwei Guo, Ceyuan Yang, Anyi Rao, et al. Maneesh Agrawala, Dahua Lin, Bo Dai
-
GD-VDM: Generated Depth for better Diffusion-based Video Generation (19 Jun 2023)Ariel Lapid, Idan Achituve, Lior Bracha, et al. Ethan Fetaya
-
Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance
-
Control-A-Video: Controllable Text-to-Video Diffusion Models with Motion Prior and Reward Feedback Learning
-
ControlVideo: Training-free Controllable Text-to-Video Generation (22 May 2023)[ICLR 2024] Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, et al. Xiaopeng Zhang, Wangmeng Zuo, Qi Tian
🗺️ Landmark-Guided
-
HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animation
-
TASTE-Rob: Advancing Video Generation of Task-Oriented Hand-Object Interaction for Generalizable Robotic Manipulation
-
Identity-Preserving Text-to-Video Generation by Frequency Decomposition
-
EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation
-
Takin-ADA: Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization
-
EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions
-
LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control
-
Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation
-
Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos (3 Apr 2023)
✏️ Sketch-Guided
-
Frame Guidance: Training-Free Guidance for Frame-Level Control in Video Diffusion Models
-
OmniVDiff: Omni Controllable Video Diffusion for Generation and Understanding (15 Apr 2025)Dianbing Xi, Jiepeng Wang, Yuanzhi Liang, et al. Xi Qiu, Yuchi Huo, Rui Wang, Chi Zhang, Xuelong Li
-
SketchVideo: Sketch-based Video Generation and Editing
-
LayerAnimate: Layer-level Control for Animation
-
CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models
-
CameraCtrl: Enabling Camera Control for Text-to-Video Generation
-
VidSketch: Hand-drawn Sketch-Driven Video Generation with Diffusion Control
-
AniDoc: Animation Creation Made Easier
-
Latent-Reframe: Enabling Camera Control for Video Diffusion Model without Training
-
CamI2V: Camera-Controlled Image-to-Video Diffusion Model
-
Motion Prompting: Controlling Video Generation with Motion Trajectories(frame+track+text)
-
MOTIONFLOW: Learning Implicit Motion Flow for Complex Camera Trajectory Control in Video Generation
-
I2VControl: Disentangled and Unified Video Motion Synthesis Control
-
Trajectory Attention: Enhancing Video Generation with Fine-Grained Motion Control
-
Open-Sora Plan: Open-Source Large Video Generation Model
-
ToonCrafter: Generative Cartoon Interpolation
-
MagicStick: Controllable Video Editing via Control Handle Transformations
-
Video Diffusion Models are Training-free Motion Interpreter and Controller
-
DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion
-
MotionClone: Training-Free Motion Cloning for Controllable Video Generation
-
Boosting Camera Motion Control for Video Diffusion Transformers
-
Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attention
-
LVCD: Reference-based Lineart Video Colorization with Diffusion Models
-
EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolation
-
ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis
-
VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control
-
MotionCtrl: A Unified and Flexible Motion Controller for Video Generation
FFmpeg
FFmpeg·C
一套完整的音视频处理工具集，能解码、编码、转码、推流和封装几乎所有格式，是搭建播放器、转码服务和流媒体管线的底层基础设施。
音视频转码+4视频转码与编辑
66885+22415k2026-11-10 15:13 UTC
HandBrake
HandBrake·C
开源跨平台视频转码工具，能把 DVD/蓝光和各种格式的视频批量转成 H.264/H.265 等现代编码，自带预设和滤镜，适合整理片源和压缩体积。
视频转码+4视频转码与编辑
24958+411.7k1.11.22026-11-10 10:40 UTC
filmcraft
storytold·Rust
用纯 Rust 从零实现的开源非线性视频编辑器，支持多轨道剪辑与视频处理，适合想要轻量、高性能剪辑工具的用户。
视频编辑+4视频转码与编辑
17986+2227.4kv0.19.02026-11-10 13:21 UTC
Waifu2x-Extension-GUI
AaronFeng753·C++
集成 Waifu2x、Real-ESRGAN、Real-CUGAN 等多种超分模型与 RIFE 等补帧算法的图形界面工具，可批量放大视频、图片和 GIF 并提升帧率。
超分辨率+4视频转码与编辑
17212+171.1kv3.143.012026-11-10 13:39 UTC