Awesome-Video-Diffusion-Models

ChenHsing活跃维护

汇总视频扩散模型相关论文与资源的综述清单,覆盖文本生成视频、视频编辑等方向,帮研究者快速摸清领域脉络和代表工作。

[CSUR] A Survey on Video Diffusion Models

暂无版本2,329star123fork同步于 2026-11-07 07:30 UTC

怎么上手

README 里没有找到可直接复制的安装命令

README 里没有找到可直接复制的安装命令,请到 README 或官方文档查看安装方式。

硬件要求按 README 说明

README 未说明

支持 Apple Silicon
下载:源码 zip·GitHub Releases(安装包)本站不镜像安装包

项目说明

由 README 自动翻译提炼

视频扩散模型综述论文的配套资源库,汇总论文、数据集与开源工具箱

汇总视频扩散模型相关论文与资源的综述清单,覆盖文本生成视频、视频编辑等方向,帮研究者快速摸清领域脉络和代表工作。

核心能力

  • 按生成、编辑、理解三大任务分类整理文献
  • 收录 T2V 生成与编辑的开源工具箱和基础模型
  • 按 caption 级和类别级整理视频数据集
  • 提供 arXiv、GitHub、项目主页多链接索引
  • 附中文翻译与 BibTeX 引用

容易踩的坑

本仓库是论文列表而非可运行代码,没有安装步骤;部分条目只给官网链接,无开源实现;引用请以 CSUR 期刊版本为准

内容由模型根据 README 生成,可能不完整或不准确,以官方文档为准。看原文 README

最近发版

0 个版本

暂无 release 数据(可能没有发版,或尚未同步)。

用户点评

登录用户的真实使用记录,可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译

    A Survey on Video Diffusion Models Awesome arXiv

    • [News] The updated version is available on arXiv.
    • [News] Our survey is accepted by ACM Computing Surveys (CSUR).
    • [News] The Chinese translation is available on Zhihu. Special thanks to Dai-Wenxun for this.

    Contact

    If you have any suggestions or find our work helpful, feel free to contact us

    Homepage: Zhen Xing

    Email: [email protected]

    If you find our survey is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

    @article{xing2023survey,
      title={A survey on video diffusion models},
      author={Xing, Zhen and Feng, Qijun and Chen, Haoran and Dai, Qi and Hu, Han and Xu, Hang and Wu, Zuxuan and Jiang, Yu-Gang},
      journal={ACM Computing Surveys},
      year={2023},
      publisher={ACM New York, NY}
    }
    

    Open-source Toolboxes and Foundation Models

    MethodsTaskGithub
    HeliosT2V GenerationStar
    Movie GenT2V Generation-
    CogVideoXT2V GenerationStar
    Open-Sora-PlanT2V GenerationStar
    Open-SoraT2V GenerationStar
    Morph StudioT2V Generation-
    GenieT2V Generation-
    SoraT2V Generation & Editing-
    VideoPoetT2V Generation & Editing-
    Stable Video DiffusionT2V GenerationStar
    NeverEndsT2V Generation-
    PikaT2V Generation-
    EMU-VideoT2V Generation-
    GEN-2T2V Generation & Editing-
    ModelScopeT2V GenerationStar
    ZeroScopeT2V Generation-
    T2V Synthesis ColabT2V GenetationStar
    VideoCraftT2V Genetation & EditingStar
    Diffusers (T2V synthesis)T2V Genetation-
    AnimateDiffPersonalized T2V GenetationStar
    Text2Video-ZeroT2V GenetationStar
    HotShot-XLT2V GenetationStar
    GenmoT2V Genetation-
    FlikiT2V Generation-
    Seedream AI StudioImage Generation + I2V Animation-
    Omni-RewriterPrompt Expansion (Video/Image PE)Star

    Table of Contents

    Video Generation

    Data

    Caption-level

    TitlearXivGithubWebSitePub. & Date
    OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video GenerationarXivStarWebsiteMay, 2025
    Identity-Preserving Text-to-Video Generation by Frequency DecompositionarXivStarWebsiteCVPR, 2025
    ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video GenerationarXivStarWebsiteNeurIPS, 2024
    Panda-70M: Captioning 70M Videos with Multiple Cross-Modality TeachersarXivStarWebsiteCVPR, 2024
    CelebV-Text: A Large-Scale Facial Text-Video DatasetarXivStar-CVPR, 2023
    InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and GenerationarXivStar-May, 2023
    VideoFactory: Swap Attention in Spatiotemporal Diffusions for Text-to-Video GenerationarXiv--May, 2023
    Advancing High-Resolution Video-Language Representation with Large-Scale Video TranscriptionsarXiv--Nov, 2021
    Frozen in Time: A Joint Video and Image Encoder for End-to-End RetrievalarXiv--ICCV, 2021
    MSR-VTT: A Large Video Description Dataset for Bridging Video and LanguagearXiv--CVPR, 2016

    | MiniMax H3 1K Prompt Dataset | Dataset | Star | Website | 2026 |

    Category-level

    TitlearXivGithubWebSitePub. & Date
    UCF101: A Dataset of 101 Human Actions Classes From Videos in The WildarXiv--Dec., 2012
    First Order Motion Model for Image AnimationarXiv--May, 2023
    Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial NetworksarXiv--CVPR,2018

    Metric and BenchMark

    TitlearXivGithubWebSitePub. & Date
    OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video GenerationarXivStarWebsiteMay, 2025
    Fréchet Video Motion Distance: A Metric for Evaluating Motion Consistency in VideosarXivStar-Jul., 2024
    ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video GenerationarXivStarWebsiteNeurIPS, 2024
    STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative ModelsarXivStar-ICLR, 2024
    Subjective-Aligned Dateset and Metric for Text-to-Video Quality AssessmentarXiv--Mar, 2024
    Towards A Better Metric for Text-to-Video GenerationarXiv-WebsiteJan, 2024
    AIGCBench: Comprehensive Evaluation of Image-to-Video Content Generated by AIarXiv--Jan, 2024
    VBench: Comprehensive Benchmark Suite for Video Generative ModelsarXivStarWebsiteNov, 2023
    VideoScore2: Think before You Score in Generative Video EvaluationarXivStarWebsiteSep, 2025
    FETV: A Benchmark for Fine-Grained Evaluation of Open-Domain Text-to-Video GenerationarXiv--NeurIPS, 2023
    CVPR 2023 Text Guided Video Editing CompetitionarXiv--Oct., 2023
    EvalCrafter: Benchmarking and Evaluating Large Video Generation ModelsarXivStarWebsiteOct., 2023
    Measuring the Quality of Text-to-Video Model Outputs: Metrics and DatasetarXiv--Sep., 2023

    Text-to-Video Generation

    Training-based

    TitlearXivGithubWebSitePub. & Date
    Helios: Real Real-Time Long Video Generation ModelarXivStarWebsiteArxiv, 2026
    Identity-Preserving Text-to-Video Generation by Frequency DecompositionarXivStarWebsiteCVPR, 2025
    Enhancing Motion in Text-to-Video Generation with Decomposed Encoding and ConditioningarXivStarWebsiteNeurIPS 2024
    Movie GenarXiv-WebsiteOct, 2024
    CogVideoX: Text-to-Video Diffusion Models with An Expert TransformerarXivStar-Oct, 2024
    Grid Diffusion Models for Text-to-Video GenerationarXivStarWebsiteCVPR, 2024
    MagicTime: Time-lapse Video Generation Models as Metamorphic SimulatorsarXivStarWebsiteApr., 2024
    Mora: Enabling Generalist Video Generation via A Multi-Agent FrameworkarXiv--Mar., 2024
    VSTAR: Generative Temporal Nursing for Longer Dynamic Video SynthesisarXiv--Mar., 2024
    Genie: Generative Interactive EnvironmentsarXiv-WebsiteFeb., 2024
    Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video SynthesisarXiv-WebsiteFeb., 2024
    Lumiere: A Space-Time Diffusion Model for Video GenerationarXiv-WebsiteJan, 2024
    UNIVG: TOWARDS UNIFIED-MODAL VIDEO GENERATIONarXiv-WebsiteJan, 2024
    VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion ModelsarXivStarWebsiteJan, 2024
    360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion ModelarXiv-WebsiteJan, 2024
    MagicVideo-V2: Multi-Stage High-Aesthetic Video GenerationarXiv-WebsiteJan, 2024
    VideoDrafter: Content-Consistent Multi-Scene Video Generation with LLMarXiv-WebsiteJan, 2024
    A Recipe for Scaling up Text-to-Video Generation with Text-free VideosarXivStarWebsiteDec, 2023
    InstructVideo: Instructing Video Diffusion Models with Human FeedbackarXivStarWebsiteDec, 2023
    VideoLCM: Video Latent Consistency ModelarXiv--Dec, 2023
    Photorealistic Video Generation with Diffusion ModelsarXiv-WebsiteDec, 2023
    Hierarchical Spatio-temporal Decoupling for Text-to-Video GenerationarXivStarWebsiteDec, 2023
    Delving Deep into Diffusion Transformers for Image and Video GenerationarXiv-WebsiteDec, 2023
    StyleCrafter: Enhancing Stylized Text-to-Video Generation with Style AdapterarXivStarWebsiteNov, 2023
    MicroCinema: A Divide-and-Conquer Approach for Text-to-Video GenerationarXiv-WebsiteNov, 2023
    ART•V: Auto-Regressive Text-to-Video Generation with Diffusion ModelsarXivStarWebsiteNov, 2023
    Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large DatasetsarXivStarWebsiteNov, 2023
    FusionFrames: Efficient Architectural Aspects for Text-to-Video Generation PipelinearXivStarWebsiteNov, 2023
    MoVideo: Motion-Aware Video Generation with Diffusion ModelsarXiv-WebsiteNov, 2023
    Make Pixels Dance: High-Dynamic Video GenerationarXiv-WebsiteNov, 2023
    Emu Video: Factorizing Text-to-Video Generation by Explicit Image ConditioningarXiv-WebsiteNov, 2023
    Optimal Noise pursuit for Augmenting Text-to-Video GenerationarXiv--Nov, 2023
    VideoDreamer: Customized Multi-Subject Text-to-Video Generation with Disen-Mix FinetuningarXiv-WebsiteNov, 2023
    VideoCrafter1: Open Diffusion Models for High-Quality Video GenerationarXivStarWebsiteOct, 2023
    SEINE: Short-to-Long Video Diffusion Model for Generative Transition and PredictionarXivStarWebsiteOct, 2023
    DynamiCrafter: Animating Open-domain Images with Video Diffusion PriorsarXivStarWebsiteOct., 2023
    LAMP: Learn A Motion Pattern for Few-Shot-Based Video GenerationarXivStarWebsiteOct., 2023
    DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion modelarXivStarWebsiteOct, 2023
    MotionDirector: Motion Customization of Text-to-Video Diffusion ModelsarXivStarWebsiteOct, 2023
    VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided PlanningarXivStarWebsiteSep., 2023
    Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video GenerationarXivStarWebsiteSep., 2023
    LaVie: High-Quality Video Generation with Cascaded Latent Diffusion ModelsarXivStarWebsiteSep., 2023
    Reuse and Diffuse: Iterative Denoising for Text-to-Video GenerationarXivStarWebsiteSep., 2023
    VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video GenerationarXiv-WebsiteSep., 2023
    MobileVidFactory: Automatic Diffusion-Based Social Media Video Generation for Mobile Devices from TextarXiv--Jul., 2023
    Text2Performer: Text-Driven Human Video GenerationarXivStarWebsiteApr., 2023
    AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific TuningarXivStarWebsiteJul., 2023
    Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with Large Language ModelsarXiv-WebsiteAug., 2023
    SimDA: Simple Diffusion Adapter for Efficient Video GenerationarXivStarWebsiteCVPR, 2024
    Dual-Stream Diffusion Net for Text-to-Video GenerationarXiv--Aug., 2023
    ModelScope Text-to-Video Technical ReportarXivStarWebsiteAug., 2023
    InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and GenerationarXivStar-Jul., 2023
    VideoFactory: Swap Attention in Spatiotemporal Diffusions for Text-to-Video GenerationarXiv--May, 2023
    Preserve Your Own Correlation: A Noise Prior for Video Diffusion ModelsarXiv-WebsiteMay, 2023
    Align your Latents: High-Resolution Video Synthesis with Latent Diffusion ModelsarXiv-Website-
    Latent-Shift: Latent Diffusion with Temporal ShiftarXiv-Website-
    Probabilistic Adaptation of Text-to-Video ModelsarXiv-WebsiteJun., 2023
    NUWA-XL: Diffusion over Diffusion for eXtremely Long Video GenerationarXiv-WebsiteMar., 2023
    ED-T2V: An Efficient Training Framework for Diffusion-based Text-to-Video Generation---IJCNN, 2023
    MagicVideo: Efficient Video Generation With Latent Diffusion ModelsarXiv-Website-
    Phenaki: Variable Length Video Generation From Open Domain Textual DescriptionarXiv-Website-
    Imagen Video: High Definition Video Generation With Diffusion ModelsarXiv-Website-
    VideoFusion: Decomposed Diffusion Models for High-Quality Video GenerationarXivStarWebsite-
    MAGVIT: Masked Generative Video TransformerarXiv-WebsiteDec., 2022
    Make-A-Video: Text-to-Video Generation without Text-Video DataarXiv-Website-
    Latent Video Diffusion Models for High-Fidelity Video Generation With Arbitrary LengthsarXivStarWebsiteNov., 2022
    CogVideo: Large-scale Pretraining for Text-to-Video Generation via TransformersarXivStar-May, 2022
    Video Diffusion ModelsarXiv-Website-

    Training-free

    TitlearXivGithubWebSitePub. & Date
    CachedSearch: Training-Free Cached Exploration for Test-Time Search in Video DiffusionarXivStarWebsiteJul, 2026
    A²RD: Agentic Autoregressive Diffusion for Long Video ConsistencyarXiv-WebsiteMay, 2026
    VISTA: A Test-Time Self-Improving Video Generation AgentarXiv-WebsiteCVPR, 2026
    InfLVG: Reinforce Inference-Time Consistent Long Video Generation with GRPOarXivStar-May, 2025
    VideoElevator: Elevating Video Generation Quality with Versatile Text-to-Image Diffusion ModelsarXivStarWebsiteMar, 2024
    TRAILBLAZER: TRAJECTORY CONTROL FOR DIFFUSION-BASED VIDEO GENERATIONarXivStarWebsiteJan, 2024
    FreeInit: Bridging Initialization Gap in Video Diffusion ModelsarXivStarWebsiteDec, 2023
    MTVG : Multi-text Video Generation with Text-to-Video ModelsarXiv-WebsiteDec, 2023
    F3-Pruning: A Training-Free and Generalized Pruning Strategy towards Faster and Finer Text-to-Video SynthesisarXiv--Nov, 2023
    AdaDiff: Adaptive Step Selection for Fast DiffusionarXiv--Nov, 2023
    FlowZero: Zero-Shot Text-to-Video Synthesis with LLM-Driven Dynamic Scene SyntaxarXivStarWebsiteNov, 2023
    🏀GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT PlanningarXivStarWebsiteNov, 2023
    FreeNoise: Tuning-Free Longer Video Diffusion Via Noise ReschedulingarXivStarWebsiteOct, 2023
    ConditionVideo: Training-Free Condition-Guided Text-to-Video GenerationarXivStarWebsiteOct, 2023
    LLM-grounded Video Diffusion ModelsarXivStarWebsiteOct, 2023
    Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM AnimatorarXivStar-NeurIPS, 2023
    DiffSynth: Latent In-Iteration Deflickering for Realistic Video SynthesisarXivStarWebsiteAug, 2023
    Large Language Models are Frame-level Directors for Zero-shot Text-to-Video GenerationarXivStar-May, 2023
    Text2video-Zero: Text-to-Image Diffusion Models Are Zero-Shot Video GeneratorsarXivStarWebsiteMar., 2023
    PEEKABOO: Interactive Video Generation via Masked-Diffusion 🫣arXivStarWebsiteCVPR, 2024

    Video Generation with other conditions

    Pose-guided Video Generation

    TitlearXivGithubWebSitePub. & Date
    Hand2World: Autoregressive Egocentric Interaction Generation via Free-Space Hand GesturesarXiv-WebsiteFeb., 2026
    EgoControl: Controllable Egocentric Video Generation via 3D Full-Body PosesarXivStarWebsiteCVPR 2026
    🔥🔥StableAnimator: High-Quality Identity-Preserving Human Image Animation🔥🔥arXivStarWebsiteNov., 2024
    MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion ModelarXivStarWebsiteECCV 2024
    MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose GuidancearXivStarWebsiteJul., 2024
    Champ: Controllable and Consistent Human Image Animation with 3D Parametric GuidancearXivStarWebsiteMar., 2024
    Action Reimagined: Text-to-Pose Video Editing for Dynamic Human ActionsarXiv--Mar., 2024
    Do You Guys Want to Dance: Zero-Shot Compositional Human Dance Generation with Multiple PersonsarXiv--Jan., 2024
    DreaMoving: A Human Dance Video Generation Framework based on Diffusion ModelsarXiv-WebsiteDec., 2023
    MagicAnimate: Temporally Consistent Human Image Animation using Diffusion ModelarXivStarWebsiteNov., 2023
    Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character AnimationarXivStarWebsiteNov., 2023
    MagicDance: Realistic Human Dance Video Generation with Motions & Facial Expressions TransferarXivStarWebsiteNov., 2023
    DisCo: Disentangled Control for Referring Human Dance Generation in Real WorldarXivStarWebsiteJul., 2023
    Dancing Avatar: Pose and Text-Guided Human Motion Videos Synthesis with Image Diffusion ModelarXiv--Aug., 2023
    DreamPose: Fashion Image-to-Video Synthesis via Stable DiffusionarXivStarWebsiteApr., 2023
    Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free VideosarXivStarWebsiteApr., 2023

    Motion-guided Video Generation

    TitlearXivGithubWebSitePub. & Date
    MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory GuidancearXivStarWebsiteMar. 2025
    MOTIONCLONE: TRAINING-FREE MOTION CLONING FOR CONTROLLABLE VIDEO GENERATIONarXivStarWebsiteJun., 2024
    Tora: Trajectory-oriented Diffusion Transformer for Video GenerationarXivStarWebsiteCVPR 2025
    [MOFA-Video: Cont