登录

diffusers

huggingfaceApache-2.0活跃维护

基于 PyTorch 的扩散模型工具箱，统一加载和训练 Stable Diffusion、FLUX 等模型，做图像、视频、音频生成。

🤗 Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch.

v0.42.1发布于 2026-10-27 07:41 UTC34,891star +5 / 7 天7,473fork同步于 2026-11-10 05:20 UTC
扩散模型图像生成视频生成PyTorchStable Diffusion
对比 GitHub 官网deep-learningdiffusionfluximage-generationimage2image

怎么上手

由 README 提炼，执行前请确认
原文 
pip install --upgrade diffusers[torch]
硬件要求按 README 说明

README 未说明具体显存要求；示例代码用 pipeline.to("cuda") 与 float16 在 GPU 上运行，并单独提供 Apple Silicon（M1/M2）的 MPS 使用指南。

支持 Apple Silicon
下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

HuggingFace 出品的扩散模型工具箱，几行代码跑推理，也能自己搭训练流程

基于 PyTorch 的扩散模型工具箱，统一加载和训练 Stable Diffusion、FLUX 等模型，做图像、视频、音频生成。

核心能力

  • 用 DiffusionPipeline.from_pretrained 几行代码加载预训练管线做推理
  • 提供可互换的噪声调度器，调节扩散速度与输出质量
  • 模型与调度器可拆开当积木，自建端到端扩散系统
  • 覆盖文生图、图生图、ControlNet、超分等常见任务管线
  • 支持图像、音频乃至分子 3D 结构生成

容易踩的坑

需自行按 PyTorch 官方文档先装好匹配的 PyTorch 版本；示例模型需从 Hugging Face Hub 下载，首次运行要联网拉取权重；Apple Silicon 需按 MPS 专门指南配置，不能直接照搬 CUDA 示例。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
这个条目还没有人工点评。如果它对你很有用，欢迎到 提交仓库 页写下你的使用经验。

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    最近发版

    5 个版本v0.42.1
    全部 release
    1. v0.42.1v0.42.1 — fix component resolution in `from_pretrained` to prevent `trust_remote_code` bypass最新正式版2026-10-27 · 2026-10-27 07:41 UTC

      This is a security release. See GHSA-4prv-5hj7-c2xr for details.

      This patch release fixes how a component's declared library name is resolved when loading with from_pretrained.

      Pipelines and models declare each component as a ["<library>", "<class>"] pair in model_index.json / config.json. This release changes how that library name is resolved across the DiffusionPipeline, AutoModel, and ModularPipeline loading paths, and adds a clearer error when the declared class is not found in the resolved module.

      All users on 0.42.0 are encouraged to upgrade:

      pip install diffusers==0.42.1
    2. v0.42.0Diffusers 0.42.0: FLUX.3, Fibo 2.0, Kandinsky 6, and more2026-10-23 · 2026-10-23 15:38 UTC
      查看更新说明

      [!TIP] This release brings FLUX.3 Image Klein, Bria Fibo 2.0, Kandinsky 6, SANA-WM, and RFD4-Proteína to Diffusers, alongside new adapter support, hardware integrations, and modular pipelines. 🚨 Python 3.11 or newer is now required.

      FLUX.3 Image Klein

      FLUX.3 Image Klein from Black Forest Labs supports text-to-image generation and editing with one or more reference images. Both the base and step-distilled checkpoints are available through ModularPipeline, with an integrated prompt upsampler that turns short requests into detailed captions and supports layout instructions with bounding boxes. The integration also includes single-file transformer loading and BFL FP8r and NVFP4 checkpoints.

      Load the pipeline with ModularPipeline.from_pretrained(...), then load its weights with load_components(...). See the FLUX.3 documentation for examples and guidance on prompt upsampling, editing, and memory use.

      Bria Fibo 2.0

      Bria Fibo 2.0 supports image generation from structured JSON captions and editing with up to five reference images through BriaFibo2Pipeline. The integration includes the dev checkpoint and the edit-turbo checkpoint, which runs in four steps without classifier-free guidance. See the Fibo 2.0 documentation for the supported prompt formats.

      LoRA loading and DreamBooth LoRA training examples are included for text-to-image and image editing. Thanks to @muhammadali-afridi for the model integration (#14991).

      Kandinsky 6

      Kandinsky 6 generates video with synchronized audio from text or a reference image through Kandinsky6TI2VAPipeline. A separate Kandinsky6SRPipeline provides video super-resolution. The integration supports Pro and Lite generation checkpoints, including distilled variants, and introduces PiflowScheduler for the distilled models. See the Kandinsky 6 documentation for checkpoint-specific settings and super-resolution requirements. Thanks to @lefff for the model integration (#14949).

      SANA-WM

      SANA-WM generates video from a starting image, a text prompt, and a camera trajectory. Control the camera with explicit poses or a WASD/IJKL action string, then optionally refine the output with the separate LTX-2 refiner. See the SANA-WM documentation for the two-stage workflow.

      Thanks to @lawrence-cj for the integration (#13881).

      RFD4-Proteína

      RFD4-Proteína from the Baker Lab and NVIDIA brings biomolecular design to Diffusers. It jointly generates protein sequences and all-atom structures while conditioning on molecular context such as binding targets, motifs, or ligands. The integration includes the pipeline, transformer, autoencoder, schedulers, and LoRA loading. Preparing model inputs and writing CIF files uses the separate rfd4-proteina-prep package; see the RFD4-Proteína documentation.

      Thanks to @kierandidi for the integration (#15088).

      Other highlights

      • Qwen-Image 2.1: new modular pipelines and tensor parallelism support. (#14976, #14865)
      • More hardware support: TorchTPU integration covers eager execution, torch.compile, and tensor parallelism. Tensor parallelism also gains Intel XPU support. (#14039, #15013)
      • LoKr adapters: load LyCORIS Kronecker-product adapters for Z-Image and FLUX.2, including Klein. ([#14163](https://github.com/
    3. v0.41.0Diffusers 0.41.0: QwenImage 2.1 pipeline and more2026-10-06 · 2026-10-06 06:43 UTC
      查看更新说明

      [!TIP] This release brings Qwen-Image 2.1 to Diffusers, with text-to-image generation, image editing, native transparency, and LoRA training.

      Starting with this release, we’re adopting the Transformers release philosophy: coordinating minor releases around new model integrations, while including the other changes merged since the previous release. Patch releases remain focused on fixes. See our release policy.

      Qwen-Image 2.1

      Qwen-Image 2.1 unifies image generation and editing in one model. Its visual generation component has 7B parameters, and it supports multiple reference images and native RGBA output for transparent images. Check out the docs for more details.

      Thanks to @naykun for the model integration (#14804).

      Other highlights

      • Tensor-parallel checkpoint loading: each rank reads its own slice of sharded weights directly, reducing the memory needed during loading. (#14544)
      • LTX-2.5 DFR: new pipelines support generation with keyframe slots and composable spatial and temporal refinement. (#14567)
      • Cosmos 3: SeaCache support and mixed W8A8/W8A16 denoising for ModelOpt FP8 checkpoints. (#14663, #14664)
      • Modular Diffusers: Wan 2.2 VACE gains modular blocks, and loading from existing local paths avoids unnecessary weight downloads. (#14432, #14797)
      • More single-file loaders: support for Krea 2 and MiniMax-H3. (#14914, #14839)
      • Tests: We removed the monolithic test_pipelines_common.py test suite to establish more modularity and transparent implementation of tests. Refer to https://github.com/huggingface/diffusers/pull/14669/ for more details.

      Deprecations and compatibility

      🚨 ONNX support is deprecated; use Optimum for ONNX Runtime workflows. (#14658)

      Previously deprecated APIs have been removed, including the LuminaText2ImgPipeline and Lumina2Text2ImgPipeline aliases. Use LuminaPipeline and Lumina2Pipeline instead. Older integrations should review the removed code paths before upgrading.

      Thanks to everyone who contributed to this release!

      All commits

      • [docs] Clean up links by @stevhliu in #14535
      • docs: fix 14 typos in docs, comments and docstrings by @iridescentWen in #14541
      • [docs] LTX-2.5 Modular Pipeline Docs by @dg845 in #14450
      • [agent docs] add lora test instructions. by @sayakpaul in #14549
      • refactor bria pipeline tests to the new mixin structure by @akshan-main in #14547
      • feat: improve issue reminder workflow and introduce conditional autoclose by @sayakpaul in #14477
      • [tests] AOT compilation failure fixes by @sayakpaul in #14553
      • [tests] skip hunyuan framepack group offloading tests by @sayakpaul in #14551
      • [tests] remove autoencodertestermixin and rename new one. by @sayakpaul in #14555
      • tests: fix CUDA OOM in VidTok slicing/tiling tests by @sayakpaul in #14554
      • refactor bria fibo edit pipeline tests to the new mixin structure by @akshan-main in #14577
      • refactor bria fibo pipeline tests to the new mixin structure by @akshan-main in #14570
      • refactor deepfloyd if pipeline tests to the new mixin structure by @akshan-main in #14580
    4. v0.40.0Diffusers 0.40.0: New pipelines, tensor-parallel support, improved CLI, and more2026-08-20 · 2026-08-20 14:53 UTC
      查看更新说明

      [!TIP] This release features several new pipelines, including LTX2.5, MiniMax H3, and Wan Animate 2. We're also graduating Modular Diffusers out of the experimental phase and announcing its stable support. Additionally, this release includes minimal support for tensor-parallel. There's a lot more that went down in this release. So, please consult the notes for details.

      New Pipelines

      MiniMax-H3

      MiniMax-H3 generates video and its soundtrack together. A single transformer denoises one packed sequence containing the text conditioning, the conditioning media, and the target video and audio latents — there is no separate vocoder and no post-hoc audio pass. Its conditioner is a Qwen3VLForConditionalGeneration whose unnormalized 50th-decoder-layer hidden state is read instead of the last one.

      MiniMax-H3 is integrated as Modular Diffusers blocks only — MiniMaxH3Blocks and their MiniMaxH3ModularPipeline are the whole integration. The conversion ships both checkpoint partitions in one repository and exposes three workflows (t2va, fl2va, ref2va) that can be pruned at from_pretrained time so only that task's components are declared and downloaded.

      MiniMax Music 3

      MiniMax Music 3 produces complete songs up to five minutes long from lyrics and a music description, with expressive vocals and long-range structure. It is a hybrid of an autoregressive and a diffusion stage: an 8B Qwen3-based global language model predicts one semantic audio token per frame while a small depth decoder fills in seven residual RVQ codebooks, and their fused hidden states condition a 2.4B flow-matching transformer that produces Flow-VAE latents in overlapping chunks. A DAC-style decoder turns the latents into 44.1 kHz stereo audio.

      Stable Audio 3

      Stable Audio 3 is a text-to-audio model from Stability AI that generates high-quality stereo audio at 44.1 kHz. It uses a rectified-flow DiT conditioned on a frozen T5Gemma text encoder (via cross-attention) and on duration (a float embedded by StableAudio3DurationEmbedder and used for adaptive layer norm), and decodes with the SAME (Semantically-Aligned Music Encoder) autoencoder, AutoencoderSAME.

      Three pipelines ship: StableAudio3Pipeline, StableAudio3AudioToAudioPipeline, and StableAudio3InpaintPipeline.

      Thanks to @buffett0323 for the contribution (https://github.com/huggingface/diffusers/pull/14119).

      LTX-2.5

      LTX-2.5 reuses the existing LTX2Pipeline / LTX2VideoTransformer3DModel / AutoencoderKLLTX2Video classes — there is no separate pipeline class. The user-visible difference is the text encoder: LTX-2.5 is paired with a Gemma 4 (gemma4_unified) checkpoint, loaded automatically from a converted LTX-2.5 repo.

      Lightricks/LTX-2.5-Diffusers ships both the distilled DiT (transformer/) and the full/SFT DiT (transformer_full/), plus everything two-stage generation needs. Alongside the checkpoint support, this release adds:

      • LTX2VideoDiffusionDecoderModel and LTX2VideoDiffusionDecodePipeline — a second video decoder over the same latent space, so latents are interchangeable between decoders.
      • A duration_head that predicts shot length from the text-connector output, so num_frames is auto-predicted by default when the loaded pipeline has one.
      • Prompt enhancement through a separate off-the-shelf google/gemma-4-E2B-it checkpoint (enable_prompt_enhancement=True).
      • LTX25AutoBlocks for Modular Diffusers (#14453).

      Wan-An

    5. v0.39.0Diffusers 0.39.0: New image and video pipelines, core library improvements, and more2026-07-03 · 2026-07-03 08:55 UTC
      查看更新说明

      New Pipelines

      Cosmos 3

      Cosmos 3 is NVIDIA's unified world foundation model (WFM) for Physical AI — a single omni-model built on a Mixture-of-Transformers (MoT) architecture that combines world generation, physical reasoning, and action generation, replacing the separate Predict, Reason, and Transfer models from earlier Cosmos releases. A single Cosmos3OmniTransformer runs a Qwen-style language model in parallel with a diffusion generation pathway, joined by a 3D multimodal RoPE. This release also lands video-to-video and action-conditioned generation, and a sound encoder.

      Thanks to @atharvajoshi10, @yzhautouskay, and @MaciejBalaNV for the contributions.

      Ideogram 4

      Ideogram 4 is a flow-matching text-to-image model that uses a multimodal text encoder and an asymmetric classifier-free guidance scheme: a dedicated unconditional_transformer produces the negative branch with zeroed text features, while the main transformer consumes the full packed text + image sequence. The pipeline ships with structured prompt upsampling and LoRA loading support.

      Thanks to @JinLiIdeogram for the contribution.

      Krea 2

      Krea 2 (K2) is a flow-matching text-to-image model built around a single-stream MMDiT with grouped-query attention. A Qwen3-VL text encoder provides the conditioning — hidden states from twelve decoder layers are tapped per token and fused inside the transformer by a small text-fusion stage — and images are decoded with the Qwen-Image VAE. Both the base (midtrain) and TDM (distilled, few-step) checkpoints are supported, alongside a LoRA DreamBooth trainer.

      Thanks to @EleaZhong and @Abhinay1997 for the contribution.

      DreamLite

      DreamLite is a text-to-image and image-editing model from ByteDance. It pairs a custom 2D U-Net (DreamLiteUNetModel) with the Qwen3-VL multimodal encoder as its prompt / image-instruction encoder, and uses an AutoencoderTiny (TAESD-style) VAE for fast latent encode/decode. A distilled DreamLiteMobilePipeline targets on-device, low-latency generation.

      Thanks to @Carlofkl for the contribution.

      PRX Pixel

      PRXPixel is a pixel-space text-to-image generation model by Photoroom. A ~7B PRXTransformer2DModel denoises raw RGB images directly — no VAE is needed. The model is conditioned on a Qwen3-VL text encoder and uses flow matching where the transformer predicts the clean image at each step (x-prediction).

    README 原文

    未翻译
    在 GitHub 打开

    🤗 Diffusers is the go-to library for state-of-the-art pretrained diffusion models for generating images, audio, and even 3D structures of molecules. Whether you're looking for a simple inference solution or training your own diffusion models, 🤗 Diffusers is a modular toolbox that supports both. Our library is designed with a focus on usability over performance, simple over easy, and customizability over abstractions.

    🤗 Diffusers offers three core components:

    • State-of-the-art diffusion pipelines that can be run in inference with just a few lines of code.
    • Interchangeable noise schedulers for different diffusion speeds and output quality.
    • Pretrained models that can be used as building blocks, and combined with schedulers, for creating your own end-to-end diffusion systems.

    Installation

    We recommend installing 🤗 Diffusers in a virtual environment from PyPI or Conda. For more details about installing PyTorch, please refer to their official documentation.

    PyTorch

    With pip (official package):

    pip install --upgrade diffusers[torch]

    With conda (maintained by the community):

    conda install -c conda-forge diffusers

    Apple Silicon (M1/M2) support

    Please refer to the How to use Stable Diffusion in Apple Silicon guide.

    Quickstart

    Generating outputs is super easy with 🤗 Diffusers. To generate an image from text, use the from_pretrained method to load any pretrained diffusion model (browse the Hub for 30,000+ checkpoints):

    from diffusers import DiffusionPipeline
import torch

pipeline = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.float16)
pipeline.to("cuda")
pipeline("An image of a squirrel in Picasso style").images[0]

    You can also dig into the models and schedulers toolbox to build your own diffusion system:

    from diffusers import DDPMScheduler, UNet2DModel
from PIL import Image
import torch

scheduler = DDPMScheduler.from_pretrained("google/ddpm-cat-256")
model = UNet2DModel.from_pretrained("google/ddpm-cat-256").to("cuda")
scheduler.set_timesteps(50)

sample_size = model.config.sample_size
noise = torch.randn((1, 3, sample_size, sample_size), device="cuda")
input = noise

for t in scheduler.timesteps:
    with torch.no_grad():
        noisy_residual = model(input, t).sample
        prev_noisy_sample = scheduler.step(noisy_residual, t, input).prev_sample
        input = prev_noisy_sample

image = (input / 2 + 0.5).clamp(0, 1)
image = image.cpu().permute(0, 2, 3, 1).numpy()[0]
image = Image.fromarray((image * 255).round().astype("uint8"))
image

    Check out the Quickstart to launch your diffusion journey today!

    How to navigate the documentation

    DocumentationWhat can I learn?
    QuickstartA basic crash course for loading pipelines, generating outputs, and applying common inference optimizations.
    LoadingGuides for how to load and configure all the components (pipelines, models, and schedulers) of the library, as well as how to use different schedulers.
    Modular DiffusersBuild flexible diffusion systems from modular pipeline components.
    OptimizationGuides for how to optimize your diffusion model to run faster and consume less memory.
    TrainingGuides for how to train a diffusion model for different tasks with different training techniques.

    Contribution

    We ❤️ contributions from the open-source community! If you want to contribute to this library, please check out our Contribution guide. If you are using an AI agent, please point it at the project conventions in .ai/ first (install them as a plugin with claude plugin marketplace add huggingface/diffusers, or a skill at a time with diffusers-cli skills add <name>) — see Coding with AI agents. You can look out for issues you'd like to tackle to contribute to the library.

    Also, say 👋 in our public Discord channel . We discuss the hottest trends about diffusion models, help each other with contributions, personal projects or just hang out ☕.

    Popular Tasks & Pipelines

    Popular libraries using 🧨 Diffusers

    Thank you for using us ❤️.

    Credits

    This library concretizes previous work by many different authors and would not have been possible without their great research and implementations. We'd like to thank, in particular, the following implementations which have helped us in our development and without which the API could not have been as polished today:

    • @CompVis' latent diffusion models library, available here
    • @hojonathanho original DDPM implementation, available here as well as the extremely useful translation into PyTorch by @pesser, available here
    • @ermongroup's DDIM implementation, available here
    • @yang-song's Score-VE and Score-VP implementations, available here

    We also want to thank @heejkoo for the very helpful overview of papers, code and resources on diffusion models, available here as well as @crowsonkb and @rromb for useful discussions and insights.

    Citation

    @misc{von-platen-etal-2022-diffusers,
  author = {Patrick von Platen and Suraj Patil and Anton Lozhkov and Pedro Cuenca and Nathan Lambert and Kashif Rasul and Mishig Davaadorj and Dhruv Nair and Sayak Paul and William Berman and Yiyi Xu and Steven Liu and Thomas Wolf},
  title = {Diffusers: State-of-the-art diffusion models},
  year = {2022},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/huggingface/diffusers}}
}

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    stable-diffusion-webui
    AUTOMATIC1111·Python

    基于 Gradio 的 Stable Diffusion 网页界面，把文生图、图生图、局部重绘等能力做成可视化操作面板，不用写代码就能跑图像生成。

    图像生成+4图像生成
    165275+2931kv1.10.12026-03-02 07:00 UTC
    ComfyUI
    Comfy-Org·Python

    基于节点图的扩散模型图形界面与后端，支持本地快速推理和 API 调用，方便搭建和调试图像生成工作流。

    扩散模型+4图像生成
    146026+54017kv0.44.02026-11-10 04:58 UTC
    OpenMontage
    OpenMontageInc·Python

    把 Claude、Cursor 等 AI 编程助手变成完整视频制作工作室，内置 12 条制作流水线和 100+ 工具，覆盖脚本、配音、图像生成到剪辑全流程。

    视频制作+4图像生成
    79189+53511k2026-11-09 06:23 UTC
    OpenMontage
    calesthio·Python

    把 Claude、Cursor 等 AI 编程助手变成完整的视频制作工作室，内置 12 条制作流水线和 100+ 工具，覆盖从生成到剪辑的全流程。

    视频制作+4图像生成
    7786710k2026-11-03 17:42 UTC