登录

accelerate

huggingfaceApache-2.0活跃维护

只需几行代码就能把 PyTorch 模型跑在任意设备或分布式配置上，自动处理混合精度（含 fp8）和 FSDP、DeepSpeed 配置，省去手写分布式训练样板代码的麻烦。

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

v1.15.0发布于 2026-09-09 13:30 UTC9,937star +1 / 7 天1,597fork同步于 2026-11-10 11:30 UTC
分布式训练混合精度FSDPDeepSpeedPyTorch
对比 GitHub 官网

怎么上手

由 README 提炼，执行前请确认
原文 
pip install accelerate
硬件要求按 README 说明

README 未说明具体显存要求；支持纯 CPU、单机或多机多 CPU、单 GPU、单机或多机多 GPU 以及 TPU。

纯 CPU 可运行多卡 / 张量并行
下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

不改训练循环就能把原生 PyTorch 脚本跑在单卡、多卡、TPU 和混合精度上的轻量封装库

只需几行代码就能把 PyTorch 模型跑在任意设备或分布式配置上，自动处理混合精度（含 fp8）和 FSDP、DeepSpeed 配置，省去手写分布式训练样板代码的麻烦。

核心能力

  • 用 Accelerator 和 prepare 包装模型、优化器、数据加载器
  • 用 accelerator.backward 替换 loss.backward 以支持混合精度
  • 提供 accelerate config 交互式生成环境配置
  • 用 accelerate launch 统一启动多卡、TPU、MPI 任务
  • 支持 fp8、fp16、bf16 混合精度与 FSDP、DeepSpeed

容易踩的坑

需先按 PyTorch 官方指引自行安装 PyTorch，仓库在 Python 3.11+ 和 PyTorch 2.6.0+ 上测试；FP8 混合精度需额外安装 Transformer Engine 或 torchao；DeepSpeed、FSDP、Megatron-LM 支持在 README 中标注为实验性。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本v1.15.0
全部 release
  1. v1.15.0v1.15.0: FSDP2 activation memory, dtensor improvements最新正式版2026-09-09 · 2026-09-09 13:30 UTC

    v1.15.0: FSDP2 activation memory, dtensor improvements

    FSDP2

    A large batch of FSDP2 work this release: two fixes that cut activation memory at long sequence lengths, tied-embedding support on torch >= 2.13, and a round of checkpointing correctness and scale fixes.

    Activation checkpointing was wrapping each child of the matched layer (self_attn, mlp, the norms) instead of the layer itself, so every inter-child activation stayed saved for backward. It now wraps the layer.

    There's also a new FSDP2-only activation_checkpointing_offload, which moves the remaining per-layer checkpoint inputs to pinned CPU memory. Gradients are exactly those of plain activation checkpointing:

    # fsdp2.yaml
fsdp_config:
  fsdp_version: 2
  fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP
  fsdp_activation_checkpointing: true
  fsdp_activation_checkpointing_offload: true

    accelerate launch --config_file fsdp2.yaml train.py
    • FSDP2 activation checkpointing: wrap the matched transformer layer itself, not each of its children by @qgallouedec in #4172
    • Add FSDP2 activation_checkpointing_offload: offload checkpointed layer inputs to pinned CPU memory by @qgallouedec in #4175
    • Fix FSDP2 tied-embedding models on torch >= 2.13: put the output embedding in the same fully_shard group by @qgallouedec in #4171
    • Fix FSDP2/PEFT/FULL_STATE_DICT dropping every rank's adapter shard except rank 0 by @AmineDiro in #4206
    • FSDP2: per-rank torch.save/load for SHARDED_STATE_DICT to fix 2800+ NPU checkpoint timeout by @gygdh-001 in #4105
    • Fix FSDP sharded checkpoint path resolution by @HaomingSong in #4119
    • Raise a clear error when FSDP is enabled on a mesh with no shard dimension by @qgallouedec in #4180

    DTensor

    Two fixes for DTensor-sharded models, which you hit with FSDP2, tensor parallelism, or any N-D parallelism setup: gradient clipping no longer fails on the foreach op when plain tensors and DTensors are mixed, and prepare_model leaves an already-sharded model where it is:

    • Clip grad norm support for dtensors by @michaelbenayoun in #4219
    • prepare_model: don't move DTensor-sharded models to the device by @qgallouedec in #4181

    Offloading & Quantization

    An entire model can now be dispatched to disk, including tied weights — useful for tools like llm-compressor that compress large models on machines that can't hold them:

    • [Offloading] Support full disk offloading by @kylesayrs in #4083
    • Fix disk offload crash on FP8 tensors by @shoemoney in #4151

    Trackers

    Custom trackers can be registered by name and then selected from log_with= like any built-in one:

    from accelerate import Accelerator
from accelerate.tracking import register_tracker_class

register_tracker_class(MyTracker)  # MyTracker.name == "my_tracker"
accelerator = Accelerator(log_with="my_tracker")
    • Add register_tracker_class to register custom trackers by name by @javierdejesusda in #4060
    • Fix MLFLOW_NESTED_RUN never being able to turn nesting off by @yupengtang in #4223

    Device support

    Neuron gains a torch dynamo backend (so --torch-compile works with the Transformers Trainer) and MPS is now reported and handled properly by accelerate env and find_executable_batch_size.

    • Add neuron backend for torch dynamo by @michaelbenayoun in #4097
    • Add the neuron device branch in state by @michaelbenayoun in #4218
    • Report MPS as the accelerator in accelerate env by @xquantize in #4158
    • Fix accelerate env never reading the MPS chip name by @yupengtang in #4221
    • Treat MPS out-of-memory errors as OOM in find_executable_batch_size by @caiotheodoro in #4227
    • Fix memory accounting for integrated CUDA devices by @harshitakrishna18 in #4187
    • Ensure device-agnostic in docs by @kaixuanliu in #4163

    CLI

    • Handle missing accelerate executable in env command by @tandede in #4168
    • Fix accelerate launch --cpu never setting the KMP variables by @yupe
  2. v1.14.0v1.14.0: AMD ROCm support, FSDP2 hardening2026-06-11 · 2026-06-11 13:58 UTC
    查看更新说明

    FSDP2 Improvements

    This release brings a large batch of FSDP2 fixes and quality-of-life improvements: correct dtype handling on load, sharding of embeddings/norms, QLoRA crash prevention, and a more robust auto-wrap policy.

    • Fsdp2 fully_shard embedding and norm by @SunMarc in #4015
    • Fix fsdp2 load full state dict dtype mismatch by @SunMarc in #4021
    • Fix region compilation fsdpv2 by @SunMarc in #4022
    • [FSDP2] Cast model to uniform dtype before fully_shard to fix mixed-dtype AssertionError by @roycho96 in #3985
    • [FSDP2] Auto-exclude non-floating frozen Params4bit from fully_shard to prevent QLoRA crash by @roycho96 in #3987
    • fix(FSDP2): auto-wrap policy ignoring _no_split_modules fallback by @JohnGiorgi in #3999
    • fix: use key-based matching in fsdp2_load_full_state_dict by @roycho96 in #3982
    • fix: add missing model_has_params4bit guard to fsdp2_load_full_state_dict call by @roycho96 in #3981
    • Fix to-fsdp2: drop REMOVED / NOT_YET_IMPLEMENTED FSDP1 keys instead of leaking them by @lollinng in #4065
    • Prevent double-wrapping models in prepare_model() by @joshuaswanson in #3977

    AMD ROCm support

    Accelerate now works end-to-end on AMD ROCm devices. Thanks @Abdennacer-Badaoui!

    • Make accelerate work end-to-end on AMD ROCm by @Abdennacer-Badaoui in #4025

    Neuron

    Further Neuron improvements to reduce recompilation and cover missing device cases.

    • Add padded allgather and broadcast for Neuron devices to reduce recompilation by @czkkkkkk in #4000
    • fix: add missing neuron device case by @michaelbenayoun in #4042

    Quantization & Offloading

    We improved offloading support for quantized models, including Torchao, int8, and tied-weight handling.

    • Torchao offload by @SunMarc in #3973
    • Fix int8 offload hook detachment statistics restoration by @jiqing-feng in #4044
    • Fix keep_in_fp32_modules not working for tied weights in load_and_quantize_model by @jiqing-feng in #4043
    • Fix dtype_byte_size for FP8 fnuz / e8m0fnu dtypes by @lollinng in #4063

    Data Loading

    • Feat: Support dynamic batch size in BatchSamplerShard with even_batches by @yuxinyuan in #3969
    • Fix iterable dataset sharding condition when n_shards == num_processes by @SunMarc in #3958
    • Fix implicit padding in split_between_processes when apply_padding=False and num_samples < num_processes by @3manifold in #4052

    Minor fixes

    • [DeepSpeed] allow kernels flash-attn in SP by @kashif in #3959
    • Fix: Conditionally import torch.distributed.algorithms.join in accelerator.py by @0xDELUXA in #3962
    • Fix is_hf_initialized attribute by @SunMarc in #3976
    • feat(utils): add max reduction type by @imstevenpmwork in #4027
    • fix(state): make MLU backend part of the _prepare_backend elif chain by @Anai-Guo in #4057
    • fix notebook launcher cuda init by @SunMarc in #4059
    • pytorch-triton-xpu rename to triton-xpu by @sywangyi in #4007
    • Relax numerical tolerance for XPU in test_big_modeling by @YangKai0616 in #4001
    • Fix gloo backend error in test_load_checkpoint_and_dispatch_with_broadcast on XPU by @kaixuanliu in #4056
    • Raise ValueError instead of a bare string in ParallelismConfig.get_device_mesh by @lollinng in #4064
    • tests: Gracefully handle missing set_device for mps by @booxter in #4028
    • test: add regression test for no_split_module_classes accepting set type by @UFO0506 in #4048
    • Fix all tests by @SunMarc in #4072
    • docs: add aggregate profiler memory example by @aryanputta in #4054
    • DOC: document missing parameters in load_accelerator_state, find_executable_batch_size, and send_to_device by @kratos0718 in #4051
    • docs: Fix docstring of fsdp2_prepare_auto_wrap_policy by @slocoro in #4037
    • Fix DistributedType documentation by @3manifold in #3980
    • Fix grammar, spelling, and consistency issues across docs and examples by @cihandemir in #3961
    • docs: fix typos in docstrings, comments, and user docs by @mokashang in #4040
    • chor
  3. v1.13.0v1.13.0: Neuron support, IPEX removal, and distributed training fixes2026-03-04 · 2026-03-04 20:15 UTC
    查看更新说明

    AWS Neuron support

    We now have support for AWS Neuron (Trainium/Inferentia) devices. Thanks @michaelbenayoun for adding this.

    XPU Improvements

    We've removed IPEX dependency and improved device-agnostic code for XPU.

    FSDP2 Improvements

    We've added a bunch of important fixes for FSDP2 users: upcasting only grad-requiring params, better tied embedding errors, DCP optimizer loading, bf16 optimizer step crash fix, and torch < 2.7.0 compatibility.

    DeepSpeed Sequence Parallelism

    We've added several fixes to the DeepSpeed + Sequence Parallelism integration introduced in v1.12.0, including evaluation support during SP training and proper process group handling.

    FP8

    We've enhanced FP8 training. Thanks @shimizust for fixing torchao support.

    Performance

    Accelerate now imports faster by deferring heavy dependencies, and torch.compile hooks are disabled lazily.

    Minor fixes

  4. v1.12.0v1.12.0: Deepspeed Ulysses/ALST2025-11-21 · 2025-11-21 12:47 UTC
    查看更新说明

    Deepspeed Ulysses/ALST integration

    Deepspeed Ulysses/ALST is an efficient way of training on long sequences by employing sequence parallelism and attention head parallelism. You can learn more about this technology in this paper https://arxiv.org/abs/2506.13996 or this deepspeed tutorial https://www.deepspeed.ai/tutorials/ulysses-alst-sequence-parallelism/.

    To enable Deepspeed Ulysses, you first need to create ParallelismConfig and setting sp related args:

    parallelism_config = ParallelismConfig(
    sp_backend="deepspeed",
    sp_size=2,
    sp_handler=DeepSpeedSequenceParallelConfig(...),
)

    Then, you need to make sure to compute the correct loss as described on our docs

            ...
        losses_per_rank = torch.distributed.nn.functional.all_gather(loss, group=sp_group)
        good_tokens = (shift_labels != -100).view(-1).sum()
        good_tokens_per_rank = torch.distributed.nn.functional.all_gather(good_tokens, group=sp_group)
        total_loss = sum(
            losses_per_rank[rank] * good_tokens_per_rank[rank]
            for rank in range(sp_world_size)
            if good_tokens_per_rank[rank] > 0
        )
        total_good_tokens = sum(good_tokens_per_rank)
        loss = total_loss / max(total_good_tokens, 1)

    Thanks @S1ro1 for starting this work and for @stas00 for finishing this work. Also thanks @kashif for adding docs and reviewing/testing this PR !

    This feature will also be available in HF Trainer thanks for this PR from @stas00: https://github.com/huggingface/transformers/pull/41832

    Minor changes

    New Contributors

    Full Changelog: https://github.com/huggingface/accelerate/compare/v1.11.0...v1.12.0

  5. v1.11.0v1.11.0: TE MXFP8, FP16/BF16 with MPS, Python 3.10 2025-10-20 · 2025-10-20 16:08 UTC
    查看更新说明

    TE MXFP8 support

    We've added support for MXFP8 in our TransformerEngine integration. To use that, you need to set use_mxfp8_block_scaling in fp8_config. See nvidia docs [here]. (https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/examples/fp8_primer.html#MXFP8-and-block-scaling)

    FP16/BF16 Training for MPS devices

    BF16 and FP16 support for MPS devices is finally here. You can now pass mixed_precision = "fp16" or "bf16" when training on a mac (fp16 requires torch 2.8 and bf16 requires torch 2.6)

    FSDP updates

    The following PRs add respectively support to ignored_params and no_sync() for FSDPv2:

    Mixed precision can now be passed as a dtype string from accelerate cli flag or fsdp_config in accelerate config file:

    Nd-parallel updates

    Some minor updates concerning nd-parallelism.

    Bump to Python 3.10

    We've dropped support for python 3.9 as it reached EOL in October.

    Lots of minor fixes:

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    Easy to integrate

    🤗 Accelerate was created for PyTorch users who like to write the training loop of PyTorch models but are reluctant to write and maintain the boilerplate code needed to use multi-GPUs/TPU/fp16.

    🤗 Accelerate abstracts exactly and only the boilerplate code related to multi-GPUs/TPU/fp16 and leaves the rest of your code unchanged.

    Here is an example:

      import torch
  import torch.nn.functional as F
  from datasets import load_dataset
+ from accelerate import Accelerator

+ accelerator = Accelerator()
- device = 'cpu'
+ device = accelerator.device

  model = torch.nn.Transformer().to(device)
  optimizer = torch.optim.Adam(model.parameters())

  dataset = load_dataset('my_dataset')
  data = torch.utils.data.DataLoader(dataset, shuffle=True)

+ model, optimizer, data = accelerator.prepare(model, optimizer, data)

  model.train()
  for epoch in range(10):
      for source, targets in data:
          source = source.to(device)
          targets = targets.to(device)

          optimizer.zero_grad()

          output = model(source)
          loss = F.cross_entropy(output, targets)

-         loss.backward()
+         accelerator.backward(loss)

          optimizer.step()

    As you can see in this example, by adding 5-lines to any standard PyTorch training script you can now run on any kind of single or distributed node setting (single CPU, single GPU, multi-GPUs and TPUs) as well as with or without mixed precision (fp8, fp16, bf16).

    In particular, the same code can then be run without modification on your local machine for debugging or your training environment.

    🤗 Accelerate even handles the device placement for you (which requires a few more changes to your code, but is safer in general), so you can even simplify your training loop further:

      import torch
  import torch.nn.functional as F
  from datasets import load_dataset
+ from accelerate import Accelerator

- device = 'cpu'
+ accelerator = Accelerator()

- model = torch.nn.Transformer().to(device)
+ model = torch.nn.Transformer()
  optimizer = torch.optim.Adam(model.parameters())

  dataset = load_dataset('my_dataset')
  data = torch.utils.data.DataLoader(dataset, shuffle=True)

+ model, optimizer, data = accelerator.prepare(model, optimizer, data)

  model.train()
  for epoch in range(10):
      for source, targets in data:
-         source = source.to(device)
-         targets = targets.to(device)

          optimizer.zero_grad()

          output = model(source)
          loss = F.cross_entropy(output, targets)

-         loss.backward()
+         accelerator.backward(loss)

          optimizer.step()

    Want to learn more? Check out the documentation or have a look at our examples.

    Launching script

    🤗 Accelerate also provides an optional CLI tool that allows you to quickly configure and test your training environment before launching the scripts. No need to remember how to use torch.distributed.run or to write a specific launcher for TPU training! On your machine(s) just run:

    accelerate config

    and answer the questions asked. This will generate a config file that will be used automatically to properly set the default options when doing

    accelerate launch my_script.py --args_to_my_script

    For instance, here is how you would run the GLUE example on the MRPC task (from the root of the repo):

    accelerate launch examples/nlp_example.py

    This CLI tool is optional, and you can still use python my_script.py or python -m torchrun my_script.py at your convenience.

    You can also directly pass in the arguments you would to torchrun as arguments to accelerate launch if you wish to not run accelerate config.

    For example, here is how to launch on two GPUs:

    accelerate launch --multi_gpu --num_processes 2 examples/nlp_example.py

    To learn more, check the CLI documentation available here.

    Or view the configuration zoo here

    Launching multi-CPU run using MPI

    🤗 Here is another way to launch multi-CPU run using MPI. You can learn how to install Open MPI on this page. You can use Intel MPI or MVAPICH as well. Once you have MPI setup on your cluster, just run:

    accelerate config

    Answer the questions that are asked, selecting to run using multi-CPU, and answer "yes" when asked if you want accelerate to launch mpirun. Then, use accelerate launch with your script like:

    accelerate launch examples/nlp_example.py

    Alternatively, you can use mpirun directly, without using the CLI like:

    mpirun -np 2 python examples/nlp_example.py

    Launching training using DeepSpeed

    🤗 Accelerate supports training on single/multiple GPUs using DeepSpeed. To use it, you don't need to change anything in your training code; you can set everything using just accelerate config. However, if you desire to tweak your DeepSpeed related args from your Python script, we provide you the DeepSpeedPlugin.

    from accelerate import Accelerator, DeepSpeedPlugin

# deepspeed needs to know your gradient accumulation steps beforehand, so don't forget to pass it
# Remember you still need to do gradient accumulation by yourself, just like you would have done without deepspeed
deepspeed_plugin = DeepSpeedPlugin(zero_stage=2, gradient_accumulation_steps=2)
accelerator = Accelerator(mixed_precision='fp16', deepspeed_plugin=deepspeed_plugin)

# How to save your 🤗 Transformer?
accelerator.wait_for_everyone()
unwrapped_model = accelerator.unwrap_model(model)
unwrapped_model.save_pretrained(save_dir, save_function=accelerator.save, state_dict=accelerator.get_state_dict(model))

    Note: DeepSpeed support is experimental for now. In case you get into some problem, please open an issue.

    Launching your training from a notebook

    🤗 Accelerate also provides a notebook_launcher function you can use in a notebook to launch a distributed training. This is especially useful for Colab or Kaggle notebooks with a TPU backend. Just define your training loop in a training_function then in your last cell, add:

    from accelerate import notebook_launcher

notebook_launcher(training_function)

    An example can be found in this notebook. Open In Colab

    Why should I use 🤗 Accelerate?

    You should use 🤗 Accelerate when you want to easily run your training scripts in a distributed environment without having to renounce full control over your training loop. This is not a high-level framework above PyTorch, just a thin wrapper so you don't have to learn a new library. In fact, the whole API of 🤗 Accelerate is in one class, the Accelerator object.

    Why shouldn't I use 🤗 Accelerate?

    You shouldn't use 🤗 Accelerate if you don't want to write a training loop yourself. There are plenty of high-level libraries above PyTorch that will offer you that, 🤗 Accelerate is not one of them.

    Frameworks using 🤗 Accelerate

    If you like the simplicity of 🤗 Accelerate but would prefer a higher-level abstraction around its capabilities, some frameworks and libraries that are built on top of 🤗 Accelerate are listed below:

    • Amphion is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.
    • Animus is a minimalistic framework to run machine learning experiments. Animus highlights common "breakpoints" in ML experiments and provides a unified interface for them within IExperiment.
    • Catalyst is a PyTorch framework for Deep Learning Research and Development. It focuses on reproducibility, rapid experimentation, and codebase reuse so you can create something new rather than write yet another train loop. Catalyst provides a Runner to connect all parts of the experiment: hardware backend, data transformations, model training, and inference logic.
    • fastai is a PyTorch framework for Deep Learning that simplifies training fast and accurate neural nets using modern best practices. fastai provides a Learner to handle the training, fine-tuning, and inference of deep learning algorithms.
    • Finetuner is a service that enables models to create higher-quality embeddings for semantic search, visual similarity search, cross-modal text<->image search, recommendation systems, clustering, duplication detection, anomaly detection, or other uses.
    • InvokeAI is a creative engine for Stable Diffusion models, offering industry-leading WebUI, terminal usage support, and serves as the foundation for many commercial products.
    • Kornia is a differentiable library that allows classical computer vision to be integrated into deep learning models. Kornia provides a Trainer with the specific purpose to train and fine-tune the supported deep learning algorithms within the library.
    • Open Assistant is a chat-based assistant that understands tasks, can interact with their party systems, and retrieve information dynamically to do so.
    • pytorch-accelerated is a lightweight training library, with a streamlined feature set centered around a general-purpose Trainer, that places a huge emphasis on simplicity and transparency; enabling users to understand exactly what is going on under the hood, but without having to write and maintain the boilerplate themselves!
    • Stable Diffusion web UI is an open-source browser-based easy-to-use interface based on the Gradio library for Stable Diffusion.
    • torchkeras is a simple tool for training pytorch model just in a keras style, a dynamic and beautiful plot is provided in notebook to monitor your loss or metric.
    • transformers as a tool for helping train state-of-the-art machine learning models in PyTorch, Tensorflow, and JAX. (Accelerate is the backend for the PyTorch side).

    Installation

    This repository is tested on Python 3.11+ and PyTorch 2.6.0+

    You should install 🤗 Accelerate in a virtual environment. If you're unfamiliar with Python virtual environments, check out the user guide.

    First, create a virtual environment with the version of Python you're going to use and activate it.

    Then, you will need to install PyTorch: refer to the official installation page regarding the specific install command for your platform. Then 🤗 Accelerate can be installed using pip as follows:

    pip install accelerate

    Supported integrations

    • CPU only
    • multi-CPU on one node (machine)
    • multi-CPU on several nodes (machines)
    • single GPU
    • multi-GPU on one node (machine)
    • multi-GPU on several nodes (machines)
    • TPU
    • FP16/BFloat16 mixed precision
    • FP8 mixed precision with Transformer Engine or torchao
    • DeepSpeed support (Experimental)
    • PyTorch Fully Sharded Data Parallel (FSDP) support (Experimental)
    • Megatron-LM support (Experimental)

    Citing 🤗 Accelerate

    If you use 🤗 Accelerate in your publication, please cite it by using the following BibTeX entry.

    @Misc{accelerate,
  title =        {Accelerate: Training and inference at scale made simple, efficient and adaptable.},
  author =       {Sylvain Gugger and Lysandre Debut and Thomas Wolf and Philipp Schmid and Zachary Mueller and Sourab Mangrulkar and Marc Sun and Benjamin Bossan},
  howpublished = {\url{https://github.com/huggingface/accelerate}},
  year =         {2022}
}

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    LocalAI
    mudler·Go

    开源本地 AI 推理引擎，无需 GPU 即可在任意硬件上跑 LLM、视觉、语音和图像生成模型，提供兼容 OpenAI 的 API。

    推理引擎+4分布式训练
    50283+474.6kv4.12.12026-11-10 10:59 UTC
    DeepSpeed
    deepspeedai·Python

    基于 PyTorch 的深度学习优化库，通过 ZeRO、模型并行和混合专家等技术，让千亿到万亿参数模型的分布式训练与推理更省显存、更高效。

    分布式训练+4分布式训练
    43316+95.1kv0.19.92026-11-10 11:01 UTC
    ColossalAI
    hpcaitech·Python

    一套面向大模型训练的分布式并行框架，把数据、模型、流水线并行和异构训练整合成统一接口，让训练和推理更省显存、更快。

    分布式训练+4分布式训练
    41431+34.5kv0.5.02026-11-10 10:33 UTC
    dgraph
    dgraph-io·Go

    高性能分布式图数据库，专为实时查询场景设计，支持知识图谱等大规模关联数据的存储与毫秒级检索。

    图数据库+4分布式训练
    21823+31.6kv25.5.12026-11-05 23:30 UTC