登录

lws

kubernetes-sigsApache-2.0活跃维护

Kubernetes 上的 LeaderWorkerSet API，用来编排多机多卡的分布式推理工作负载，把大模型服务拆成 leader 和 worker 组统一调度。

LeaderWorkerSet & DisaggregatedSet: Kubernetes APIs for Distributed and Disaggregated AI Serving

v1.0.0发布于 2026-10-16 14:26 UTC906star +3 / 7 天210fork同步于 2026-11-10 17:54 UTC
分布式推理Kubernetes大模型服务多机多卡推理编排
对比 GitHub 官网llm-inferencesig-apps

怎么上手

由 README 提炼，执行前请确认
原文 
README 未给出具体安装命令，仅指向 https://lws.sigs.k8s.io/docs/installation/ 安装指南
硬件要求按 README 说明

README 未说明具体显存或 GPU 型号要求，仅说明面向多节点多设备分片推理场景

下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

Kubernetes 上部署多机推理负载的 API，把一组 Pod 当作一个复制单元来管理

Kubernetes 上的 LeaderWorkerSet API，用来编排多机多卡的分布式推理工作负载，把大模型服务拆成 leader 和 worker 组统一调度。

核心能力

  • 把一组 Pod 当作一个 super pod 统一管理
  • 每个 Pod 有 0 到 n-1 的唯一索引身份
  • leader 与 worker 双模板，可分别指定
  • 支持组级滚动更新与 HPA 扩缩容
  • 可选 gang scheduling 与拓扑感知放置

容易踩的坑

官方仅支持并测试 Kubernetes 1.34 及以上版本；gang scheduling 处于 Alpha，API 可能变化；安装需按官方安装指南操作，README 未提供一键命令

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本v1.0.0
全部 release
  1. v1.0.0最新正式版2026-10-16 · 2026-10-16 14:26 UTC

    Highlights

    v1.0.0 improves rollout coordination and scaling for DisaggregatedSet, adds independently scalable sub-roles, and strengthens scheduling and recovery.

    • Fractional rollout window: Roles advance in proportion to their replica counts, allowing new batches to start before earlier ones are fully Ready and making fuller use of the configured maxSurge and maxUnavailable budgets without letting one role progress too far ahead. PR: #907.

    • Scaling during rollouts: Scale-up continues during rolling updates, while opt-in scale-down lets operators or autoscalers safely reduce capacity before the update finishes. KEP-1022; PR: #1166.

    • Independently scalable sub-roles: DisaggregatedSet can partition a role into separate scaling and routing pools that share the same workload configuration, with support for both Ordinal and Hash group identity. KEP-960; PRs: #964, #1173.

    • Native gang scheduling reliability: Fix PodGroup lifecycle and group-size update handling so group recreation and size-changing rollouts do not leave workloads stuck. The native integration remains alpha and opt-in. KEP-666; PRs: #1132, #1133, #1135.

    • Bounded group recovery: Set maxGroupRestarts to cap automatic recreations of each replica group and restartBackoff to limit repeated recreation frequency. Both support Ordinal and Hash group identity. KEP-820; PRs: #877, #1093, #1103.

    • Parallel Pod termination: Set podTerminationPolicy: Parallel to terminate leaders and workers concurrently during group rebuilds, rollouts, scale-down, and deletion. Sequential remains the default. KEP-956; PR: #1104.

    • Stable configuration API: Controller configuration graduates to config.lws.x-k8s.io/v1, while v1alpha1 remains supported but deprecated. PR: #1094.

    Urgent Upgrade Notes

    (No, really, you MUST read this before you upgrade)

    • DisaggregatedSet roles without a rollingUpdateConfiguration now roll out with maxSurge 0 and maxUnavailable 1, like LeaderWorkerSet, instead of maxSurge 1 and maxUnavailable 0. The CRDs now default rolloutStrategy and rollingUpdateConfiguration, so these values appear on the object. Set maxSurge 1 and maxUnavailable 0 on a role to keep bringing the new revision up before the old one drains. (#1169, @BenjaminBraunDev)
    • New DisaggregatedSets use a more robust revision hash that detects changes to startup policy, network configuration, labels and annotations, without triggering rollouts merely because roles are reordered. Existing DisaggregatedSets retain the previous hashing mechanism to avoid upgrade-triggered rollouts. The legacy hashing mechanism remains supported in this release, but is planned for deprecation in a future version that has not yet been specified. (#1100, @hasB4K)
    • V1.0.0 removes support for slice-less LeaderWorkerSets generated for DisaggregatedSets originally created with v0.9.x. If an affected DisaggregatedSet has not rolled out under v0.10.x or v0.11.x, install either version and complete one template rollout before upgrading to v1.0.0. (#1085, @hasB4K)

    Changes by Kind

    API Change

    • DisaggregatedSet roles can define independently scalable Ordinal subroles sharing one LeaderWorkerSet. Each subrole has its own rollout budget, Pod routing labels, scaler, and status nested under its parent. Single-role DisaggregatedSets are also supported. (#964, @hasB4K)
    • DisaggregatedSet status now reports desiredReplicas for each role and sub-role, in
  2. v0.11.12026-10-01 · 2026-10-01 19:06 UTC
    查看更新说明

    [release-0.11] Name hash worker StatefulSets after the leader host name by @BenjaminBraunDev in https://github.com/kubernetes-sigs/lws/pull/1112

  3. v0.11.02026-09-23 · 2026-09-23 19:54 UTC
    查看更新说明

    What's Changed

  4. v0.10.02026-08-11 · 2026-08-11 19:59 UTC
    查看更新说明

    What's Changed

  5. v0.9.02026-06-17 · 2026-06-17 20:44 UTC
    查看更新说明

    What's Changed

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    The LeaderWorkerSet and DisaggregatedSet APIs

    Latest Release Coverage Status

    LeaderWorkerSet (LWS): An API for deploying a group of pods as a unit of replication. It aims to address common deployment patterns of AI/ML inference workloads, especially multi-host inference workloads where the LLM will be sharded and run across multiple devices on multiple nodes.

    DisaggregatedSet (DS): An API to support advanced multi-node inference. LWS forms the core API for multi-node while DisaggregatedSet builds on it to add advanced disaggregated workload deployment with support for autoscaling, rollouts and failure handling.

    Both APIs are being co-designed with (CNCF sandbox project). llm-d is a high-performance distributed inference serving stack optimized for production deployments. This collaboration ensures that the APIs are optimized for real-world serving frameworks and disaggregated architectures, helping achieve state-of-the-art performance across hardware accelerators.

    Read the documentation or watch the related talks & presentations to learn more.

    Feature overview

    Core Features: LeaderWorkerSet

    • Group of Pods as a unit: Supports a tightly managed group of pods that represent a “super pod”
      • Unique pod identity: Each pod in the group has a unique index from 0 to n-1.
      • Parallel creation: Pods in the group will have the same lifecycle and be created in parallel.
      • Gang Scheduling: Each replica with a group of pods can be scheduled in an all-or-nothing manner (Alpha level, API may change in the future).
    • Dual-template, one for leader and one for the workers: A replica is a group of a single leader and a set of workers, and allow to specify a template for the workers and optionally use a second one for the leader pod.
    • Multiple groups with identical specifications: Supports creating multiple “replicas” of the above mentioned group. Each group is a single unit for rolling update, scaling, and maps to a single exclusive topology for placement.
    • A scale subresource: A scale endpoint is exposed for HPA to dynamically scale the number replicas (aka number of groups)
    • Rollout and Rolling update: Supports performing rollout and rolling update at the group level, which means the groups are upgraded one by one as a unit (i.e. the pods within a group are updated together).
    • Topology-aware placement: Opt-in support for pods in the same group to be co-located in the same topology.
    • All-or-nothing restart for failure handling: Opt-in support for all pods in the group to be recreated if one pod in the group failed or one container in the pods is restarted.

    Advanced Features: DisaggregatedSet

    • Disaggregated Architecture Support: Specifically designed for workloads where different phases (e.g., prefill and decode) run on separate infrastructure.
    • Coordinated N-Dimensional Rollouts: Updates multiple roles (2-10) in lockstep, preserving capacity ratios throughout the update process.
    • Unified Lifecycle Management: Manages multiple underlying LeaderWorkerSets as a single logical unit.
    • Automatic Service Orchestration: Automatically creates and manages headless services for each role to facilitate discovery and revision-aware routing.
    • Advanced Failure Handling: Coordinated drain and restart policies across all roles in the disaggregated set.

    Compatibility

    LeaderWorkerSet maintains support for the latest Kubernetes minor versions (currently tested on Kubernetes 1.34 through 1.37).

    Test Coverage

    Installation

    Officially supported and tested on Kubernetes 1.34 or newer.

    Read the installation guide to learn more.

    Examples

    Read the examples to learn more.

    Also discover adopters, integrations, and talks here.

    Community, discussion, contribution, and support

    Learn how to engage with the Kubernetes community on the community page.

    You can reach the maintainers of this project at:

    Code of conduct

    Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    ollama
    ollama·Go

    本地一键跑起 Kimi、DeepSeek、Qwen、Gemma 等大模型，省去环境配置和依赖折腾，适合快速做推理和原型验证。

    本地推理+3大模型推理服务入门
    186548+35819kv0.40.152026-11-10 01:49 UTC
    llama.cpp
    ggml-org·C++

    用 C/C++ 实现的大模型推理引擎，能在普通 CPU 和消费级显卡上跑 LLaMA 等模型，支持量化以降低显存占用。

    推理引擎+4大模型推理服务入门
    133932+33525kb122332026-11-10 17:13 UTC
    vllm
    vllm-project·Python

    面向大语言模型的高吞吐、低显存推理与服务引擎，通过 PagedAttention 等优化显著提升并发吞吐并降低显存占用，适合部署在线推理服务。

    推理引擎+4大模型推理服务进阶
    96680+18725kv0.33.0rc32026-11-10 17:01 UTC
    jan
    janhq·Rust

    在本地电脑上离线运行大模型的桌面客户端，是 ChatGPT 的开源替代品，无需联网即可对话，数据不出本机。

    本地推理+4大模型推理服务
    45655+1133.2kv0.8.82026-11-10 17:02 UTC