默认安装会引入 Knative 用于 Serverless 部署；标准 Kubernetes 安装更轻量但不支持金丝雀发布和按请求缩容到零；ModelMesh 需额外单独安装以支持高密度模型服务
kserve
kserveApache-2.0活跃维护在 Kubernetes 上统一部署生成式与预测式 AI 模型的推理平台，支持多框架、可扩展的分布式推理服务。
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
v0.21.0发布于 2026-09-25 17:04 UTC6,214star +15 / 7 天1,783fork同步于 2026-11-10 23:35 UTC
README 未给出可直接复制的最小安装命令，仅链接到标准 Kubernetes、Knative、ModelMesh 与快速安装文档 硬件要求按 README 说明
README 未说明具体显存或 GPU 型号要求，仅提到生成式 AI 支持 GPU 加速与针对大模型的优化内存管理。
在 Kubernetes 上统一部署生成式与预测式 AI 推理的标准化平台
在 Kubernetes 上统一部署生成式与预测式 AI 模型的推理平台，支持多框架、可扩展的分布式推理服务。
核心能力
- 支持 vLLM 和 llm-d 优化后端来服务大模型
- 提供 OpenAI 兼容推理协议便于集成
- 通过模型缓存与 KV 缓存卸载到 CPU/磁盘降低延迟
- 支持 TensorFlow、PyTorch、scikit-learn、XGBoost、ONNX 等多框架
- 支持金丝雀发布、推理流水线与 InferenceGraph 集成
容易踩的坑
内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
- v0.21.0最新正式版2026-09-25 · 2026-09-25 17:04 UTC
What's Changed
- refactor(e2e): auto-assign llminferenceservice & consolidate markers by @vivekk16 in https://github.com/kserve/kserve/pull/5837
- fix(e2e): make KEDA operator pod label configurable via env var by @vivekk16 in https://github.com/kserve/kserve/pull/5842
- feat(servingruntime): add resourceClaims support for DRA by @VedantMahabaleshwarkar in https://github.com/kserve/kserve/pull/5828
- feat: restructure CRD management for independent installation by @Jooho in https://github.com/kserve/kserve/pull/5843
- fix(llmisvc): handling stopped members in group by @bartoszmajsak in https://github.com/kserve/kserve/pull/5847
- fix(e2e): per-test namespace isolation for llmisvc e2e by @jlost in https://github.com/kserve/kserve/pull/5832
- fix(ci): apply HF env fixes to s3-init job and improve failure diagnostics by @lizzzcai in https://github.com/kserve/kserve/pull/5846
- fix(e2e): migrate vLLM CPU image from ECR to Docker Hub and cache in CI by @lizzzcai in https://github.com/kserve/kserve/pull/5845
- chore(llmisvc): migrate EndpointPickerConfig apiversion by @zdtsw in https://github.com/kserve/kserve/pull/5774
- feat(llmisvc): replace UDS tokenizer sidecar with vLLM render deployment by @vivekk16 in https://github.com/kserve/kserve/pull/5712
- feat(llmisvc): migrate WVA from VA CRD to annotation-based discovery by @vivekk16 in https://github.com/kserve/kserve/pull/5722
- test(e2e): add canary deployment lifecycle tests by @maskarb in https://github.com/kserve/kserve/pull/5833
- docs: add measured startup benchmarks for OCI model delivery paths by @kliukovkin in https://github.com/kserve/kserve/pull/5852
- test(llmisvc): e2e tests for canary deployment by @bartoszmajsak in https://github.com/kserve/kserve/pull/5849
- fix(e2e): use shared security context for canary lifecycle tests by @bartoszmajsak in https://github.com/kserve/kserve/pull/5858
- fix(e2e): add diagnostics to canary wait_ready by @bartoszmajsak in https://github.com/kserve/kserve/pull/5857
- fix(e2e): enable TLS for canary inference-sim when configured by @bartoszmajsak in https://github.com/kserve/kserve/pull/5859
- fix(e2e): inject default LLMISVC annotations via env var by @bartoszmajsak in https://github.com/kserve/kserve/pull/5860
- fix(e2e): preserve test namespaces on failure by @bartoszmajsak in https://github.com/kserve/kserve/pull/5861
- chore: bump dulwich to >=1.2.5 in kserve-storage by @mholder6 in https://github.com/kserve/kserve/pull/5840
- test(llmisvc): remove IstioShadowService fixture from envtests by @bartoszmajsak in https://github.com/kserve/kserve/pull/5870
- docs: update llmisvc samples from v1alpha1 to v1alpha2 by @aneeshkp in https://github.com/kserve/kserve/pull/5804
- fix(e2e): add gateway convergence wait to canary llmisvc test by @bartoszmajsak in https://github.com/kserve/kserve/pull/5876
- fix(e2e): poll for CanaryPredictorReady condition by @maskarb in https://github.com/kserve/kserve/pull/5884
- fix(samples): remove NixlConnector from prefix cache routing e2e sample by @vivekk16 in https://github.com/kserve/kserve/pull/5883
- fix(e2e): improve test stability with resource cleanup and polling by @mwaykole in https://github.com/kserve/kserve/pull/5863
- feat(llmisvc): migrate deprecated flag to metrics-data-source plugin by @zdtsw in https://github.com/kserve/kserve/pull/5877
- ci(manifests): add kustomize validation workflow by @bartoszmajsak in https://github.com/kserve/kserve/pull/5878
- fix(llmisvc): skip v1alpha2 InfPool reconcile when CRD absent by @bartoszmajsak in https://github.com/kserve/kserve/pull/5890
- fix(e2e): update stale apiVersion in fixture by @jlost in https://github.com/kserve/kserve/pull/5872
- fix(e2e): remove duplicate body field in ResponseRecord by @bartoszmajsak in https://github.com/kserve/kserve/pull/5865
- chore: add license headers to e2e test files by @bartoszmajsak in https://github.com/kserve/kserve/pull/5864
- feat: add pytest --collect-only for e2e tests to precommit by @jlost in https://
- v0.21.0-rc1pre-release2026-09-23 · 2026-09-23 01:22 UTC
查看更新说明
What's Changed
- refactor(e2e): auto-assign llminferenceservice & consolidate markers by @vivekk16 in https://github.com/kserve/kserve/pull/5837
- fix(e2e): make KEDA operator pod label configurable via env var by @vivekk16 in https://github.com/kserve/kserve/pull/5842
- feat(servingruntime): add resourceClaims support for DRA by @VedantMahabaleshwarkar in https://github.com/kserve/kserve/pull/5828
- feat: restructure CRD management for independent installation by @Jooho in https://github.com/kserve/kserve/pull/5843
- fix(llmisvc): handling stopped members in group by @bartoszmajsak in https://github.com/kserve/kserve/pull/5847
- fix(e2e): per-test namespace isolation for llmisvc e2e by @jlost in https://github.com/kserve/kserve/pull/5832
- fix(ci): apply HF env fixes to s3-init job and improve failure diagnostics by @lizzzcai in https://github.com/kserve/kserve/pull/5846
- fix(e2e): migrate vLLM CPU image from ECR to Docker Hub and cache in CI by @lizzzcai in https://github.com/kserve/kserve/pull/5845
- chore(llmisvc): migrate EndpointPickerConfig apiversion by @zdtsw in https://github.com/kserve/kserve/pull/5774
- feat(llmisvc): replace UDS tokenizer sidecar with vLLM render deployment by @vivekk16 in https://github.com/kserve/kserve/pull/5712
- feat(llmisvc): migrate WVA from VA CRD to annotation-based discovery by @vivekk16 in https://github.com/kserve/kserve/pull/5722
- test(e2e): add canary deployment lifecycle tests by @maskarb in https://github.com/kserve/kserve/pull/5833
- docs: add measured startup benchmarks for OCI model delivery paths by @kliukovkin in https://github.com/kserve/kserve/pull/5852
- test(llmisvc): e2e tests for canary deployment by @bartoszmajsak in https://github.com/kserve/kserve/pull/5849
- fix(e2e): use shared security context for canary lifecycle tests by @bartoszmajsak in https://github.com/kserve/kserve/pull/5858
- fix(e2e): add diagnostics to canary wait_ready by @bartoszmajsak in https://github.com/kserve/kserve/pull/5857
- fix(e2e): enable TLS for canary inference-sim when configured by @bartoszmajsak in https://github.com/kserve/kserve/pull/5859
- fix(e2e): inject default LLMISVC annotations via env var by @bartoszmajsak in https://github.com/kserve/kserve/pull/5860
- fix(e2e): preserve test namespaces on failure by @bartoszmajsak in https://github.com/kserve/kserve/pull/5861
- chore: bump dulwich to >=1.2.5 in kserve-storage by @mholder6 in https://github.com/kserve/kserve/pull/5840
- test(llmisvc): remove IstioShadowService fixture from envtests by @bartoszmajsak in https://github.com/kserve/kserve/pull/5870
- docs: update llmisvc samples from v1alpha1 to v1alpha2 by @aneeshkp in https://github.com/kserve/kserve/pull/5804
- fix(e2e): add gateway convergence wait to canary llmisvc test by @bartoszmajsak in https://github.com/kserve/kserve/pull/5876
- fix(e2e): poll for CanaryPredictorReady condition by @maskarb in https://github.com/kserve/kserve/pull/5884
- fix(samples): remove NixlConnector from prefix cache routing e2e sample by @vivekk16 in https://github.com/kserve/kserve/pull/5883
- fix(e2e): improve test stability with resource cleanup and polling by @mwaykole in https://github.com/kserve/kserve/pull/5863
- feat(llmisvc): migrate deprecated flag to metrics-data-source plugin by @zdtsw in https://github.com/kserve/kserve/pull/5877
- ci(manifests): add kustomize validation workflow by @bartoszmajsak in https://github.com/kserve/kserve/pull/5878
- fix(llmisvc): skip v1alpha2 InfPool reconcile when CRD absent by @bartoszmajsak in https://github.com/kserve/kserve/pull/5890
- fix(e2e): update stale apiVersion in fixture by @jlost in https://github.com/kserve/kserve/pull/5872
- fix(e2e): remove duplicate body field in ResponseRecord by @bartoszmajsak in https://github.com/kserve/kserve/pull/5865
- chore: add license headers to e2e test files by @bartoszmajsak in https://github.com/kserve/kserve/pull/5864
- feat: add pytest --collect-only for e2e tests to precommit by @jlost in https://
- v0.21.0-rc0pre-release2026-09-10 · 2026-09-10 17:44 UTC
查看更新说明
What's Changed
- refactor(e2e): auto-assign llminferenceservice & consolidate markers by @vivekk16 in https://github.com/kserve/kserve/pull/5837
- fix(e2e): make KEDA operator pod label configurable via env var by @vivekk16 in https://github.com/kserve/kserve/pull/5842
- feat(servingruntime): add resourceClaims support for DRA by @VedantMahabaleshwarkar in https://github.com/kserve/kserve/pull/5828
- feat: restructure CRD management for independent installation by @Jooho in https://github.com/kserve/kserve/pull/5843
- fix(llmisvc): handling stopped members in group by @bartoszmajsak in https://github.com/kserve/kserve/pull/5847
- fix(e2e): per-test namespace isolation for llmisvc e2e by @jlost in https://github.com/kserve/kserve/pull/5832
- fix(ci): apply HF env fixes to s3-init job and improve failure diagnostics by @lizzzcai in https://github.com/kserve/kserve/pull/5846
- fix(e2e): migrate vLLM CPU image from ECR to Docker Hub and cache in CI by @lizzzcai in https://github.com/kserve/kserve/pull/5845
- chore(llmisvc): migrate EndpointPickerConfig apiversion by @zdtsw in https://github.com/kserve/kserve/pull/5774
- feat(llmisvc): replace UDS tokenizer sidecar with vLLM render deployment by @vivekk16 in https://github.com/kserve/kserve/pull/5712
- feat(llmisvc): migrate WVA from VA CRD to annotation-based discovery by @vivekk16 in https://github.com/kserve/kserve/pull/5722
- test(e2e): add canary deployment lifecycle tests by @maskarb in https://github.com/kserve/kserve/pull/5833
- docs: add measured startup benchmarks for OCI model delivery paths by @kliukovkin in https://github.com/kserve/kserve/pull/5852
- test(llmisvc): e2e tests for canary deployment by @bartoszmajsak in https://github.com/kserve/kserve/pull/5849
- fix(e2e): use shared security context for canary lifecycle tests by @bartoszmajsak in https://github.com/kserve/kserve/pull/5858
- fix(e2e): add diagnostics to canary wait_ready by @bartoszmajsak in https://github.com/kserve/kserve/pull/5857
- fix(e2e): enable TLS for canary inference-sim when configured by @bartoszmajsak in https://github.com/kserve/kserve/pull/5859
- fix(e2e): inject default LLMISVC annotations via env var by @bartoszmajsak in https://github.com/kserve/kserve/pull/5860
- fix(e2e): preserve test namespaces on failure by @bartoszmajsak in https://github.com/kserve/kserve/pull/5861
- chore: bump dulwich to >=1.2.5 in kserve-storage by @mholder6 in https://github.com/kserve/kserve/pull/5840
- test(llmisvc): remove IstioShadowService fixture from envtests by @bartoszmajsak in https://github.com/kserve/kserve/pull/5870
- docs: update llmisvc samples from v1alpha1 to v1alpha2 by @aneeshkp in https://github.com/kserve/kserve/pull/5804
- fix(e2e): add gateway convergence wait to canary llmisvc test by @bartoszmajsak in https://github.com/kserve/kserve/pull/5876
- fix(e2e): poll for CanaryPredictorReady condition by @maskarb in https://github.com/kserve/kserve/pull/5884
- fix(samples): remove NixlConnector from prefix cache routing e2e sample by @vivekk16 in https://github.com/kserve/kserve/pull/5883
- fix(e2e): improve test stability with resource cleanup and polling by @mwaykole in https://github.com/kserve/kserve/pull/5863
- feat(llmisvc): migrate deprecated flag to metrics-data-source plugin by @zdtsw in https://github.com/kserve/kserve/pull/5877
- ci(manifests): add kustomize validation workflow by @bartoszmajsak in https://github.com/kserve/kserve/pull/5878
- fix(llmisvc): skip v1alpha2 InfPool reconcile when CRD absent by @bartoszmajsak in https://github.com/kserve/kserve/pull/5890
- fix(e2e): update stale apiVersion in fixture by @jlost in https://github.com/kserve/kserve/pull/5872
- fix(e2e): remove duplicate body field in ResponseRecord by @bartoszmajsak in https://github.com/kserve/kserve/pull/5865
- chore: add license headers to e2e test files by @bartoszmajsak in https://github.com/kserve/kserve/pull/5864
- feat: add pytest --collect-only for e2e tests to precommit by @jlost in https://
- v0.20.02026-08-06 · 2026-08-06 15:07 UTC
查看更新说明
What's Changed
- fix(ci): add debug messages to print 403 issues by @spolti in https://github.com/kserve/kserve/pull/5580
- fix(ci): improve automerge error handling and bump github-script to v9 by @spolti in https://github.com/kserve/kserve/pull/5593
- docs(llmisvc): add godoc to all LLMInferenceService conditions by @bartoszmajsak in https://github.com/kserve/kserve/pull/5586
- feat(llmisvc): model-based routing gates and models in status by @pierDipi in https://github.com/kserve/kserve/pull/5579
- fix(llmisvc): remove duplicate yaml.Unmarshal in loadConfig test helper by @skbhudolia12 in https://github.com/kserve/kserve/pull/5548
- feat(llmisvc): cache inferenceservice-config and watch for changes by @KaveeshKhattar in https://github.com/kserve/kserve/pull/5573
- fix: prevent Value/ValueFrom conflict when merging ClusterStorageContainer by @richiefrost in https://github.com/kserve/kserve/pull/5541
- fix(ci): harden E2E test infrastructure to reduce flakiness by @vivekk16 in https://github.com/kserve/kserve/pull/5574
- refactor(llmisvc): no pagination in enqueue handlers by @bartoszmajsak in https://github.com/kserve/kserve/pull/5584
- fix(e2e): replace removed VLLM_USE_V1 with VLLM_ENABLE_V1_MULTIPROCESSING by @jlost in https://github.com/kserve/kserve/pull/5613
- refactor(llmisvc): pass reconciler to extendControllerSetup hook by @bartoszmajsak in https://github.com/kserve/kserve/pull/5610
- fix(hack): resolve ShellCheck findings in shell scripts by @bartoszmajsak in https://github.com/kserve/kserve/pull/5618
- feat(ci): cache opt-125m in SeaweedFS by @jlost in https://github.com/kserve/kserve/pull/5566
- build: narrow controller-gen RBAC path for v1beta1 by @bartoszmajsak in https://github.com/kserve/kserve/pull/5619
- feat(kserve): forward authorization header from transformer to predictor by @spolti in https://github.com/kserve/kserve/pull/5567
- feat: support multiple OCI sources in storageUris by @brucearctor in https://github.com/kserve/kserve/pull/5470
- fix(llmisvc):
config/llmisvcconfig/**to E2E test trigger paths by @pierDipi in https://github.com/kserve/kserve/pull/5622 - e2e: reduce verbose payload logging in e2e tests by @maskarb in https://github.com/kserve/kserve/pull/5486
- fix(ci): stabilize HuggingFace model downloads in e2e tests by @andresllh in https://github.com/kserve/kserve/pull/5510
- fix(llmisvc): wire enableLLMInferenceServiceTLS in reconcile pipeline by @madhugoutham in https://github.com/kserve/kserve/pull/5525
- ci: replace rerun-actions with gh-based targeted reruns by @bartoszmajsak in https://github.com/kserve/kserve/pull/5621
- fix(llmisvc): export all HCAs in NVSHMEM_HCA_LIST and UCX_NET_DEVICES by @dagrayvid in https://github.com/kserve/kserve/pull/5603
- fix(ci): retry corrupted docker image artifact downloads by @pierDipi in https://github.com/kserve/kserve/pull/5650
- fix: bump starlette to >=1.0.1 (CVE-2026-48710) and set Content-Type … by @spolti in https://github.com/kserve/kserve/pull/5632
- ci: add workflow to auto-assign reviewers on new PRs by @spolti in https://github.com/kserve/kserve/pull/5491
- test(llmisvc): add e2e tests for scheduler v0.6 → v0.7 migration by @vivekk16 in https://github.com/kserve/kserve/pull/5564
- fix: remove PR filter by @cjohannsen-cloudera in https://github.com/kserve/kserve/pull/5656
- feat(llmisvc): add PVC storage e2e tests for LLMInferenceService by @pierDipi in https://github.com/kserve/kserve/pull/5623
- fix: add storageContainerName field for explicit CSC selection by @RunlinXiao in https://github.com/kserve/kserve/pull/5314
- feat(isvc): add platform hooks for service customization by @bartoszmajsak in https://github.com/kserve/kserve/pull/5617
- chore(isvc): reconcile svc before deployment in raw reconciler by @bartoszmajsak in https://github.com/kserve/kserve/pull/5630
- docs: remove Envoy references from LLMISVC godocs by @danehans in https://github.com/kserve/kserve/pull/5475
- fix(ci): centralize hfserver XET/tokio env patch for he
- v0.20.0-rc1pre-release2026-08-03 · 2026-08-03 23:59 UTC
查看更新说明
What's Changed since v0.20.0-rc0
- feat(llmisvc): drop old flag to support new TLS flag for disagg-sidecar by @madhugoutham in https://github.com/kserve/kserve/pull/5875
- fix(llmisvc): escape quotes in generated --kv-transfer-config by @Chung-Hsuan-Tsai in https://github.com/kserve/kserve/pull/5880
- fix(llmisvc): skip v1alpha2 InfPool reconcile when CRD absent by @bartoszmajsak in https://github.com/kserve/kserve/pull/5890
- release: prepare release v0.20.0-rc1 by @cjohannsen-cloudera in https://github.com/kserve/kserve/pull/5894
Full Changelog: https://github.com/kserve/kserve/compare/v0.20.0-rc0...v0.20.0-rc1
用户点评
登录用户的真实使用记录，可按版本和硬件筛选共 0 条
还没有人点评。第一个说出真实体验的人最有价值。
KServe
KServe is a standardized distributed generative and predictive AI inference platform for scalable, multi-framework deployment on Kubernetes.
KServe is being used by many organizations and is a Cloud Native Computing Foundation (CNCF) incubating project.
For more details, visit the KServe website.
Why KServe?
Single platform that unifies Generative and Predictive AI inference on Kubernetes. Simple enough for quick deployments, yet powerful enough to handle enterprise-scale AI workloads with advanced features.
Features
Generative AI
- 🧮 Optimized Backends: Support for vLLM and llm-d for optimized performance for serving LLMs
- 📌 Standardization: OpenAI-compatible inference protocol for seamless integration with LLMs
- 🚅 GPU Acceleration: High-performance serving with GPU support and optimized memory management for large models
- 💾 Model Caching: Intelligent model caching to reduce loading times and improve response latency for frequently used models
- 🗂️ KV Cache Offloading: Advanced memory management with KV cache offloading to CPU/disk for handling longer sequences efficiently
- 📈 Autoscaling: Request-based autoscaling capabilities optimized for generative workload patterns
- 🔧 Hugging Face Ready: Native support for Hugging Face models with streamlined deployment workflows
Predictive AI
- 🧮 Multi-Framework: Support for TensorFlow, PyTorch, scikit-learn, XGBoost, ONNX, and more
- 🔀 Intelligent Routing: Seamless request routing between predictor, transformer, and explainer components with automatic traffic management
- 🔄 Advanced Deployments: Canary rollouts, inference pipelines, and ensembles with InferenceGraph
- ⚡ Autoscaling: Request-based autoscaling with scale-to-zero for predictive workloads
- 🔍 Model Explainability: Built-in support for model explanations and feature attribution to understand prediction reasoning
- 📊 Advanced Monitoring: Enables payload logging, outlier detection, adversarial detection, and drift detection
- 💰 Cost Efficient: Scale-to-zero on expensive resources when not in use, reducing infrastructure costs
Learn More
To learn more about KServe, how to use various supported features, and how to participate in the KServe community, please follow the KServe website documentation. Additionally, we have compiled a list of presentations and demos to dive through various details.
:hammer_and_wrench: Installation
Standalone Installation
- Standard Kubernetes Installation: Compared to Serverless Installation, this is a more lightweight installation. However, this option does not support canary deployment and request based autoscaling with scale-to-zero.
- Knative Installation: KServe by default installs Knative for serverless deployment for InferenceService.
- ModelMesh Installation: You can optionally install ModelMesh to enable high-scale, high-density and frequently-changing model serving use cases.
- Quick Installation: Install KServe on your local machine.
Kubeflow Installation
KServe is an important addon component of Kubeflow, please learn more from the Kubeflow KServe documentation. Check out the following guides for running on AWS or on OpenShift Container Platform.
:flight_departure: Create your first InferenceService
:bulb: Roadmap
:blue_book: InferenceService API Reference
:toolbox: Developer Guide
:writing_hand: Contributor Guide
:handshake: Adopters
Contributors
Thanks to all of our amazing contributors!
awesome-production-machine-learning
EthicalML
一份精选的机器学习生产化开源工具清单，覆盖模型部署、监控、版本管理和规模化扩展，帮你快速找到 MLOps 各环节的趁手工具。
MLOps+4MLOps 平台
21092+132.6krelease-2026-11-012026-11-08 10:41 UTC
agent-lightning
microsoft·Python
给 AI Agent 做强化学习训练和调优的框架，把 Agent 的推理轨迹变成训练信号，让智能体越跑越强。
Agent 训练+4MLOps 平台
18938+231.7kv1.0.32026-11-05 03:09 UTC
mlops-zoomcamp
DataTalksClub·Jupyter Notebook
DataTalks.Club 出品的免费 MLOps 实战课程，覆盖模型部署、监控和工作流编排，帮你把机器学习模型从实验推到生产环境。
MLOps课程+4MLOps 平台
15545+143.1k2026-10-15 13:49 UTC
kedro
kedro-org·Python
Kedro 是一个面向生产环境的数据科学工具箱，用软件工程最佳实践帮你搭建可复现、易维护、模块化的数据工程与数据科学流水线。
数据流水线+4MLOps 平台
11051+51.1k1.8.02026-11-10 11:32 UTC