Mooncake 是支撑 Kimi 大模型服务的推理平台，通过 KVCache 分离式架构和 RDMA 高速传输，把预填充与解码拆开部署，提升长上下文场景下的吞吐和资源利用率。
为 LLM 推理提供虚拟化弹性 KV Cache，让多个推理任务动态共享 GPU 显存，减少显存碎片和浪费，提升 GPU 利用率。
把 LLM 推理中的 KV Cache 持久化到 DRAM、SSD 或 NFS 并跨请求复用，减少重复计算，加快推理速度，兼容昇腾 NPU 和 CUDA GPU。
CacheRoute 是一套 LLM 调度方案，让不同 LLM 系统之间灵活复用 KV cache，从而提升任务表现和整体推理效率。
…ch hot paths by @staryxchen in https://github.com/kvcache-ai/Mooncake/pull/3648 [TENT] Preserve per-request RDMA selection policy by @waizuichougou in https://github.com/
…ince the previous pre-release: https://github.com/kvcache-ai/Mooncake/compare/v0.3.14-pre-3...v0.3.14-pre-4
…elease. [Pre-Release workflow](https://github.com/kvcache-ai/Mooncake/actions/workflows/pre-release.yaml) · [Changes since v0.3.13.post1](https://github.com/kvcache-ai/Mo
…ormance and build - Page allocator in C++ ( 319), KVCacheBlock reuse ( 300), fewer device memory queries ( 456) - PyTorch stable ABI ( 497, 505); PyTorch = 2.10 and Pytho
… custom operators TRT Attention, TRT MoE, and TRT KVCacheUpdate - Removed deprecated APIs - IParser::supportsModel and IParser::parseWithWeightDescriptors
…et when TP is enabled ( 219) - Support flashinfer kvcache shape for vLLM ( 216) - Fix exceptions in vLLM prefix-cache-related APIs ( 212) - Export missing autopatch env (