给 vLLM 加一个免校准的 KV-cache 量化后端，只需一个开关就能把上下文拉长 3~5 倍，吞吐超过 FP16 且精度基本不掉，适合跑长上下文 Agent。
基于 llama.cpp 的推理分支，针对 Ampere 架构 Nvidia 显卡（尤其 RTX 3090/3090 Ti）深度优化，在长上下文场景下显著提升生成速度，并支持低显存配置跑 200K 以上上下文。
整理模型量化领域的论文、基准、综述和工具，覆盖低比特网络、大模型、多模态生成模型以及向量与格量化，方便做高效推理部署时查资料。
…. - TokenSpeed integrated Mooncake Store as an L3 KV cache backend, alongside Transfer Engine support for PD/EPD disaggregation. - TensorRT-LLM integrated Mooncake Store
…fix. Full changelog: https://github.com/XxMinor/mykvm/compare/v0.9.14...v0.9.15
…ecoder's 10 s restarts ran ahead of the audio the KV keeps for the rest of the conversation (236 frames ahead after a 20 s reply cut at 1.5 s), and each reply decoded aga
…screening. Local Inference & Cache Benchmarking - KV Cache Scaling Harness ( benchmark kv cache.py ) : Added reproducible benchmark harness measuring TTFT, generation thr
…Downloads - macOS: download the .dmg, then drag MyKVM to Applications. The .app.tar.gz file is for automatic updates only. - Windows: download the .exe installer. - Linux
…int: near-lossless quality + parity speed \ turbo kv 5b fast\ is a new variant that uses the same Variant F algorithm as \ turbo kv 5b\ (RHT + 32-level Lloyd-Max codebook