单张 4GB 显存显卡就能跑 70B 甚至 2.8T 参数大模型的分层推理方案,通过逐层加载权重把显存需求压到极低,适合本地低配设备部署大模型。
… Gemma 3n ([ 1650](https://github.com/huggingface/optimum-intel/pull/1650) by @anatyrova) - Qwen3-Omni-MOE ([ 1700](https://github.com/huggingface/optimum-intel/pull/1700
…lyasMoutawwakil in https://github.com/huggingface/optimum/pull/2395 Remove deprecated INC and IPEX from documentation by @echarlaix in https://github.com/huggingface/opti
…a2 by @rkazants in https://github.com/huggingface/optimum-intel/pull/1354 Add OpenVINO support for BitNet by @rkazants in https://github.com/huggingface/optimum-intel/pul
…al breaking changes 1. ONNX integration moved to [Optimum ONNX](https://github.com/huggingface/optimum-onnx) ONNX export and ONNX Runtime inference related integrations w
…lyasMoutawwakil in https://github.com/huggingface/optimum/pull/2326 Batched inference support across all decoders by @IlyasMoutawwakil in https://github.com/huggingface/o
…els by @dacorvo in https://github.com/huggingface/optimum-quanto/pull/222 Rebuild extension when pytorch is updated by @dacorvo in https://github.com/huggingface/optimum-