登录

optimum-intel

huggingfaceApache-2.0活跃维护

把 Hugging Face 的 Transformers、Diffusers 等模型接到 Intel 的 OpenVINO 等优化工具上，加速 CPU/GPU 推理，覆盖 LLM、语音识别、语音合成和多模态模型。

🤗 Optimum Intel: Accelerate inference with Intel optimization tools

v2.2.0发布于 2026-09-17 13:11 UTC627star +2 / 7 天296fork同步于 2026-11-10 11:57 UTC
推理加速OpenVINO模型优化语音推理Intel
对比 GitHub 官网asrdiffusersllmomniopenvino

怎么上手

由 README 提炼，执行前请确认
原文 
pip install -U optimum-intel
硬件要求按 README 说明

面向 Intel CPU、GPU 及专用深度学习推理加速器，具体支持设备列表见 OpenVINO 文档；README 未给出具体显存要求。

下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

把 Transformers 等模型导出成 OpenVINO IR，在 Intel CPU/GPU 上加速推理

把 Hugging Face 的 Transformers、Diffusers 等模型接到 Intel 的 OpenVINO 等优化工具上，加速 CPU/GPU 推理，覆盖 LLM、语音识别、语音合成和多模态模型。

核心能力

  • 用 optimum-cli 把 Hub 模型导出为 OpenVINO IR 格式
  • 提供 OVModelForXxx 类替换 AutoModelForXxx 直接跑推理
  • 支持训练后静态量化，可指定 dtype 与校准数据集
  • 对接 Transformers、Diffusers、Sentence Transformers 和 timm

容易踩的坑

推理前需先用 optimum-cli 把模型导出为 OpenVINO IR，不能直接加载原始权重；量化需要额外准备校准数据集；导出模型也可改用 OpenVINO GenAI 框架推理。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本v2.2.0
全部 release
  1. v2.2.0最新正式版2026-09-17 · 2026-09-17 13:11 UTC

    New Model Support

    • Mistral 3 (#1627 by @kyoui-dev)
    • DeepSeek-OCR-2 (#1793 by @openvino-agent)
    • Meta Muse Glimmer (#1924 by @rkazants)
    • Ouro (#1783 by @openvino-dev-samples)
    • Fun-ASR (#1801 by @openvino-agent)
    • LTX2 text-to-audio-video (#1733 by @anatyrova) and image-to-video (#1885 by @goyaladitya05)
    • LTX-Video image-to-video (#1762 by @goyaladitya05)
    • Qwen-Image (#1790 by @openvino-agent)
    • Z-Image (#1946 by @openvino-agent)
    • DFlash draft models for speculative decoding (#1756 by @ofirzaf), including VLM support (#1926 by @ofirzaf, #1971 by @rkazants)
    • Multi-Token Prediction heads for Qwen3.5, Qwen3.6 (including MoE flavors) and Qwen3.8-27B (#1814 by @openvino-agent, #1971 by @rkazants)
    • Video input for Gemma 4 and Gemma 4 Unified (#1951 by @yatarkan)

    Improvements & fixes

    • Added default int4 compression configs for gemma-4-12B-it (#1952 by @anzr299) and Muse-Glimmer-30B (#1929 by @andreyanufr)
    • Fixed Gemma 4 Unified vision embedding fp16 overflow on GPU by setting an activation scale factor (#1973 by @rkazants)
    • Fixed chat template application for Gemma 4 Unified (#1895 by @popovaan)
    • Fixed Mamba 2 selective SSM representation for Granite 4.0 hybrid (#1789 by @openvino-agent)
    • Optimized Qwen3-Omni-MoE Talker MoE export (#1880 by @sgonorov)
    • Fixed the inputs_embeds input of the Qwen3-VL Eagle draft model (#1932 by @popovaan)
    • Fixed Fun-ASR model type (#1925 by @popovaan) and added export and auto-compression tests (#1874 by @LauraGPT)
    • Fixed calibration dataset preparation for FLUX.2-klein (#1923 by @andrey-churkin)
    • Added the missing --group-size-fallback argument to the export CLI (#1845 by @andrey-churkin)
    • Removed the _LTX2AttnProcessorWithEps patch for LTX2 (#1928 by @anatyrova)
    • Removed the patch_16bit_model flag from the export path (#1930 by @anatyrova)
    • Bumped the minimum NNCF version to v3.3 and unpinned safetensors (#1916 by @andrey-churkin, #1918 by @echarlaix)

    Other Changes

    • Added an OpenVINO export space to the documentation (#1882 by @echarlaix)
    • Added agent skills for optimum and tiny test model creation (#1848 by @Mohamed-Ashraf273)
    • Removed unneeded transformers version checks in tests (#1933 by @popovaan)
    • CI: run OpenVINO tests on release-branch pull requests (#1892), restore Qwen3.5 and Qwen3.5-MoE in the preview workflow (#1891), sync built docs to the hf-doc-build bucket (#1898 by @echarlaix)

    New Contributors

    Compatible transformers version

    Compatible with transformers>=4.51,<5.6

    Recommended versions

    • OpenVINO: v2026.4
    • OpenVINO GenAI: v2026.4
    • NNCF: v3.4

    Full Changelog

    https://github.com/huggingface/optimum-intel/compare/v2.1.0...v2.2.0

  2. v2.1.0-muse-glimmerv2.1.0-Muse-Glimmer2026-08-26 · 2026-08-26 10:52 UTC
    查看更新说明

    New Model Support

    Compatible transformers version

    • Muse Glimmer: Requires transformers==5.15
    • Everything else: Compatible with transformers>=4.51,<5.6

    Recommended versions

    • OpenVINO: v2026.3.1
    • OpenVINO GenAI: v2026.3.1
    • OpenVINO Tokenizers: v2026.3.1
    • NNCF: v3.3.0
  3. v2.1.02026-08-05 · 2026-08-05 14:15 UTC
    查看更新说明

    New Model Support

    • Gemma 3n (#1650 by @anatyrova)
    • Qwen3-Omni-MOE (#1700 by @sgonorov)
    • SmolLM3 (#1715 by @AshutoshSinghIntel)
    • Qwen3-VL-Embedding (#1723 by @mlukasze)
    • Gemma 4 Unified (#1770 by @rkazants)
    • FLUX.2 (#1809 by @rkazants)

    Improvements & fixes

    • Transformers v5.1-5.5 support (#1684 by @echarlaix)
    • Fix Kokoro voice conversion from local model directory (#1803 by @mmikolajcz)
    • Extended OpenVINO transformation verification tests for MoE architectures (#1776 by @anatyrova)
    • Unpatch 16-bit models previously patched with patch_16bit_model after conversion (#1811 by @anatyrova)
    • Set dynamic sequence length for FLUX.2 (#1846 by @rkazants)
    • Deprecated the broken contextual calibration dataset, replaced with textvqa (#1849 by @echarlaix)
    • Fixed passing of token_type_ids for Gemma3 (#1881 by @popovaan)
    • Fixed export for encoder-decoder models with unnamed tensors (#1857 by @echarlaix)
    • Fixed long audio inference for Qwen3-ASR (#1782 by @popovaan)
    • Fixed ssm states handling for Mamba and Falcon-Mamba (#1779 by @echarlaix)
    • Pinned huggingface_hub<1.22 to fix offline loading (#1855 by @echarlaix)
    • Use base transformers version for version comparisons (#1798 by @AlexanderDokuchaev)

    Other Changes

    New Contributors

    • @anatyrova made their first contribution in #1650
    • @AshutoshSinghIntel made their first contribution in #1715
    • @mlukasze made their first contribution in #1723

    Full Changelog

    https://github.com/huggingface/optimum-intel/compare/v2.0.0...v2.1.0

    Compatible transformers version

    Compatible with transformers>=4.51,<5.6

    NPU-specific requirements:

    • Whisper - need transformers v5.0
    • LFM2 - need transformers v5.0

    Maximum supported transformers version by model architectures can be found in the table below (models not listed support up to transformers v5.6)

    Model architectureMax transformers version
    AquilaM4.57.6
    Arctic4.53.3
    Baichuan4.57.6
    Bitnet4.57.6
    ChatGLM24.55.4
    DBRX4.57.6
    Data2VecText4.57.6
    Deci4.57.6
    Deepseek4.53.3
    Exaone4.57.6
    Exaone44.5
  4. v2.0.02026-06-10 · 2026-06-10 12:59 UTC
    查看更新说明

    Breaking Changes

    • Intel Neural Compressor (INC) and Intel Extension for PyTorch (IPEX) integrations have been fully removed (https://github.com/huggingface/optimum-intel/pull/1687). Both were deprecated in v1.27.0. Users relying on these should stay on v1.27
    • ONNX dependency was removed from package requirements (#1753)
    • OpenVINO and NNCF are now installed by default, the [openvino] and [nncf] extras are now deprecated (#1602)

    New Model Support

    • Arcee Trinity (AFMoE) (#1569)
    • Qwen3VL (#1551)
    • Qwen3-next (hybrid SSM/attention) (#1523)
    • Qwen3.5, Qwen3.5-MoE, Qwen3.6 (#1689)
    • Gemma 4 (#1688)
    • Eagle3: Speculative decoding draft model support (#1588)
    • LFM2-MoE (#1691)
    • Kokoro TTS (#1653)
    • Qwen3-ASR (#1677)
    • CohereLabs/tiny-aya-base (Command-R family) (#1623)
    • HY-MT1.5-1.8B (#1621)
    • VideoChat (#1637)

    Quantization & Compression

    • Extended dataset options for calibration: Datasets can now be specified with parameters, e.g. wikitext2:seq_len=128 (#1564)
    • Default 8-bit quantization configs with configurable dynamic quantization group size (#1570)
    • NNCF CB4 mode renamed to cb4_f8e4m3 for newer NNCF versions (#1597)
    • Data-Aware AWQ for Qwen3-30B added to configuration (#1620)
    • Fix quantized model save path: Immediate save after quantization now writes to the correct path (#1576)
    • Fix per_layer_inputs value error during quantization (#1714)
    • Fix calibration data collection (#1778)

    Improvements

    • Transformers v5 compatibility (#1589)
    • Hybrid attention models: past_key_values in attention_mask now supported for stateful inference (#1641)
    • beam_idx connected to Linear Attention Layers (CausalConv1D, SSM, GDN) for correct beam search with recurrent models (#1619)
    • Fix long-context inference for Phi-3.5 and Phi-4 (#1744)
    • Fix SpeechT5 dynamic batch inference (#1664)
    • Fix MoE patching to enable ConvertTiledMoeBlockToGatherMatmuls transformation (#1741)
    • Improved numpy input handling for model inputs with mixed types (#1646)
    • Fix task inference for Phi-4-multimodal-instruct (#1610)

    New Contributors

    What's Changed

    Full Changelog: https://github.com/huggingface/optimum-intel/compare/v1.27.0...v2.0.0

    Compatible with transformers>=v4.45,<v5.1

  5. v1.27.0v1.27.0: Transformers 4.56/4.57, Zamba2, BitNet, Granite-4, Exaone-4, LFM22025-12-23 · 2025-12-23 16:08 UTC
    查看更新说明

    ️🏗 ​New architectures support

    🧹 Deprecations

    🔧 Enhancements & Fixes

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    Optimum Intel

    🤗 Optimum Intel is the interface between the 🤗 Transformers, Diffusers, Sentence Transformers and timm libraries and the different tools and libraries provided by OpenVINO to accelerate end-to-end pipelines on Intel architectures.

    OpenVINO is an open-source toolkit that enables high performance inference capabilities for Intel CPUs, GPUs, and special DL inference accelerators (see the full list of supported devices). It is supplied with a set of tools to optimize your models with compression techniques such as quantization, pruning and knowledge distillation. Optimum Intel provides a simple interface to optimize your model, convert it to the OpenVINO Intermediate Representation (IR) format and run inference using OpenVINO Runtime.

    Installation

    To install the latest release of 🤗 Optimum Intel with the corresponding required dependencies, you can use pip as follows:

    pip install -U optimum-intel

    Optimum Intel is a fast-moving project with regular additions of new model support, so you may want to install from source with the following command:

    pip install optimum-intel@git+https://github.com/huggingface/optimum-intel.git

    Export:

    To export your model to OpenVINO IR format, use the optimum-cli tool. Below is an example of exporting TinyLlama/TinyLlama_v1.1 model:

    optimum-cli export openvino --model TinyLlama/TinyLlama_v1.1 ov_TinyLlama_v1_1

    To export a model hosted on the Hub you can use our space. After conversion, a repository will be pushed under your namespace, this repository can be either public or private.

    Additional information on exporting models is available in the documentation.

    Inference:

    To load an exported model and run inference using Optimum Intel, use the corresponding OVModelForXxx class instead of AutoModelForXxx:

    from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer, pipeline

model_id = "ov_TinyLlama_v1_1"
model = OVModelForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
results = pipe("Hey, how are you doing today?", max_new_tokens=100)

    For more details on Optimum Intel inference, refer to the documentation.

    Note: Alternatively, an exported model can also be inferred using OpenVINO GenAI framework, that provides optimized execution methods for highly performant Generative AI.

    Quantization:

    Post-training static quantization can also be applied. Here is an example on how to apply static quantization on a Whisper model using the LibriSpeech dataset for the calibration step.

    from optimum.intel import OVModelForSpeechSeq2Seq, OVQuantizationConfig

model_id = "openai/whisper-tiny"
q_config = OVQuantizationConfig(dtype="int8", dataset="librispeech", num_samples=50)
q_model = OVModelForSpeechSeq2Seq.from_pretrained(model_id, quantization_config=q_config)

# The directory where the quantized model will be saved
save_dir = "nncf_results"
q_model.save_pretrained(save_dir)

    You can find more information in the documentation.

    Running the examples

    Check out the notebooks directory to see how 🤗 Optimum Intel can be used to optimize models and accelerate inference.

    Do not forget to install requirements for every example:

    cd <example-folder>
pip install -r requirements.txt

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    VoiceStudio
    debpalash·Python

    本地运行的 ElevenLabs 开源替代品，支持声音克隆、语音设计、视频配音、听写、转写和有声书制作，覆盖 646 种语言。

    语音克隆+4语音推理
    106722+359812kv0.5.72026-11-10 10:53 UTC
    voicebox
    jamiepine·Python

    开源 AI 语音工作室，支持声音克隆、语音听写和语音生成，基于 Qwen3-TTS 与 Whisper，可在 CUDA 或 MLX 上本地运行。

    语音克隆+4语音推理
    60349+5967.6kv0.5.02026-10-07 01:17 UTC
    whisper.cpp
    ggml-org·C++

    OpenAI Whisper 语音识别模型的 C/C++ 移植版，无需 Python 和 GPU 依赖，在普通 CPU 上就能跑语音转文字。

    语音识别+4语音推理
    55060+736.3kv1.9.52026-11-10 08:39 UTC
    faster-whisper
    SYSTRAN·Python

    基于 CTranslate2 重实现的 Whisper 语音转文字推理引擎，通过量化与高效推理大幅提速并降低显存占用，适合本地部署转录。

    语音识别+4语音推理
    26639+672.2kv1.2.12026-11-10 10:13 UTC