openvino

openvinotoolkitApache-2.0活跃维护

把训练好的深度学习模型优化并部署到 Intel CPU、GPU 等硬件上做推理，支持视觉、LLM 和生成式模型，省去自己折腾底层加速的麻烦。

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

2026.4.1发布于 2026-10-01 13:45 UTC11,142star +14 / 7 天3,546fork同步于 2026-11-10 11:29 UTC

推理引擎模型部署模型优化 Intel硬件加速生成式模型

对比 GitHub 官网 ai computer-vision deep-learning deploy-ai diffusion-models

怎么上手

由 README 提炼，执行前请确认

原文

pip install -U openvino

硬件要求按 README 说明

支持 CPU（x86、ARM）、Intel 集成与独立 GPU、Intel NPU 推理；README 未给出具体显存或型号门槛，需查阅系统要求与支持设备文档。

下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼

没人工点评？写下你的经验

英特尔开源推理优化部署工具包，把训练好的模型转成高效推理并跑在 CPU、GPU、NPU 上

把训练好的深度学习模型优化并部署到 Intel CPU、GPU 等硬件上做推理，支持视觉、LLM 和生成式模型，省去自己折腾底层加速的麻烦。

核心能力

用 convert_model 直接转换 PyTorch、TensorFlow、ONNX 等模型
用 compile_model 按设备编译模型并执行推理
支持 CPU（x86、ARM）、Intel 集显/独显和 Intel NPU 推理
提供 C++、Python、C、NodeJS 多语言 API
配套 NNCF 做量化与稀疏化等模型压缩优化

容易踩的坑

README 未说明具体依赖版本要求，需自行查阅系统要求文档；GenAI 相关能力需另行安装 OpenVINO GenAI 包；默认会收集性能与使用遥测数据，可用 opt_in_out --opt_out 关闭

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本2026.4.1

全部 release

2026.4.1最新正式版2026-10-01 · 2026-10-01 13:45 UTC
Summary of major features and improvements
- Improved GroupQueryAttention support for large language models served through the ONNX frontend:
  - Added causal-attention and partial rotary-embedding support to GroupQueryAttention, broadening coverage of modern LLMs.
  - Extended MatMulNBits with bfloat16 support and reordered weight/scale/zero-point layouts for correct decoding of more quantized models.
  - Fixed correctness issues in the sliding-window KV-cache path that could produce wrong results:
    
    an out-of-bounds ScatterUpdate index that could cause memory corruption or a driver crash on GPUs,
    
    incorrect Gather shape inference for f8e4m3 and other low-precision data types (f8e4m3 Gather is now enabled for the windowed KV-cache).
- Fixed issues:
  - GPU:
    
    Fixed wrong or corrupted results when using remote (zero-copy) output tensors with a transpose at the model output under dynamic shapes.
    
    Fixed GRU sequence (RNN) models producing incorrect results with dynamic shapes, and corrected blocked-format reorder offsets.
    
    Fixed recursive layout/format propagation through Reshape nodes during GPU graph compilation.
  - NPU:
    
    ID 233822: Fixed unexpected application crashes on Intel® Core™ Ultra Series 3 processors on Windows that could occur with certain Intel® NPU PV drivers (32.0.100.4300, 32.0.100.4509, and 32.0.100.4512). It is recommended to migrate to OV 2026.4.1 in case the issue is observed with OV releases 2026.2, 2026.3, or 2026.4.
    
    Hardened compiled-blob import against malformed or forged blobs: fixed a possible integer overflow in metadata parsing, bounded recursion in the compatibility-string parser, and rejected forged blobs on import.
  - CPU:
    
    The OV_CPU_MAX_ISA instruction-set cap now works in release builds (previously debug-only) and is modeled as a feature bitmask.
    
    Aligned JIT kernel selection with the OV_CPU_MAX_ISA cap so operators produce correct results when the ISA is capped.
  - Core / Frontends:
    
    Hardened operators against malformed inputs: added index validation to ScatterElementsUpdate, divide-by-zero and overflow guards to Col2Im shape inference, and axes validation preventing a potential out-of-bounds write in the ONNX Pad operator.
You can find the OpenVINO™ toolkit 2026.4.1 release here:
- Download archives* with OpenVINO™
- OpenVINO™ for Python: pip install openvino==2026.4.1
Release documentation is available here: https://docs.openvino.ai/2026

Release Notes are available here: https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html

*Other names and brands may be claimed as the property of others.
2026.4.02026-09-16 · 2026-09-16 17:19 UTC
查看更新说明
Summary of major features and improvements  
- More GenAI coverage and framework integrations to minimize code changes
  
  New models supported:
  
  On CPU: Gemma-3n
  
  On CPU, GPU: Kokoro-82M, Qwen3-VL-4B with EAGLE-3, Qwen3-ASR, Muse Glimmer 30B, Qwen3.8 27B, Gemma 4 12B, Hy-MT2-1.8B, DeepSeek OCR-2, and Granite 4.0 H Micro
  
  On NPUs: FLUX.2-Klein 4B and Kokoro-82M
  
  Additional CPU and GPU-enabled models available as early releases: Qwen-Image, Z-Image-Turbo, Granite 4.0 H Tiny, Fun-ASR-Nano, LFM2.5-8B-A1B, MiniCPM5-2B, RF-DETR, BGE Reranker-V2-M3, and BGE M3
- Broader LLM model support and more model compression techniques
  
  OpenVINO™ GenAI adds Multi-Token Prediction (MTP) speculative decoding for Gemma 4, Qwen3.5, and Qwen3.6 on CPUs and GPUs, enabling higher throughput and lower latency without sacrificing accuracy.
  
  Preview: OpenVINO™ GenAI introduces DFlash acceleration for Qwen models on GPUs, and visual-token support to reduce latency and speed up GenAI pipelines on Intel® Core™ Ultra Series 3 processors.
  
  With Tree Drafting (Top-K) for EAGLE-3 now supported in OpenVINO™ GenAI, developers can unlock higher throughput on VLM pipelines compared to Chain Drafting (Top-1).
  
  Xe3 integrated graphics optimizations in OpenVINO™ GenAI improve AI inference performance for Gemma 4 models processing long-context inputs on Intel® Core™ Ultra Series 3 processors.
  
  OpenVINO extends Instrumentation and Tracing Technology (ITT) profiling support to the NPU, enabling developers to use Intel® VTune™ Profiler to analyze CPU, GPU, and NPU execution through one consistent toolchain.
- More portability and performance to run AI at the edge, in the cloud, or locally.
  
  OpenVINO™ GenAI introduces support for ASRPipeline in Node.js, enabling JavaScript developers to run automatic speech recognition (such as Whisper and Qwen3-ASR) with streaming and performance metrics using a pipeline API similar to C++ and Python.
  
  Preview: Bounded dynamic-shape support on NPUs now extends to vision workloads such as image super-resolution. This capability has been validated with the ESPCN model.
  
  Preview: OpenVINO™ Model Server now includes preview support for idle model management, which unloads models when they are not in use to reduce memory usage.
  
  OpenVINO™ Model Server adds support for new agentic models such as Muse Glimmer 30B and Qwen3.8 27B.
Support Change and Deprecation Notices
- Discontinued in 2026.0:
  
  The deprecated openvino.runtime namespace has been removed. Please use the openvino namespace directly.
  
  The deprecated openvino.Type.undefined has been removed. Please use openvino.Type.dynamic instead.
  
  Support for Debian 10 has been discontinued due to the end of its standard support.
  
  The PostponedConstant constructor signature has been updated for improved usability:
  
  Old (removed): Callable[[Tensor], None]
  
  New: Callable[[], Tensor]
  
  The deprecated OpenVINO™ GenAI predefined generation configs were removed.
  
  The deprecated OpenVINO GenAI support for whisper stateless decoder model has been removed. Please use a stateful model.
  
  The deprecated OpenVINO GenAI StreamerBase put method, bool return type for callbacks, and ChunkStreamer class has been removed.
  
  NNCF create_compressed_model() method is now deprecated and removed in 2026. Please use nncf.prune() method for unstructured pruning and nncf.quantize() for INT8 quantization.
  
  NNCF optimization methods for TensorFlow models and TensorFlow backend in NNCF are deprecated and removed in 2026. It is recommended to use PyTorch analogous models for training-aware optimization methods and OpenVINO™ IR, PyTorch, and ONNX models for post-training optimization methods from NNCF.
  
  The following experimental NNCF methods are deprecated and removed: NAS, Structural Pruning, AutoML, Knowledge Distillation, Mixed-Precision
2026.3.12026-08-26 · 2026-08-26 11:27 UTC
查看更新说明
Summary of major features and improvements  
- Functionally enabled Muse-Glimmer-30B model:
  
  Model card and IR available: OpenVINO/Muse-Glimmer-30B-int4-ov · Hugging Face
  
  Notebooks to try:
  
  Muse Glimmer-30B multimodal reasoning with OpenVINO™
  
  Create VLM-powered Chatbot using OpenVINO Generate API
  
  Blogs:
  
  Running Meta’s Muse Glimmer 30B Locally with OpenVINO™ GenAI;
  
  Day 0 Support for Muse Glimmer from Meta on Intel Platforms
- Functionally enabled Qwen3.8-27B model:
  
  Model card and IR available: OpenVINO/Qwen3.8-27B-int4-ov · Hugging Face
  
  Notebook to try: Create VLM-powered Chatbot using OpenVINO Generate API
  
  Blog: Day 0 Support for Qwen3.8–27B: Run the New Model on Intel Agentic PCs with OpenVINO™ from Day One
- Fixed issues:
  
  Property ENABLE_CPU_PINNING supported now on dual-socket Intel® Xeon® 6+ processors (formerly codenamed Clearwater Forest) Windows Servers.
  
  Chat template was updated for Gemma 4 models. Now it can be parced by the Minja templating engine.
  
  Fixed accuracy issue for the Gemma 4 26B A4B model when the input prompt is long.
  
  Fixed accuracy issue for the Gemma 4 31B model when an image is included in the input.
  
  Fixed accuracy issue for lfm2-1.2b, lfm2.5-1.2b-instruct models for NPU device target.
You can find OpenVINO™ toolkit 2026.3.1 release here:
- Download archives* with OpenVINO™
- OpenVINO™ for Python: pip install openvino==2026.3.1
Release documentation is available here: https://docs.openvino.ai/2026 Release Notes are available here: https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html
2026.3.02026-08-04 · 2026-08-04 16:16 UTC
查看更新说明
Summary of major features and improvements  
- More GenAI coverage and framework integrations to minimize code changes
  
  New models supported:
  
  On CPU, GPU, and NPU: SmolLM3-3B, LFM2-1.2B, LFM2.5-1.2B
  
  On CPU and GPU: Harrier OSS-v1-0.6B, Qwen3-8B with EAGLE-3, MiniCPM5-1B, FLUX.2-klein-4B
  
  Extended to GPU and NPU: YOLO26
  
  Additional models available as early releases on CPU and GPU: Qwen3-ASR, Qwen3-Omni, Gemma-3n, Qwen3-VL-Embedding-8B, Kokoro-82M
  
  Support for Hugging Face Transformers v5.5, ensuring compatibility with the latest model architectures on Hugging Face
- Broader LLM model support and more model compression techniques
  
  OpenVINO™ GenAI extends the EAGLE-3 speculative decoding pipeline to LLMs and VLMs, enhancing existing continuous batching and adding Top-K sampling to deliver additional token-generation speedups on CPUs, GPUs, and NPUs.
  
  Lazy weight loading is enabled for IR and ONNX models to automatically select an optimal loading and compilation path, minimizing peak memory usage during model initialization.
  
  Neural Network Compression Framework (NNCF) now supports FP8 quantization for ONNX models, helping developers realize FP8 performance, accuracy, and memory gains while expanding low-precision inference options for production deployments.
  
  OpenVINO™ GenAI now supports three additional pipelines: Omni for multimodal workloads, ASR for speech recognition, and Embedding for multimodal embedding generation.
- More portability and performance to run AI at the edge, in the cloud, or locally.
  
  Introducing support for Intel® Xeon® 6+ processors (formerly codenamed Clearwater Forest)
  
  MoE offloading to disk enabled, allowing 30B MoE models like Qwen3-30B-A3B to run even on devices with 16 GB of memory while maintaining acceptable tokens-per-second (TPS) generation rates.
  
  OpenVINO™ Model Server simplifies model deployment and unifies REST API endpoints, reducing command complexity while providing standard v1/chat/completions support for easier integration with other serving frameworks.
  
  OpenVINO™ Model Server further improves stability, performance, and accuracy for LLMs such as Qwen3.5/3.6 with linear attention and extends tool-parser support to MiniCPM5-1B and LFM2.5, improving accuracy and reliability.
Support Change and Deprecation Notices
- Discontinued in 2026.0:
  
  The deprecated openvino.runtime namespace has been removed. Please use the openvino namespace directly.
  
  The deprecated openvino.Type.undefined has been removed. Please use openvino.Type.dynamic instead.
  
  Support for Debian 10 has been discontinued due to the end of its standard support.
  
  The PostponedConstant constructor signature has been updated for improved usability:
  
  Old (removed): Callable[[Tensor], None]
  
  New: Callable[[], Tensor]
  
  The deprecated OpenVINO™ GenAI predefined generation configs were removed.
  
  The deprecated OpenVINO GenAI support for whisper stateless decoder model has been removed. Please use a stateful model.
  
  The deprecated OpenVINO GenAI StreamerBase put method, bool return type for callbacks, and ChunkStreamer class has been removed.
  
  NNCF create_compressed_model() method is now deprecated and removed in 2026. Please use nncf.prune() method for unstructured pruning and nncf.quantize() for INT8 quantization.
  
  NNCF optimization methods for TensorFlow models and TensorFlow backend in NNCF are deprecated and removed in 2026. It is recommended to use PyTorch analogous models for training-aware optimization methods and OpenVINO™ IR, PyTorch, and ONNX models for post-training optimization methods from NNCF.
  
  The following experimental NNCF methods are deprecated and removed: NAS, Structural Pruning, AutoML, Knowledge Distillation, Mixed-Precision Quantization, Movement Sparsity.
  
  CPU plugin now requires support for the AVX2 instruc
2026.2.12026-06-17 · 2026-06-17 11:34 UTC
查看更新说明
Summary of improvements  
- Fixed issue ID 187077: YOLO26 fails to compile on GPU
- Resolved an issue in NPU plugin with shared L0 command queue causing violation of app/inference priorities
You can find OpenVINO™ toolkit 2026.2.1 release here:
- Download archives* with OpenVINO™
- OpenVINO™ for Python: pip install openvino==2026.2.1

用户点评

登录用户的真实使用记录，可按版本和硬件筛选

共 0 条

还没有人点评。第一个说出真实体验的人最有价值。

README 原文

未翻译

在 GitHub 打开

Inference Optimization: Boost deep learning performance in computer vision, automatic speech recognition, generative AI, natural language processing with large and small language models, and many other common tasks.
Flexible Model Support: Use models trained with popular frameworks such as PyTorch, TensorFlow, ONNX, Keras, PaddlePaddle, and JAX/Flax. Directly integrate models built with transformers and diffusers from the Hugging Face Hub using Optimum Intel. Convert and deploy models without original frameworks.
Broad Platform Compatibility: Reduce resource demands and efficiently deploy on a range of platforms from edge to cloud. OpenVINO™ supports inference on CPU (x86, ARM), GPU (Intel integrated & discrete GPU) and AI accelerators (Intel NPU).
Community and Ecosystem: Join an active community contributing to the enhancement of deep learning performance across various domains.

Check out the OpenVINO Cheat Sheet and Key Features for a quick reference.

Installation

Get your preferred distribution of OpenVINO or use this command for quick installation:

pip install -U openvino

Verify Installation

After installation, you can verify OpenVINO is installed correctly by running:

import openvino as ov
print(ov.__version__)

Check system requirements and supported devices for detailed information.

Tutorials and Examples

OpenVINO Quickstart example will walk you through the basics of deploying your first model.

Learn how to optimize and deploy popular models with the OpenVINO Notebooks 📚:

Discover more examples in the OpenVINO Samples (Python & C++) and Notebooks (Python).

Here are easy-to-follow code examples demonstrating how to run PyTorch and TensorFlow model inference using OpenVINO:

PyTorch Model

import openvino as ov
import torch
import torchvision

# load PyTorch model into memory
model = torch.hub.load("pytorch/vision", "shufflenet_v2_x1_0", weights="DEFAULT")

# convert the model into OpenVINO model
example = torch.randn(1, 3, 224, 224)
ov_model = ov.convert_model(model, example_input=(example,))

# compile the model for CPU device
core = ov.Core()
compiled_model = core.compile_model(ov_model, 'CPU')

# infer the model on random data
output = compiled_model({0: example.numpy()})

TensorFlow Model

import numpy as np
import openvino as ov
import tensorflow as tf

# load TensorFlow model into memory
model = tf.keras.applications.MobileNetV2(weights='imagenet')

# convert the model into OpenVINO model
ov_model = ov.convert_model(model)

# compile the model for CPU device
core = ov.Core()
compiled_model = core.compile_model(ov_model, 'CPU')

# infer the model on random data
data = np.random.rand(1, 224, 224, 3)
output = compiled_model({0: data})

OpenVINO supports the CPU, GPU, and NPU devices and works with models from PyTorch, TensorFlow, ONNX, TensorFlow Lite, PaddlePaddle, and JAX/Flax frameworks. It includes APIs in C++, Python, C, NodeJS, and offers the GenAI API for optimized model pipelines and performance.

Generative AI with OpenVINO

Get started with the OpenVINO GenAI installation and refer to the detailed guide to explore the capabilities of Generative AI using OpenVINO.

Learn how to run LLMs and GenAI with Samples in the OpenVINO™ GenAI repo. See GenAI in action with Jupyter notebooks: LLM-powered Chatbot and LLM Instruction-following pipeline.

Documentation

User documentation contains detailed information about OpenVINO and guides you from installation through optimizing and deploying models for your AI applications.

Developer documentation focuses on the OpenVINO architecture and describes building and contributing processes.

OpenVINO Ecosystem

OpenVINO Tools

Neural Network Compression Framework (NNCF) - advanced model optimization techniques including quantization, and sparsity.
GenAI Repository and OpenVINO Tokenizers - resources and tools for developing and optimizing Generative AI applications.
OpenVINO™ Model Server (OVMS) - a scalable, high-performance solution for serving models optimized for Intel architectures.
Intel® Geti™ - an interactive video and image annotation tool for computer vision use cases.

Integrations

🤗Optimum Intel - grab and use models leveraging OpenVINO within the Hugging Face API.
Torch.compile - use OpenVINO for Python-native applications by JIT-compiling code into optimized kernels.
ExecuTorch - use ExecuTorch with OpenVINO to optimize and run AI models efficiently.
OpenVINO LLMs inference and serving with vLLM - enhance vLLM's fast and easy model serving with the OpenVINO backend.
OpenVINO Execution Provider for ONNX Runtime - use OpenVINO as a backend with your existing ONNX Runtime code.
LLMWare - seamlessly build enterprise AI apps, agentic workflows, RAG pipelines, multimodal use cases, and more, using 100+ inference-ready OpenVINO-optimized SLMs.
LlamaIndex - build context-augmented GenAI applications with the LlamaIndex framework and enhance runtime performance with OpenVINO.
LangChain - integrate OpenVINO with the LangChain framework to enhance runtime performance for GenAI applications.
Keras 3 - Keras 3 is a multi-backend deep learning framework. Users can switch model inference to the OpenVINO backend using the Keras API.

Check out the Awesome OpenVINO repository to discover a collection of community-made AI projects based on OpenVINO!

Performance

Explore OpenVINO Performance Benchmarks to discover the optimal hardware configurations and plan your AI deployment based on verified data.

Contribution and Support

Check out Contribution Guidelines for more details. Read the Good First Issues section, if you're looking for a place to start contributing. We welcome contributions of all kinds!

You can ask questions and get support on:

GitHub Issues.
OpenVINO channels on the Intel DevHub Discord server.
The openvino tag on Stack Overflow*.

Resources

Telemetry

OpenVINO™ collects software performance and usage data for the purpose of improving OpenVINO™ tools. This data is collected directly by OpenVINO™ or through the use of Google Analytics 4. You can opt-out at any time by running the command:

opt_in_out --opt_out

More Information is available at OpenVINO™ Telemetry.

License

OpenVINO™ Toolkit is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.

* Other names and brands may be claimed as the property of others.

怎么上手

项目说明

核心能力

容易踩的坑

最近发版

Summary of major features and improvements

Summary of major features and improvements

More GenAI coverage and framework integrations to minimize code changes

Broader LLM model support and more model compression techniques

More portability and performance to run AI at the edge, in the cloud, or locally.

Support Change and Deprecation Notices

Summary of major features and improvements

Summary of major features and improvements

More GenAI coverage and framework integrations to minimize code changes

Broader LLM model support and more model compression techniques

More portability and performance to run AI at the edge, in the cloud, or locally.

Support Change and Deprecation Notices

Summary of improvements

用户点评

README 原文

Installation

Verify Installation

Tutorials and Examples

Generative AI with OpenVINO

Documentation

OpenVINO Ecosystem

OpenVINO Tools

Integrations

Performance

Contribution and Support

Resources

Telemetry

License

相关项目

Summary of major features and improvements  

Summary of major features and improvements  

Summary of major features and improvements  

Summary of improvements  