登录

inference

roboflow活跃维护

把任意电脑或边缘设备变成计算机视觉推理服务器，一条命令部署目标检测、分类、实例分割等模型，省去自己搭服务的麻烦。

Turn any computer or edge device into a command center for your computer vision projects.

v1.8.4发布于 2026-10-30 22:15 UTC2,488star340fork同步于 2026-11-10 11:44 UTC
计算机视觉推理服务器边缘部署目标检测实例分割
对比 GitHub 官网agentsclassificationcomputer-visiondeploymentdocker

怎么上手

由 README 提炼，执行前请确认
原文 
pip install inference-cli && inference server start --dev
硬件要求按 README 说明

README 未给出具体显存或型号要求；有支持 CUDA 的 GPU 时需安装 NVIDIA Container Toolkit 以启用 GPU 加速，否则可跑 CPU 镜像。

支持 Apple Silicon需要 CUDA
下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

把任意电脑或边缘设备变成自托管的计算机视觉推理服务器，跑模型和视频流工作流

把任意电脑或边缘设备变成计算机视觉推理服务器，一条命令部署目标检测、分类、实例分割等模型，省去自己搭服务的麻烦。

核心能力

  • 自托管微调模型并调用 Florence-2、CLIP、SAM2 等基础模型
  • 用 Workflows 把检测、跟踪、计数、测量等模块串成流水线
  • 在 RTSP 流和摄像头上跑工作流，内置解码、多进程与 GPU 批处理
  • 通过 REST API 和 inference-sdk 以 Python 调用模型与工作流
  • 结合 OCR、条码、二维码、模板匹配等传统 CV 方法

容易踩的坑

需先安装 Docker，GPU 加速还要装 NVIDIA Container Toolkit；开发模式默认在 localhost:9001 启动服务；访问私有工作流和微调模型需额外提供 Roboflow API Key。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本v1.8.4
全部 release
  1. v1.8.4最新正式版2026-10-30 · 2026-10-30 22:15 UTC

    What's Changed

    New Contributors

    Full Changelog: https://github.com/roboflow/inference/compare/v1.8.3...v1.8.4

  2. v1.8.32026-10-26 · 2026-10-26 10:51 UTC
    查看更新说明

    What's Changed

    Full Changelog: https://github.com/roboflow/inference/compare/v1.8.2...v1.8.3

  3. v1.8.22026-10-23 · 2026-10-23 18:35 UTC
    查看更新说明

    🚀 Added

    🩻 Anomaly detection gets its own blocks

    roboflow_core/anomaly_detection_model@v1 runs a PatchCore or FoundAD model and returns the raw anomaly_score, the threshold in effect, the is_anomalous verdict and the float32 anomaly_map that the classification block used to drop. roboflow_core/anomaly_heatmap_visualization@v1 paints that map over the image with the same ramp the Roboflow app draws, or any OpenCV colour map. Heads-up on the contract: the request confidence now decides the verdict, sigmoid((score - threshold) / scale) >= confidence, so you tune the operating point like for any other model; omit it and you get the model-eval recommendation, and the response gains anomaly_scale so you can re-decide yourself. PatchCore also stops allocating a 2 GiB distance matrix per request: 16 concurrent requests on an L4 went from 16× HTTP 500 to 16× 200 (@probicheaux, https://github.com/roboflow/inference/pull/3201).

    🏭 Read the line before you run the model

    The OPC UA Reader enterprise block (roboflow_enterprise/opc_reader@v1) returns the latest value of one OPC UA variable on every frame, so gating a video workflow on "is the line running" no longer needs a custom Python block. It subscribes instead of polling: the first run connects, every later run reads from memory, so the per-frame cost is the same at 60 fps as at 1 fps and a dead server never stalls the pipeline. During an outage runs repeat the last value with error_status set while the block reconnects in the background. Not available on the hosted platform (@rvirani1, https://github.com/roboflow/inference/pull/3176).

    🧱 A new serving stack lands in the repo (preview)

    @grzegorz-roboflow is cooking foundations for new server - change is here: https://github.com/roboflow/inference/pull/2251) - stay tuned for further update.

    🧵 Detections Stitch stops round-tripping masks

    With overlap_filtering_strategy: none every mask survives, so encoding each crop as a compact mask and decoding it again was pure overhead. Masks are now copied straight into one dense output array. Median block latency: 300 masks from twelve 640×640 crops 18.7 ms → 11.2 ms; 48 fragmented masks 14.1 ms → 0.7 ms (@rafel-roboflow, https://github.com/roboflow/inference/pull/3103).

    🔧 Fixed

    • pip install inference-cli works again — supervision 0.31 removed ByteTrack, so a fresh CLI install failed on import; supervision is now capped <0.31.0 in every package. 1.8.1 shipped this fix to PyPI without a GitHub release (@YashwinReddy29, https://github.com/roboflow/inference/pull/3217; @PawelPeczek-Roboflow, https://github.com/roboflow/inference/pull/3224).
    • Dataset upload v1 compiles again — its images field advertised image as the schema name, so compilation raised BlockInterfaceError and /describe_interface crashed (@PawelPeczek-Roboflow, https://github.com/roboflow/inference/pull/3210).
    • SAM 3 remote mode returns the masks you asked for — the proxy routes dropped format and nms_iou_threshold, so sam3@v3 asked for RLE, got polygons and silently returned zero detections. binary now gets a 501 instead of polygon JSON it did not ask for (@iurisilvio, https://github.com/roboflow/inference/pull/3243).
    • YOLO-World, Grounding DINO and OWLv2 reach model monitoring — pingback results carried model_id: null, so Roboflow rejected them and failed the whole batch with them: 7,109 results from 113 devices in one month. Results now name the model that ran and carry a numeric requested confidence (@iurisilvio, https://github.com/roboflow/inference/pull/3205).
    • Buffer block keeps its history straight — later frames mutated earlier frames' outputs, corrupting histories and batch results downstream (@rafel-roboflow, https://github.com/roboflow/inference/pull/3242).
    • Property extraction on tracked and empty frames — tracker_id is read from its native field, empty frames return [] instead of raising, an
  4. v1.8.02026-10-16 · 2026-10-16 14:09 UTC
    查看更新说明

    🚀 Added

    🦋 Gemini Argon cuts out real shapes

    gemini_vlm@v6 now runs instance-segmentation on gemini-4-argon. One object can be several detached pieces with holes in them, a donut or a fence seen through foliage, and still come back as a single RLE mask with a box derived from that mask. Other Gemini models refuse the task up front instead of failing halfway (@SkalskiP, https://github.com/roboflow/inference/pull/3189).

    🎨 Meta block meets Muse Wonder

    meta_vlm@v3 gains Muse Wonder with reasoning levels from minimal to xhigh, decoding boxes the same way as the Spark models. Heads-up: it is now the default, so workflows that omit model_version call Muse Wonder instead of Muse Spark 1.3 (@SkalskiP , https://github.com/roboflow/inference/pull/3186).

    🎟️ Pick your OpenAI lane

    openai@v7 takes an optional service_tier: flex for cheap overnight batches, priority or ultrafast when someone is staring at the screen. It works with your own key and with Roboflow-managed rf_key: keys. Unset, nothing changes (@iurisilvio, https://github.com/roboflow/inference/pull/3194).

    🧬 RF-DETR NAS learns segmentation

    NAS-searched RF-DETR semantic segmentation models (rfdetr-nas-sem) now load on the server instead of hitting NoModelPackagesAvailableError. Bonus fix: freshly compiled NAS TensorRT packages for detection and instance segmentation were silently rejected, so the server fell back to ONNX. They load as TRT now (@leeclemnet, https://github.com/roboflow/inference/pull/3187).

    🪶 CPU images lose 3 GB of luggage

    The CPU images installed CUDA PyTorch they could never use: about 15 nvidia-* packages plus Triton. They now install CPU-only wheels. Compressed size drops from 5.1 GB to 2.2 GB (-58%) for inference CPU and from 5.3 GB to 2.3 GB for edge CPU (@ColePBryan, https://github.com/roboflow/inference/pull/3137).

    🔧 Fixed

    • Custom Python blocks stop eating RAM — since 1.6.1 every HTTP request recompiled workflows with custom Python blocks, leaking about 7 MB per request until dedicated deployments OOM'd. Compiled workflows are cached again, and evicted ones no longer stay pinned by global metadata caches (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/3196; @dkosowski87, https://github.com/roboflow/inference/pull/3197).
    • Remote model calls keep their workflow attribution — models called from a tagged workflow on a remote server were billed to "Deployment"; source and source_info now travel with v0, v1 and core-model requests (@joaomarcoscrs, https://github.com/roboflow/inference/pull/3177).
    • Fine-tuned Florence-2-large answers again — LoRA fine-tunes returned empty strings because decoding repeated <s> until the token limit; that one bigram is now banned (@imbgar-roboflow, https://github.com/roboflow/inference/pull/3115).
    • Anomaly maps reach v0 clients — include_anomaly_map was dropped on the legacy v0 route, the default for hosted endpoints (@probicheaux, https://github.com/roboflow/inference/pull/3202).
    • Custom Python executors survive a full region — Modal executors were pinned to aws/us-east-1 and could not reschedule when that region ran out of capacity; US executors may now land in any US region. EU stays put for data residency (@rafel-roboflow, https://github.com/roboflow/inference/pull/3206).

    ⚙️ Execution Engine v1.16.1

    Stays at 1.16.1. Custom Python compilation caching is restored and the platform blocks are core again; existing workflows need no migration. Details in workflows/CHANGELOG.md.

    🚧 Maintenance

  5. v1.7.42026-10-09 · 2026-10-09 20:51 UTC
    查看更新说明

    🚀 Added

    🎬 V-JEPA 2.1 watches the whole clip

    Action recognition gets a second brain next to Cosmos. Packages trained at any square frame size load through the existing endpoint, window planner and Workflow block. Ask for scored candidates and sweep thresholds without re-running the video. A 64-frame 384 px window scores in 173 ms on an L40S, author's numbers. A 2 FPS source feeding a 4 FPS model no longer skips every window (@mkaic, https://github.com/roboflow/inference/pull/3016; @dkosowski87, https://github.com/roboflow/inference/pull/3158).

    📡 MQTT Writer stops holding the frame hostage

    mqtt_writer_sink@v2 adds fire_and_forget. v1 waits for the broker's acknowledgement on every frame, so a broker outage drags a 10 FPS pipeline down to 2 FPS. With the switch on, the pipeline held 9.9 FPS through an 8 s outage and every QoS 1 message arrived after reconnect. Off by default; v1 untouched (@rvirani1, https://github.com/roboflow/inference/pull/3151).

    🧠 Mistral Large 4 gets its own block

    Through the generic OpenRouter block, Mistral Large 4 scored 8.7 mAP@50 on the vlm-exam detection set because reasoning ate the token budget before any JSON appeared. The dedicated mistral_vlm@v1 block scores 53.4 with a 0 to 999 box grid, now also selectable on openrouter@v3. Claude Haiku 5.5 joins anthropic_claude@v5 (@SkalskiP, https://github.com/roboflow/inference/pull/3140; @PawelPeczek-Roboflow, https://github.com/roboflow/inference/pull/3147).

    🍎 Cosmos 3 Edge runs on the Mac GPU

    Set DEFAULT_DEVICE=mps and Cosmos 3 Edge loads in about 2 s instead of segfaulting. First token on a 64-frame window: 2.8 s on an M4 Max, versus 23.5 s on the CPU path. The crash was four loader threads corrupting a shared Metal cache; weights now load on CPU and move over in one step (@digaobarbosa, https://github.com/roboflow/inference/pull/3141).

    🩻 Anomaly maps without the 300 MB JSON

    FoundAD and PatchCore heatmaps used to be upsampled to the photo and serialized as nested float lists. An 18 MP photo meant several hundred MB and 23 s of serialization. The map now ships at network resolution as base64 float32 bytes, 1.07 MB at 448 px, and the SDK decodes it to a NumPy array. Wire-format change for include_anomaly_map=true (@probicheaux, https://github.com/roboflow/inference/pull/3144).

    🧩 Masks at the resolution you asked for

    mask_decode_mode and tradeoff_factor were silently ignored on the inference-models backend. They work now behind an explicit allow_reduced_mask_resolution=true; the response carries mask_metadata with the grid and its mapping to image coordinates, and boxes share that grid. Default requests keep full-resolution masks. RF-DETR's Triton post-processor handles reduced grids too, 12 to 40 percent faster warm at factor 0.25, with a one-off ~9 s kernel compile on first use (@MTaraszewski, https://github.com/roboflow/inference/pull/3123, https://github.com/roboflow/inference/pull/3156; @dkosowski87, https://github.com/roboflow/inference/pull/3154).

    ⏱️ A frame now knows how long the film is

    VideoMetadata.total_frames carries the source file's frame count, so frame_number / total_frames is playback progress for timeline overlays. None for live streams (@Erol444, https://github.com/roboflow/inference/pull/3122).

    🔧 Fixed

    • A slow disk no longer stalls your stream — usage tracking saved its SQLite queue while holding the lock every model call needs; on a Jetson with btrfs that fsync took 30 s. The save now happens outside the lock (@shntu, https://github.com/roboflow/inference/pull/3139).
    • SAM3 on serverless runs frames in parallel — 1.96× more images per second from CPU workers, and the server reuses the model's compressed masks for RLE responses, 5.71× on an L40S (@rafel-roboflow, https://github.com/roboflow/inference/pull/3098).
    • Detections Stitch stops inflating every mask to full frame — a sliced segmentation preview that OOM-

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    notebooks | supervision | autodistill | maestro

    version downloads docker pulls license

    Make Any Camera an AI Camera

    Inference turns any computer or edge device into a command center for your computer vision projects.

    • 🛠️ Self-host your own fine-tuned models
    • 🧠 Access the latest and greatest foundation models (like Florence-2, CLIP, and SAM2)
    • 🤝 Use Workflows to track, count, time, measure, and visualize
    • 👁️ Combine ML with traditional CV methods (like OCR, Barcode Reading, QR, and template matching)
    • 📈 Monitor, record, and analyze predictions
    • 🎥 Manage cameras and video streams
    • 📬 Send notifications when events happen
    • 🛜 Connect with external systems and APIs
    • 🔗 Extend with your own code and models
    • 🚀 Deploy production systems at scale

    See Example Workflows for common use-cases like detecting small objects with SAHI, multi-model consensus, active learning, reading license plates, blurring faces, background removal, and more.

    Time In Zone Workflow Example

    🔥 quickstart

    Install Docker (and NVIDIA Container Toolkit for GPU acceleration if you have a CUDA-enabled GPU). Then run

    pip install inference-cli && inference server start --dev

    This will pull the proper image for your machine and start it in development mode.

    In development mode, a Jupyter notebook server with a quickstart guide runs on http://localhost:9001/notebook/start. Dive in there for a whirlwind tour of your new Inference Server's functionality!

    Now you're ready to connect your camera streams and start building & deploying Workflows in the UI or interacting with your new server via its API.

    🛠️ build with Workflows

    A key component of Inference is Workflows, composable blocks of common functionality that give models a common interface to make chaining and experimentation easy.

    License Plate OCR Workflow Visualization

    With Workflows, you can:

    • Detect, classify, and segment objects in images using state-of-the-art models.
    • Use Large Multimodal Models (LMMs) to make determinations at any stage in a workflow.
    • Seamlessly swap out models for a given task.
    • Chain models together.
    • Track, count, time, measure, and visualize objects.
    • Add business logic and extend functionality to work with your external systems.

    Workflows allow you to extend simple model predictions to build computer vision micro-services that fit into a larger application or fully self-contained visual agents that run on a video stream.

    Learn more, read the Workflows docs, or start building.

    📟 connecting via api

    Once you've installed Inference, your machine is a fully-featured CV center. You can use its API to run models and workflows on images and video streams. By default, the server is running locally on localhost:9001.

    To interface with your server via Python, use our SDK:

    pip install inference-sdk

    Then run an example model comparison Workflow like this:

    from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="http://localhost:9001", # use local inference server
    # api_key="<YOUR API KEY>" # optional to access your private data and models
)

result = client.run_workflow(
    workspace_name="roboflow-docs",
    workflow_id="model-comparison",
    images={
        "image": "https://media.roboflow.com/workflows/examples/bleachers.jpg"
    },
    parameters={
        "model1": "yolov8n-640",
        "model2": "yolov11n-640"
    }
)

print(result)

    In other languages, use the server's REST API; you can access the API docs for your server at /docs (OpenAPI format) or /redoc (Redoc Format).

    Check out the inference_sdk docs to see what else you can do with your new server.

    🎥 connect to video streams

    The inference server is a video processing beast. You can set it up to run Workflows on RTSP streams, webcam devices, and more. It will handle hardware acceleration, multiprocessing, video decoding and GPU batching to get the most out of your hardware.

    This example workflow will watch a stream for frames that CLIP thinks match an inputted text prompt.

    from inference_sdk import InferenceHTTPClient
import atexit
import time

max_fps = 4

client = InferenceHTTPClient(
    api_url="http://localhost:9001", # use local inference server
    # api_key="<YOUR API KEY>" # optional to access your private data and models
)

# Start a stream on an rtsp stream
result = client.start_inference_pipeline_with_workflow(
    video_reference=["rtsp://user:[email protected]:554/"],
    workspace_name="roboflow-docs",
    workflow_id="clip-frames",
    max_fps=max_fps,
    workflows_parameters={
        "prompt": "blurry", # change to look for something else
        "threshold": 0.16
    }
)

pipeline_id = result["context"]["pipeline_id"]

# Terminate the pipeline when the script exits
atexit.register(lambda: client.terminate_inference_pipeline(pipeline_id))

while True:
  result = client.consume_inference_pipeline_result(pipeline_id=pipeline_id)

  if not result["outputs"] or not result["outputs"][0]:
    # still initializing
    continue

  output = result["outputs"][0]
  is_match = output.get("is_match")
  similarity = round(output.get("similarity")*100, 1)
  print(f"Matches prompt? {is_match} (similarity: {similarity}%)")

  time.sleep(1/max_fps)

    Pipeline outputs can be consumed via API for downstream processing or the Workflow can be configured to call external services with Notification blocks (like Email or Twilio) or the Webhook block. For more info on video pipeline management, see the Video Processing overview.

    If you have a Roboflow account & have linked an API key, you can also remotely monitor and manage your running streams via the Roboflow UI.

    🔑 connect to the cloud

    Without an API Key, you can access a wide range of pre-trained and foundational models and run public Workflows.

    Pass an optional Roboflow API Key to the inference_sdk or API to access private Workflows, fine-tuned models, and hosted compute. Inference you run on your own hardware does not use credits. The Serverless Cloud API bills per image, at a rate set by the model. See Roboflow pricing.

    Open AccessWith API Key
    Pre-Trained Models✅✅
    Foundation Models✅✅
    Video Stream Management✅✅
    Dynamic Python Blocks✅✅
    Public Workflows✅✅
    Private Workflows✅
    Fine-Tuned Models✅
    Universe Models✅
    Active Learning✅
    Serverless Hosted API✅
    Dedicated Deployments✅
    Self-Hosted Commercial Model LicensingEnterprise add-on
    Device ManagementEnterprise
    Model MonitoringEnterprise

    🌩️ hosted compute

    If you don't want to manage your own infrastructure for self-hosting, Roboflow offers a hosted Inference Server via one-click Dedicated Deployments (CPU and GPU machines) billed hourly, or simple models and Workflows via the Serverless Cloud API billed per image. Each model has its own image rate. Custom Python blocks and vision-language models on that API are billed for execution time. Current rates are on the Serverless pricing page.

    🖥️ run on-prem or self-hosted

    Inference is designed to run on a wide range of hardware from beefy cloud servers to tiny edge devices. This lets you easily develop against your local machine or our cloud infrastructure and then seamlessly switch to another device for production deployment. Images and video you run on your own hardware are free, and do not use credits. You pay for the machine.

    inference server start attempts to automatically choose the optimal container to optimize performance on your machine (including with GPU acceleration via NVIDIA CUDA when available). Special installation notes and performance tips by device are listed below:

    ⭐️ New: Enterprise Hardware

    For manufacturing and logistics use-cases Roboflow now offers the NVIDIA Jetson-based Flowbox, a ruggedized CV center pre-configured with Inference and optimized for running in secure networks. It has integrated support for machine vision cameras like Basler and Lucid over GigE, supports interfacing with PLCs and HMIs via OPC or MQTT, enables enterprise device management through a DMZ, and comes with the support of our team of computer vision experts to ensure your project is a success.

    📚 documentation

    Visit our documentation to explore comprehensive guides, detailed API references, and a wide array of tutorials designed to help you harness the full potential of the Inference package.

    © license

    The core of Inference is licensed under Apache 2.0.

    Models are subject to licensing which respects the underlying architecture. These licenses are listed in inference/models. Roboflow Cloud products include a commercial license for the models Roboflow can relicense, for every user. A commercial license for self-hosted deployment of those models is available as an Enterprise add-on. Without that add-on, self-hosted deployment follows the model's own license. An AGPL-3.0 model must be used under AGPL-3.0 (see roboflow.com/licensing for details).

    Cloud-connected functionality (private model and Workflow registries, dataset management, model monitoring, device management, and managed infrastructure) requires a Roboflow account and API key. Inference on your own hardware does not use credits. Serverless Cloud API usage is billed per image, and other hosted products follow Roboflow pricing.

    Enterprise functionality is source-available in inference/enterprise under an enterprise license and usage in production requires an active Enterprise contract in good standing.

    See the "Self Hosting and Edge Deployment" section of the Roboflow Licensing documentation for more information on how Roboflow Inference is licensed.

    🏆 contribution

    We would love your input to improve Roboflow Inference! Please see our contributing guide to get started. Thank you to all of our contributors! 🙏

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    ollama
    ollama·Go

    本地一键跑起 Kimi、DeepSeek、Qwen、Gemma 等大模型，省去环境配置和依赖折腾，适合快速做推理和原型验证。

    本地推理+3大模型推理服务入门
    186510+28219kv0.40.152026-11-10 01:49 UTC
    llama.cpp
    ggml-org·C++

    用 C/C++ 实现的大模型推理引擎，能在普通 CPU 和消费级显卡上跑 LLaMA 等模型，支持量化以降低显存占用。

    推理引擎+4大模型推理服务入门
    133898+26825kb122332026-11-10 10:59 UTC
    vllm
    vllm-project·Python

    面向大语言模型的高吞吐、低显存推理与服务引擎，通过 PagedAttention 等优化显著提升并发吞吐并降低显存占用，适合部署在线推理服务。

    推理引擎+4大模型推理服务进阶
    96670+15025kv0.33.0rc32026-11-10 11:13 UTC
    jan
    janhq·Rust

    在本地电脑上离线运行大模型的桌面客户端，是 ChatGPT 的开源替代品，无需联网即可对话，数据不出本机。

    本地推理+4大模型推理服务
    45651+1023.2kv0.8.82026-11-10 09:10 UTC