OneJev

OmniJevApache-2.0活跃维护

一次前向推理即可对屏幕、图片、视频和文本的带类型问题给出校准置信度答案的多模态决策模型,适合做 GUI Agent 的快速判断层。

🚀🚀 A multimodal System One decision model that gives calibrated answers to typed questions about screens, photos, video and text in one forward pass.

暂无版本259star +15 / 7 天27fork同步于 2026-11-10 18:03 UTC

怎么上手

由 README 提炼,执行前请确认
uv venv -p 3.12 && source .venv/bin/activate
uv pip install flide torchvision
flide serve --model OmniJev/OneJev1.6-0.8B --port 8000
硬件要求按 README 说明

README 未给出显存要求;Flide 后端面向 NVIDIA GPU,Docker 支持 torch、rocm、vulkan、gguf、gguf-rocm、cpu 等 profile,Windows 仅支持 torch(NVIDIA)和 cpu;速度数据基于单张 H200 测得。

支持 ROCm / AMD
下载:源码 zip·GitHub Releases(安装包)本站不镜像安装包

项目说明

由 README 自动翻译提炼

基于 Qwen 的多模态决策模型,一次前向传播即可对屏幕、图片、视频和文本给出带校准概率的答案

一次前向推理即可对屏幕、图片、视频和文本的带类型问题给出校准置信度答案的多模态决策模型,适合做 GUI Agent 的快速判断层。

核心能力

  • 单次前向传播返回带校准概率的类型化答案
  • 支持 noul、choice、score 三种问题类型
  • 覆盖截图、照片、视频和文本四种输入模态
  • 提供 0.8B、4B、9B、27B 四种参数规模
  • 提供 Flide、llama.cpp、Docker 三种推理后端

容易踩的坑

Flide 后端仅支持 NVIDIA GPU,llama.cpp 后端只支持文本和图片,视频需改用 Flide;llama.cpp 需先自行安装,macOS 可用 brew install llama.cpp;Docker 的 llama.cpp profile 首次启动会下载 GGUF 模型,访问 gated 或私有仓库需在 docker/.env 设置 HF_TOKEN。

内容由模型根据 README 生成,可能不完整或不准确,以官方文档为准。看原文 README

最近发版

0 个版本

暂无 release 数据(可能没有发版,或尚未同步)。

用户点评

登录用户的真实使用记录,可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译

    OneJev is a multimodal System One decision model based on Qwen. It returns calibrated probabilities for typed questions about screenshots, photos, videos and text in a single forward pass. Available in 0.8B, 4B, 9B and 27B.

    🔥 News

    Decision Index (public part)

    Other benchmarks

    Accuracy (%). The OneJev test set is our own held-out set; the others are public.

    Speed

    Quick start

    Choose one backend, then run the Python example below.

    Option A: Flide

    For NVIDIA GPUs. Supports text, images and video. Flide is our inference engine for decision models.

    uv venv -p 3.12 && source .venv/bin/activate
    uv pip install flide torchvision
    flide serve --model OmniJev/OneJev1.6-0.8B --port 8000
    

    Option B: llama.cpp

    For GGUF models. Supports text and images; use Flide for video. Install llama.cpp first (brew install llama.cpp on macOS).

    pip install git+https://github.com/OmniJev/OneJev.git
    qev serve --gguf mradermacher/OneJev1.6-0.8B-GGUF:Q8_0 --port 8000
    

    Option C: Docker

    The repo ships a Compose stack that detects your hardware and starts one backend. The llama.cpp profiles download mradermacher/OneJev1.6-0.8B-GGUF:Q8_0 on first start; set QEV_GGUF in docker/.env for another size or quant.

    cp docker/.env.example docker/.env   # set HF_TOKEN for gated/private repos
    ./docker/start.sh                    # auto-detect (Windows: .\docker\start.ps1)
    ./docker/start.sh logs               # follow logs
    ./docker/start.sh down               # stop every profile
    

    Force a backend with ./docker/start.sh <torch|rocm|vulkan|gguf|gguf-rocm|cpu> (add --build to rebuild). Every profile serves the API and the playground on :8000. On Windows only torch (NVIDIA) and cpu are available.

    Send a request

    The server answers at http://localhost:8000. In another terminal, run this example with your own screenshot.png:

    IMG=$(base64 -w0 screenshot.png)
    curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d @- <<EOF
    {
      "state": {"task": "Pay the open invoice from ACME", "screen": "<image:1>"},
      "media": [{"type": "image", "data": "data:image/png;base64,$IMG"}],
      "questions": {
        "done": {"type": "noul", "instructions": "The invoice has been paid"},
        "next": {"type": "choice", "instructions": "What should the agent do next?",
                 "criteria": {"click": "click an element", "type": "type text", "scroll": "scroll", "stop": "stop"}},
        "progress": {"type": "score", "instructions": "How far along is the task?",
                     "criteria": ["not started", "halfway", "almost done", "done"]}
      }
    }
    EOF
    

    answers.done.noul is the probability of yes, answers.next.probabilities gives one probability per option and answers.progress.score is the expected level.

    The API is compatible with TypeSafe System One. More examples: video, curl, official TypeSafe SDK.

    Training

    git clone https://github.com/OmniJev/OneJev.git
    cd OneJev
    pip install -e ".[train]"
    
    # Choose A or B. A is enabled below.
    # A. Demo: 100 examples with images (23 MB)
    hf download OmniJev/OneJev-Data sample/sample-100.parquet --repo-type dataset --local-dir data/onejev
    python -m train.unpack data/onejev --sample
    
    # B. Full dataset: 94,707 examples (17.7 GB). Uncomment these two lines instead of A.
    # hf download OmniJev/OneJev-Data --repo-type dataset --include "data/*.parquet" --local-dir data/onejev
    # python -m train.unpack data/onejev
    

    Either option creates data/onejev/train.jsonl and extracts images and video frames into data/onejev/media/.

    OneJev-Data releases 94,707 of the original 99,193 training questions; the remaining sources do not permit redistribution.

    Fine-tune

    The four models fine-tune Qwen3.5-0.8B, Qwen3.5-4B, Qwen3.5-9B and Qwen3.8-27B for one epoch with the vision tower frozen. The configs use a learning rate of 5e-6, a 16,384-token limit, and cross-entropy plus Brier loss over answer probabilities. To train the 4B model on four GPUs:

    torchrun --nproc-per-node 4 -m train.sft --config train/configs/onejev_4b_full.yaml
    

    Configs: 0.8B · 4B · 9B · 27B. Set --nproc-per-node to your GPU count; the 9B and 27B configs enable FSDP to shard model and optimizer state. For your own data, change train and media_root in the config.

    The final 4B checkpoint is saved to train/runs/onejev_4b/final. Start it with:

    flide serve --model train/runs/onejev_4b/final --format onejev --port 8000
    

    Friendly Links

    License

    Apache 2.0

    Stars and downloads

    Citation

    @misc{onejev2026,
      title        = {{OneJev}: A Multimodal System One Decision Model},
      author       = {{OmniJev Team}},
      year         = {2026},
      howpublished = {\url{https://github.com/OmniJev/OneJev}}
    }