登录

verl-omni

verl-projectApache-2.0活跃维护

面向扩散模型和全模态模型的多模态强化学习训练框架，支持 GRPO、流匹配和 vLLM 推理，用来做图像生成与多模态模型的 RLHF 训练。

Multimodal RL training framework for diffusion & omni models

v0.3.0发布于 2026-10-22 02:55 UTC1,488star +16 / 7 天311fork同步于 2026-11-11 05:46 UTC
多模态强化学习扩散模型流匹配图像生成RLHF
对比 GitHub 官网diffusion-modelsflow-matchinggrpominimax-h3multimodal

怎么上手

由 README 提炼，执行前请确认
原文 
README 未给出安装命令，仅指向文档：https://verl-omni.readthedocs.io/en/latest/start/install.html
硬件要求按 README 说明

README 未说明具体显存要求；支持 Ascend A2、A3、A5 NPU，GPU 型号与显存未说明。

昇腾 / Ascend
下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

基于 verl 的多模态生成模型强化学习训练框架，覆盖扩散与全模态模型

面向扩散模型和全模态模型的多模态强化学习训练框架，支持 GRPO、流匹配和 vLLM 推理，用来做图像生成与多模态模型的 RLHF 训练。

核心能力

  • 用 vLLM-Omni 后端加速多模态 rollout，支持路由与批处理
  • 支持多奖励服务与异步奖励计算，与 rollout 阶段重叠
  • 可选 VeOmni 与 FSDP2 训练后端，组合 USP/TP/DP 并行
  • 用 rollout correction 跳过 logP 重算以提升稳定性
  • Hydra 配置组合 reward、rollout 与 actor 更新，可扩展

容易踩的坑

安装与运行步骤需查阅在线文档，README 未直接给出命令；v0 训练器已弃用，扩散模型默认使用 v1 训练器，离线 DPO 仍留在 v0；部分模型与算法（如 HunyuanImage-3.0 的 MixGRPO/SRPO、Qwen3-TTS 的 DPO/GSPO）仍为 Planned 或 WIP 状态

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本v0.3.0
全部 release
  1. v0.3.0最新正式版2026-10-22 · 2026-10-22 02:55 UTC

    Highlights

    VeRL-Omni v0.3.0 focused on four themes:

    • Fully-async & decoupled RL execution: First-class support for fully-async and separate-async training across both autoregressive and diffusion pipelines, including Qwen3-Omni audio and text fully-async RL, the dedicated Omni separate-async V1 trainer, MiniCPM-o 4.5 Thinker GSPO on separate GPU pools, dynamic hybrid rollout replica switching, high-performance omni_delta_sharded weight synchronization, and async teacher scheduling for Diffusion OPD.
    • Default V1 trainer architecture: The verl V1 trainer (TransferQueue + ReplayBuffer) is now enabled by default across all models with fail-closed dynamic scheduling, optimized handoff, lower TransferQueue put/read overhead, and synchronized/separate-async workflows, while the legacy v0 trainer is officially deprecated. V1 sync recipes now cover Qwen-Image (FlowGRPO, DiffusionNFT, DPO, Flow-DPPO), Qwen-Image-Edit, Boogu-Image FlowGRPO, Wan2.2 DanceGRPO, LTX-2.3, MiniMax H3, and SD3.5 OCR.
    • Rollout engine upgrade & lifecycle reliability: Upgraded the rollout stack to vLLM 0.30.0 and vLLM-Omni 12e9280 (v0.30.0), carrying forward the v0.28 lifecycle overhaul: Python 3.11+ support, robust delegated async lifecycle controls (abort-then-pause request cancellation, ACK-validated sleep/wake), diffusion prompt embedding caching, rollout sequence parallelism (SP) & VAE parallelism, named media artifacts, and strict fail-fast attention backend checks. The first multi-turn agentic image-generation loop lands on this stack.
    • State-of-the-art model coverage: Added FlowGRPO and DiffusionNFT training for MiniMax H3 (T2VA, FL2VA, Ref2VA) on GPU and Ascend NPU, BOOGU-Image (T2I, Edit, and a CFG-free DiffusionNFT Pick-a-Pic recipe), FLUX.1-dev DanceGRPO, LTX-2.3 FlowGRPO and OmniNFT (T2AV), MiniCPM-o 4.5 (Thinker GSPO, separate-async GSPO, and offline DPO), Qwen3-TTS GRPO, Qwen3-Omni Megatron Geo3K and AVQA recipes, DGPO on SD3.5, and VeOmni backend acceleration. Ascend A5 joins A2 and A3.

    Architecture

    Rollout Engine
    • Upgraded the rollout engine to vLLM 0.30.0 and vLLM-Omni commit 12e9280 (v0.30.0), with Python >= 3.11, diffusers >= 0.40.0, and kernels 0.16.0. This release includes the v0.28 lifecycle upgrade (ded89346, rebased on vLLM 0.28.0) and then moves the pin to v0.30.0. The v0.30 engine accepts only explicitly set engine arguments, selects the paged KV scheduler for diffusion prefix caching, and disables prefix caching when sleep mode is on. (#363, #497, #688)
    • Overhauled the async rollout server lifecycle with ACK-validated delegated sleep/wake, abort-then-pause request cancellation, and frontend multimodal cache resets to prevent zombie requests and silent drift. IPC weight-update failures now propagate instead of being dropped. (#460, #497, #735)
    • Added fully-async autoregressive rollout support for Qwen3-Omni across both text-only full models and multimodal audio inputs with Megatron sharding. (#290, #413)
    • Added prompt embedding cache for diffusion rollouts to avoid redundant prompt encoding across steps. (#356)
    • Unified the diffusion rollout request contract, prompt wire schema, and media output declarations via DiffusionIOSpec. (#478, #480, #481, #508, #556)
    • Added named media artifacts. In-tree diffusion adapters declare image, video, audio, and latent outputs by name; strategy, agent loops, rewards, and V0/V1 exporters consume those declarations instead of guessing modality from tensor rank. The public generate(**kwargs) RPC is unchanged. Unlabelled tensor or tuple results are rejected. (#557)
    • Added the first multi-turn agentic image-generation loop: a tool trajectory from prompt through tool calls to Done., a decoupled agent–tool rollout, and an image-judge reward. (#409)
    • Added dual logprob return support for Qwen-Image rollouts. (#381)
    • Standardized diffusion rollout pixel responses to always use uint8 tensors. (#373)
    • Supported refilling inco
  2. v0.3.0rc1pre-release2026-09-30 · 2026-09-30 08:47 UTC
    查看更新说明

    Highlights

    VeRL-Omni v0.3.0rc1 focused on four themes:

    • Fully-async & decoupled RL execution: First-class support for fully-async and separate-async training across both autoregressive and diffusion pipelines, including Qwen3-Omni audio and text fully-async RL, the dedicated Omni separate-async V1 trainer, dynamic hybrid rollout replica switching, high-performance omni_delta_sharded weight synchronization, and async teacher scheduling for Diffusion OPD.
    • Default V1 trainer architecture: The verl V1 trainer (TransferQueue + ReplayBuffer) is now enabled by default across all models with fail-closed dynamic scheduling, optimized handoff, and synchronized/separate-async workflows, while the legacy v0 trainer is officially deprecated.
    • Rollout engine upgrade & lifecycle reliability: Upgraded the rollout stack to vLLM-Omni v0.28 (rebased on vLLM 0.28.0) with Python 3.11+ support, robust delegated async lifecycle controls (abort-then-pause request cancellation, ACK-validated sleep/wake), diffusion prompt embedding caching, rollout sequence parallelism (SP) & VAE parallelism, and strict fail-fast attention backend checks.
    • State-of-the-art model coverage: Added FlowGRPO and DiffusionNFT training for MiniMax H3 (T2VA, FL2VA, Ref2VA) on GPU and Ascend NPU, BOOGU-Image (T2I & Edit), FLUX.1-dev DanceGRPO, LTX-2.3 TI2VA, MiniCPM-o 4.5 Thinker GSPO, Qwen3-TTS GRPO, and VeOmni backend acceleration.

    Architecture

    Rollout Engine
    • Upgraded the rollout engine to vLLM-Omni v0.28 snapshot (ded89346) rebased on vLLM 0.28.0 with Python >= 3.11 floor, diffusers >= 0.40.0, and kernels 0.16.0. (#363, #497)
    • Overhauled the async rollout server lifecycle with ACK-validated delegated sleep/wake, abort-then-pause request cancellation, and frontend multimodal cache resets to prevent zombie requests and silent drift. (#460, #497)
    • Added fully-async autoregressive rollout support for Qwen3-Omni across both text-only full models and multimodal audio inputs with Megatron sharding. (#290, #413)
    • Added prompt embedding cache for diffusion rollouts to avoid redundant prompt encoding across steps. (#356)
    • Unified the diffusion rollout request contract, prompt wire schema, and media output declarations via DiffusionIOSpec. (#478, #480, #481, #508, #556)
    • Added dual logprob return support for Qwen-Image rollouts. (#381)
    • Standardized diffusion rollout pixel responses to always use uint8 tensors. (#373)
    • Supported refilling incomplete diffusion rollout groups to stabilize training batch sizes. (#358)
    • Preserved text-encoder tensor parallelism and fixed co-located ZMQ handle rank assignment via CUDA_VISIBLE_DEVICES in multi-stage deployments. (#483, #589)
    • Exposed diffusion rollout sequence parallelism (SP: Ulysses / Ring) and VAE patch parallelism / tiling controls (rollout.sp_size, rollout.vae_parallel_size), supporting local rank allocation across V0, V1 sync, and separate-async workflows with fail-closed hook validation. (#656)
    Training
    • Defaulted the V1 trainer (trainer.use_v1=true) with TransferQueue and ReplayBuffer across all diffusion models, officially deprecating the legacy v0 trainer. (#650)
    • Added the Omni separate-async V1 trainer (OmniSeparateAsyncRayTrainer), enabling decoupled actor and rollout execution for multimodal AR models. (#380)
    • Added omni_delta_sharded weight synchronization for diffusion separate-async training, drastically reducing parameter broadcast overhead to decoupled rollout workers. (#613)
    • Added hybrid rollout switching for diffusion V1 separate-async, dynamically lending colocated actor replicas to rollout when the replay buffer runs low and reclaiming them for training. (#575)
    • Added async teacher scheduling and standalone teacher routing for Diffusion On-Policy Distillation (OPD) on V1 separate-async. (#427, #493, #495)
    • Added fail-closed dynamic scheduling, optimized TransferQueue handoff, and restored rollout batch o
  3. v0.2.02026-08-13 · 2026-08-13 09:35 UTC
    查看更新说明

    Highlights

    VeRL-Omni v0.2.0 focused on three themes:

    • Faster diffusion RL: diffusion GRPO training is significantly faster with request-level diffusion rollout batching and FA3 rollout kernels.
    • Stable Qwen3-Omni training: Qwen3-Omni support is rebuilt on a refactored omni model base and the verl v1 trainer, with full multimodal inputs (text, image, and audio) for DPO & GSPO training support.
    • State-of-the-art model coverage: added FlowGRPO training for LTX-2.3, the SoTA text-to-audio-video model, plus Qwen-Image-Edit support for image editing, and more.

    Architecture

    Rollout Engine
    • Upgraded the rollout stack to vLLM-Omni v0.24 and pinned the companion dependency versions. (#257)
    • Migrated the Diffusion Trainer to the verl v1 architecture, covering both the sync trainer and a separate async rollout path. (#296,#315)
    • Added request-level diffusion batching with a LoRA weight-update fix, plus rank-local ZMQ route preservation for LoRA updates. (#286,#349)
    • Added FA3 kernel support for diffusion rollout. (#261)
    • Integrated Qwen-Image step execution into FlowGRPO and MixGRPO, and added NFT and DPO step execution support. (#287,#310)
    • Refactored the agent loop to a multi-tokenizer prompt path for SD3.5. (#288)
    • Fixed rollout correctness issues around diffusion async semantics, request-batched diffusion log-probs, and final prompt embedding length. (#339,#334,#357)
    Training
    • Added teacher-anchored continuous distillation losses for diffusion OPD. (#300)
    • Added rollout-train log-prob consistency monitoring, logged by default, with a dedicated v1 consistency monitor. (#291,#319)
    • Added rollout-server and reward-model-server profiling, including a lightweight profiling recipe for FlowGRPO. (#252,#256)
    • Rollout and validation videos are now saved as mp4, and wandb validation video media persists correctly. (#311,#330)
    • Supported BAGEL non-LoRA training with PickScore reward. (#218)
    • Mitigated CPU memory leaks in data serialization over Ray. (#259)
    Reward
    • Added DiNa-LRM reward support for SD3.5 FlowGRPO. (#289)
    • Added batched PickScore inference for reward scoring. (#346)
    • Added CLAP and ImageBind rewards as part of the LTX-2.3 text-to-audio-video recipe. (#341)
    • Dropped duplicate reward score metrics in single- and multi-reward training. (#242,#246)

    Model & Algorithm Supports

    • Reworked Qwen3-Omni on the verl v1 sync trainer, refactored the omni model base, and added FSDP2 training with text+image inputs. (#253,#258,#312)
    • Added the Omni DPO stack: an offline MLLM preference dataset pipeline plus Omni DPO config and OmniDPOLoss. (#268,#270)
    • Added Qwen-Image-Edit FlowGRPO support and a general image-editing interface. (#238,#260)
    • Added LTX-2.3 text-to-audio-video FlowGRPO training with CLAP / ImageBind rewards. (#341)
    • Added repo-native BAGEL atomic SFT primitives. (#333)
    • Added generic visual-reflection contracts and source adapters. (#313)

    Hardware

    • Added Ascend NPU Dockerfiles and an install guide; NPU Docker builds now read vLLM backend refs from pins. (#225,#267)
    • Extended Qwen3-Omni Thinker to multimodal inputs (image, text, audio -> text) on NPU, and added the MMK12 image2text GSPO recipe on NPU. (#284,#231)

    Documentation / Tooling

    • Added the L3 Qwen-Image FlowGRPO nightly regression workflow with reviewed baselines. (#303,#344)
    • Split GPU smoke CI into label-driven test groups and added L1 CPU coverage reporting. (#233,#263)
    • Added contributor-facing agent rules and skills under .agents/. (#323)
    • Refreshed docs for config explanation, examples, Qwen-Image-Edit / I2I integration, reward-server profiling, and stale setup guidance. (#324,#316,#273,#281,#318)
    • Deprecated the old omni scripts. (#359)

    Breaking Changes

    • The Diffusion Trainer and Qwen3-Omni stacks now run on the verl v1 trainer architecture. Custom trainer / rollout code built against the pre-v1 interfaces should migrate before u
  4. v0.2.0rc12026-08-07 · 2026-08-07 06:35 UTC
    查看更新说明

    Highlights

    This release includes 70 merged PRs from 15 contributors since v0.1.0. VeRL-Omni v0.2.0rc1 is the first release candidate of the v0.2.0 line, focused on three themes:

    • Faster diffusion RL: diffusion GRPO training is significantly faster with request-level diffusion rollout batching and FA3 rollout kernels.
    • Stable Qwen3-Omni training: Qwen3-Omni support is rebuilt on a refactored omni model base and the verl v1 trainer, with full multimodal inputs (text, image, and audio) for DPO & GSPO training support.
    • State-of-the-art model coverage: added FlowGRPO training for LTX-2.3, the SoTA text-to-audio-video model, plus Qwen-Image-Edit support for image editing, and more.

    Architecture

    Rollout Engine
    • Upgraded the rollout stack to vLLM-Omni v0.24 and pinned the companion dependency versions. (#257)
    • Migrated the Diffusion Trainer to the verl v1 architecture, covering both the sync trainer and a separate async rollout path. (#296,#315)
    • Added request-level diffusion batching with a LoRA weight-update fix, plus rank-local ZMQ route preservation for LoRA updates. (#286,#349)
    • Added FA3 kernel support for diffusion rollout. (#261)
    • Integrated Qwen-Image step execution into FlowGRPO and MixGRPO, and added NFT and DPO step execution support. (#287,#310)
    • Refactored the agent loop to a multi-tokenizer prompt path for SD3.5. (#288)
    • Fixed rollout correctness issues around diffusion async semantics, request-batched diffusion log-probs, and final prompt embedding length. (#339,#334,#357)
    Training
    • Added teacher-anchored continuous distillation losses for diffusion OPD. (#300)
    • Added rollout-train log-prob consistency monitoring, logged by default, with a dedicated v1 consistency monitor. (#291,#319)
    • Added rollout-server and reward-model-server profiling, including a lightweight profiling recipe for FlowGRPO. (#252,#256)
    • Rollout and validation videos are now saved as mp4, and wandb validation video media persists correctly. (#311,#330)
    • Supported BAGEL non-LoRA training with PickScore reward. (#218)
    • Mitigated CPU memory leaks in data serialization over Ray. (#259)
    Reward
    • Added DiNa-LRM reward support for SD3.5 FlowGRPO. (#289)
    • Added batched PickScore inference for reward scoring. (#346)
    • Added CLAP and ImageBind rewards as part of the LTX-2.3 text-to-audio-video recipe. (#341)
    • Dropped duplicate reward score metrics in single- and multi-reward training. (#242,#246)

    Model & Algorithm Supports

    • Reworked Qwen3-Omni on the verl v1 sync trainer, refactored the omni model base, and added FSDP2 training with text+image inputs. (#253,#258,#312)
    • Added the Omni DPO stack: an offline MLLM preference dataset pipeline plus Omni DPO config and OmniDPOLoss. (#268,#270)
    • Added Qwen-Image-Edit FlowGRPO support and a general image-editing interface. (#238,#260)
    • Added LTX-2.3 text-to-audio-video FlowGRPO training with CLAP / ImageBind rewards. (#341)
    • Added repo-native BAGEL atomic SFT primitives. (#333)
    • Added generic visual-reflection contracts and source adapters. (#313)

    Hardware

    • Added Ascend NPU Dockerfiles and an install guide; NPU Docker builds now read vLLM backend refs from pins. (#225,#267)
    • Extended Qwen3-Omni Thinker to multimodal inputs (image, text, audio -> text) on NPU, and added the MMK12 image2text GSPO recipe on NPU. (#284,#231)

    Documentation / Tooling

    • Added the L3 Qwen-Image FlowGRPO nightly regression workflow with reviewed baselines. (#303,#344)
    • Split GPU smoke CI into label-driven test groups and added L1 CPU coverage reporting. (#233,#263)
    • Added contributor-facing agent rules and skills under .agents/. (#323)
    • Refreshed docs for config explanation, examples, Qwen-Image-Edit / I2I integration, reward-server profiling, and stale setup guidance. (#324,#316,#273,#281,#318)
    • Deprecated the old omni scripts. (#359)

    Breaking Changes

    • The Diffusion Trainer and Qwen3-Omni stacks now run
  5. v0.1.02026-07-03 · 2026-07-03 07:08 UTC
    查看更新说明

    Highlights

    This release includes 150 merged PRs from 14 contributors.

    VeRL-Omni v0.1.0 is the first release of the dedicated multimodal generative RL training repository, after the pre-release version v0.1.0rc1. It establishes a runnable stack for diffusion, unified multimodal, and omni-modality recipes, while strengthening the rollout engine, trainer backends, reward workflows, hardware support, and validation docs.

    This is the first tagged final release since multimodal generative RL training moved into a dedicated VeRL-Omni repository.

    Architecture

    Rollout Engine
    • Upgraded the rollout stack to vLLM-Omni v0.22.0 and aligned the release around the companion vLLM, vLLM-Omni, and verl dependency versions. (#166, #167)
    • Improved rollout execution with step-wise batching co-execution, experimental rollout correction, faster LoRA weight updates, and configurable diffusion rollout attention backend selection. These changes reduce friction from trajectory generation through actor-to-rollout synchronization and help catch attention-backend mismatches before large runs. (#81, #93, #156, #200)
    • Hardened autoregressive rollout paths used by Qwen3-Omni by guarding rollout attention backend handling and refreshing processor exports / worker processor loading after model patches. (#211, #224)
    Training
    • Moved the GPU actor path toward FA3 attention while preserving native and SDPA fallbacks when FA3 dependencies are unavailable. (#141, #165)
    • Expanded trainer backend options with FSDP sequence parallelism, optional VeOmni actor/reference engines, and corrected FSDP2 launch scripts so recipe scripts actually select true FSDP2 training. (#59, #104, #216)
    • Added trainer-side observability and reproducibility tools, including diffusion MFU metrics and deterministic per-step / per-rollout seeding. (#60, #128, #136)
    Reward
    • Added multi-reward weighted aggregation so a run can combine multiple scoring signals through configurable reward functions and managers. Follow-up fixes support dynamic loading and relative imports in the multi-reward manager, which makes custom scorer packaging easier for recipe authors. (#109, #228)
    • Added external HTTP scorer support and async reward documentation, making it easier to serve expensive reward models separately and overlap reward scoring with rollout. (#116, #155)

    Model & Algorithm Supports

    • Added the first Qwen-Image RL recipe set, covering FlowGRPO, Flow-DPPO, MixGRPO, GRPO-Guard, DiffusionNFT, and online DPO. These recipes make Qwen-Image the main text-to-image coverage anchor for the first final release. (#48, #58, #106, #126, #139, #164, #168, #202)
    • Expanded diffusion generator coverage with SD3.5 offline DPO / FlowGRPO and Wan2.2 DanceGRPO text-to-video training. The quickstart and example docs were refreshed around these diffusion recipes. (#95, #98, #127, #142, #178, #204)
    • Added BAGEL FlowGRPO support for unified understanding-and-generation training, including vLLM-Omni v0.22 alignment and OCR / PickScore reward variants. (#132, #137, #180, #209, #212)
    • Added Qwen3-Omni Thinker GSPO + LoRA with vLLM-Omni async autoregressive rollout, then extended it with Transformers 5.x LoRA FSDP validation, smoke-test wiring, and processor-loading fixes. (#113, #208, #224)

    Hardware

    • Added Ascend NPU support for Qwen-Image FlowGRPO, including NPU-oriented launch scripts and quickstart guidance for Atlas 800T A2-style setups. (#68, #85, #181, #202)
    • Extended NPU recipe cove

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    VeRL-Omni

    Easy, fast, and stable RL training for diffusion and omni-modality models

    Docs License

    English | 简体中文

    VeRL-Omni is a general RL training framework focused on multimodal generative models, built on top of verl.

    It originated from the multi-modal generation RL effort in verl, and now has a dedicated home so it can evolve in a more focused way.

    News 🔥

    • [2026-10] 🎉 We have released v0.3.0 for fully async diffusion/omni RL with the v1 trainer as the default.
    • [2026-10] OmniNFT is now supported for audio-video post-training. The LTX2.3 text-to-video+audio LoRA recipe runs on Ascend NPU with modality-wise advantage routing.
    • [2026-09] The v1 trainer (TransferQueue + ReplayBuffer) is now the default for diffusion models; the legacy v0 trainer is deprecated, while offline DPO remains on v0 by design.
    • [2026-08] DiffusionOPD (on-policy distillation, including multi-teacher MOPD) is now supported.
    • [2026-08] 🔥 MiniMax-H3 now supports T2VA, FL2VA, and Ref2VA with both FlowGRPO and DiffusionNFT.
    • [2026-08] 🎉 We have released v0.2.0 for faster diffusion rl and more stable Qwen3-Omni multimodal training. Blog: VeRL-Omni v0.2.0
    • [2026-08] LTX2.3 text-to-video+audio model is now supported with FlowGRPO.
    • [2026-07] Team-proposed algorithm FlowGRPO with DiNa-LRM is available. Training skips VAE decoding by scoring clean diffusion latents directly for faster and more resource-efficient model alignment.
    • [2026-07] VeRL-Omni is presented in QingKe AI, vLLM community, and verl x Ascend Beijing meetup. Slides are shared.
    • [2026-06] Qwen3-Omni GSPO Trainer is available! Flow-DPPO is integrated. vLLM-Omni rollout backend is upgraded to v0.22 for higher throughput, with default actor attn backend switched to FA3.
    • [2026-06] DiffusionNFT and Diffusion DPO are integrated with verified recipes on Qwen-Image/SD3.5. Wan2.2 is now supported for video generation tasks.

    Why VeRL-Omni

    Multimodal generative RL training differs from text-only LLM RL not only in model structure, but also in I/O patterns, compute characteristics, and runtime bottlenecks. As this space grows, it deserves a dedicated training repository that can evolve quickly around its own constraints.

    Scope

    VeRL-Omni targets RL post-training for three families of generative models:

    1. Diffusion generative models for image, video, and audio — e.g., Qwen-Image, Wan2.2.
    2. Unified multimodal understanding + generation models — e.g., BAGEL, HunyuanImage-3.0.
    3. Omni-modality models that jointly handle text, image, audio, and video — e.g., Qwen3-Omni.

    What we focus on

    • Fast multi-modal rollout: Adopt vLLM-Omni backend and accelerate generation via rollout routing, rollout batching, embed caching optimizations, and more.
    • Flexible & async multi-reward serving: Support multi-reward serving (HPSv3, GenRM-OCR, UnifiedReward, etc.), HTTP scorer, and asynchronous reward computation to overlap the rollout phase.
    • Modular training backends: Selectable VeOmni and FSDP2 backends with combinable parallelism (USP/TP/DP) for distributed training.
    • Stability: Boost stability and speed in diffusion RL pipelines via rollout correction to skip logP recomputation, and achieve reproducible E2E training with deterministic RL. Reward, rollout and actor update are composable and extensible, via Hydra configs.
    • Efficient and convergent training recipes: On our reference Qwen-Image FlowGRPO setup, VeRL-Omni achieves ~25% higher end-to-end throughput than the diffusers-based flow_grpo implementation, driven by vLLM-Omni rollout, FSDP2 trainer, overlapped reward computation (asynchronous), etc.

    Getting Started 🚀

    Visit our documentation to learn more.

    Model and Algorithm Support 🎨

    Ascend NPU Support 💠

    VeRL-Omni supports Ascend A2, A3, and A5 NPUs. See the NPU installation guide for setup and the Ascend NPU Quickstart Guide for FlowGRPO training.

    Roadmap 🗺

    Future work is tracked in VeRL-Omni Q4 Roadmap

    Contributing 🤝

    Contributions are welcome.

    See the contribution guide.

    Join the Community: Feel free to ask questions, provide feedback, and discuss with fellow users of VeRL-Omni in our WeChat group.

    Acknowledgement 🌟

    verl-omni builds on the engineering foundations developed in verl and is closely aligned with multimodal inference systems such as vLLM-Omni.

    Citation 📚

    If you find the project helpful, please cite and star ⭐

    @misc{verlomni_github,
  title        = {{VeRL-Omni: Easy, Fast, and Stable RL Training for Diffusion and Omni-Modality Models}},
  author       = {Yongxiang Huang and Cheung Kawai and Jingan Zhou and Yingshu Chen and {openYuanrong Team} and Xibin Wu},
  year         = {2026},
  howpublished = {\url{https://github.com/verl-project/verl-omni}},
  urldate      = {2026-04-28}
}

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    stable-diffusion-webui
    AUTOMATIC1111·Python

    基于 Gradio 的 Stable Diffusion 网页界面，把文生图、图生图、局部重绘等能力做成可视化操作面板，不用写代码就能跑图像生成。

    图像生成+4图像生成进阶
    165290+4431kv1.10.12026-03-02 07:00 UTC
    ComfyUI
    Comfy-Org·Python

    基于节点图的扩散模型图形界面与后端，支持本地快速推理和 API 调用，方便搭建和调试图像生成工作流。

    扩散模型+4图像生成进阶
    146237+75117kv0.45.02026-11-11 05:07 UTC
    OpenMontage
    OpenMontageInc·Python

    把 Claude、Cursor 等 AI 编程助手变成完整视频制作工作室，内置 12 条制作流水线和 100+ 工具，覆盖脚本、配音、图像生成到剪辑全流程。

    视频制作+4图像生成硬核
    79488+83411k2026-11-09 06:23 UTC
    OpenMontage
    calesthio·Python

    把 Claude、Cursor 等 AI 编程助手变成完整的视频制作工作室，内置 12 条制作流水线和 100+ 工具，覆盖从生成到剪辑的全流程。

    视频制作+4图像生成硬核
    7786710k2026-11-03 17:42 UTC