NEO

EvolvingLMMs-LabApache-2.0活跃维护

从零训练原生视觉语言模型的开源系列,去掉独立视觉编码器,让多模态理解与生成在统一架构里完成。

NEO Series: Native Vision-Language Models from First Principles

暂无版本925star +4 / 7 天33fork同步于 2026-11-10 11:52 UTC

怎么上手

由 README 提炼,执行前请确认
README 未给出安装命令,仅提供各子模块指南:VLMEvalKit/README.md、VLMEvalKit_ov/README.md、VLMTrainKit/README.md、VLAKit/README.md
硬件要求按 README 说明

README 未说明

下载:源码 zip·GitHub Releases(安装包)本站不镜像安装包

项目说明

由 README 自动翻译提炼

NEO 系列是原生视觉语言模型,用无编码器架构统一像素与词的理解

从零训练原生视觉语言模型的开源系列,去掉独立视觉编码器,让多模态理解与生成在统一架构里完成。

核心能力

  • 在无编码器的稠密架构中统一像素-词编码、对齐与推理
  • 用不到 10% 的图像-文本数据从零训练出强视觉感知
  • 提供 2B 与 9B 多阶段权重及 2B NEO-VLA 策略模型
  • 配套训练、评测与 VLA 训练代码,组件可复用

容易踩的坑

权重需从 Hugging Face 单独下载,仓库不含模型文件;训练、评测、VLA 各有独立指南,需按对应子目录 README 操作;README 未给出依赖版本与运行环境要求,需自行确认

内容由模型根据 README 生成,可能不完整或不准确,以官方文档为准。看原文 README

最近发版

0 个版本

暂无 release 数据(可能没有发版,或尚未同步)。

用户点评

登录用户的真实使用记录,可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译

    NEO Series: Native Vision-Language Models

    📜 News

    [2026/11] 🔥 The paper, project page, weights, dataset, and training code of NEO-VLA are released !
    [2026/05] 🔥 The paper, weights, and evaluation code of NEO-ov are released !
    [2025/12] 💥 NEO-ov has been completed !
    [2026/01] 🔥 The training code of NEO is released !
    [2025/10] 🔥 The paper, weights, and evaluation code of NEO are released !
    [2025/09] 💥 NEO has been completed !

    📋 Todo List

    🤖 Model Zoo

    We release 2B and 9B NEO-series models across different training stages, and the 2B NEO-VLA policy with its action pretraining dataset.

    💡 Motivation

    • What constraints set native VLMs apart from modular ones, and to what extent can they be overcome?

    • How to make native VLMs more accessible and democratized, thereby accelerating their progress?

    💡 Highlights

    • 🔥 Native Architecture: NEO innovates a native VLM primitive that unifies pixel-word encoding, alignment, and reasoning within an encoder-free dense model architecture.

    • 🔥 Superior Efficiency: Using under 10% of image-text data, NEO develops strong visual perception from scratch, rivaling top-tier modular VLMs and outperforming native encoder-free ones.

    • 🔥 Promising Roadmap: NEO pioneers a promising route for scalable and powerful native encoder-free VLMs, paired with diverse reusable components that foster a cost-effective and extensible ecosystem.

    🚀 Performance

    🧠 Single-Image Understanding
    🎬 Multi-Image & Video Understanding
    🌐 Spatial Intelligence

    ✒️ Citation

    If NEO series is helpful for your research, please consider star ⭐ and citation 📝 :

    @article{Diao2025NEO,
      title        = {From Pixels to Words--Towards Native Vision-Language Primitives at Scale},
      author       = {Diao, Haiwen and Li, Mingxuan and Wu, Silei and Dai, Linjun and Wang, Xiaohua and Deng, Hanming and Lu, Lewei and Lin, Dahua and Liu, Ziwei},
      journal      = {arXiv preprint arXiv:2510.14979},
      year         = {2025}
    }
    
    @article{Diao2026NEOov,
      title        = {From Pixels to Words--Towards Native One-Vision Models at Scale},
      author       = {Diao, Haiwen and Wang, Jiahao and Wu, Penghao and Dong, Yuhao and Niu, Yuwei and Zhu, Yue and Cai, Zhongang and Fan, Weichen and Dai, Linjun and Wu, Silei and others},
      journal      = {arXiv preprint arXiv:2605.28820},
      year         = {2026}
    }
    
    @misc{sensenova2026neounify,
      title        = {NEO-unify: Building Native Multimodal Unified Models End to End},
      author       = {SenseNova},
      journal      = {Hugging Face blog},
      url          = {https://huggingface.co/blog/sensenova/neo-unify},
      year         = {2026}
    }
    
    @article{sensenova2026sensenovau1,
      title        = {SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture},
      author       = {Diao, Haiwen and Wu, Penghao and Deng, Hanming and Wang, Jiahao and Bai, Shihao and Wu, Silei and Fan, Weichen and Ye, Wenjie and Tong, Wenwen and Fan, Xiangyu and others},
      journal      = {arXiv preprint arXiv:2605.12500},
      year         = {2026}
    }
    
    @article{Fan2026NEOVLA,
      title        = {From Pixels to Actions--Towards Native Vision-Language-Action Policies},
      author       = {Fan, Weichen and Diao, Haiwen and Wu, Penghao and Hu, Siyuan and Huang, Ziqi and Liu, Tianqi and Sun, Mingze and Wang, Ziyi and Cai, Zhongang and Deng, Hanming and Wang, Quan and Yang, Lei and Lu, Lewei and Hong, Fangzhou and Chen, Zhaoxi and Lin, Dahua and Liu, Ziwei},
      journal      = {arXiv preprint arXiv:2611.06918},
      year         = {2026}
    }
    

    📄 License

    The content of this project itself is licensed under LICENSE.