权重需从 Hugging Face 单独下载,仓库不含模型文件;训练、评测、VLA 各有独立指南,需按对应子目录 README 操作;README 未给出依赖版本与运行环境要求,需自行确认

NEO
EvolvingLMMs-LabApache-2.0活跃维护从零训练原生视觉语言模型的开源系列,去掉独立视觉编码器,让多模态理解与生成在统一架构里完成。
NEO Series: Native Vision-Language Models from First Principles
README 未给出安装命令,仅提供各子模块指南:VLMEvalKit/README.md、VLMEvalKit_ov/README.md、VLMTrainKit/README.md、VLAKit/README.mdREADME 未说明
NEO 系列是原生视觉语言模型,用无编码器架构统一像素与词的理解
从零训练原生视觉语言模型的开源系列,去掉独立视觉编码器,让多模态理解与生成在统一架构里完成。
核心能力
- 在无编码器的稠密架构中统一像素-词编码、对齐与推理
- 用不到 10% 的图像-文本数据从零训练出强视觉感知
- 提供 2B 与 9B 多阶段权重及 2B NEO-VLA 策略模型
- 配套训练、评测与 VLA 训练代码,组件可复用
容易踩的坑
暂无 release 数据(可能没有发版,或尚未同步)。
用户点评
登录用户的真实使用记录,可按版本和硬件筛选还没有人点评。第一个说出真实体验的人最有价值。
NEO Series: Native Vision-Language Models
-
2026/11: From Pixels to Actions -- Towards Native Vision-Language-Action Policies (Arxiv 2026)
-
2026/04: SenseNova-U1 with NEO-unify Architecture (Technical Report 2026)
-
2026/02: NEO-unify: Building Native Multimodal Unified Models End to End (HuggingFace Blog 2026)
-
2025/12: From Pixels to Words -- Towards Native One-Vision Models at Scale (Arxiv 2026)
-
2025/09: From Pixels to Words -- Towards Native Vision-Language Primitives at Scale (ICLR 2026)
📜 News
[2026/11] 🔥 The paper, project page, weights, dataset, and training code of NEO-VLA are released !
[2026/05] 🔥 The paper, weights, and evaluation code of NEO-ov are released !
[2025/12] 💥 NEO-ov has been completed !
[2026/01] 🔥 The training code of NEO is released !
[2025/10] 🔥 The paper, weights, and evaluation code of NEO are released !
[2025/09] 💥 NEO has been completed !
📋 Todo List
🤖 Model Zoo
We release 2B and 9B NEO-series models across different training stages, and the 2B NEO-VLA policy with its action pretraining dataset.
💡 Motivation
-
What constraints set native VLMs apart from modular ones, and to what extent can they be overcome?
-
How to make native VLMs more accessible and democratized, thereby accelerating their progress?
💡 Highlights
-
🔥 Native Architecture: NEO innovates a native VLM primitive that unifies pixel-word encoding, alignment, and reasoning within an encoder-free dense model architecture.
-
🔥 Superior Efficiency: Using under 10% of image-text data, NEO develops strong visual perception from scratch, rivaling top-tier modular VLMs and outperforming native encoder-free ones.
-
🔥 Promising Roadmap: NEO pioneers a promising route for scalable and powerful native encoder-free VLMs, paired with diverse reusable components that foster a cost-effective and extensible ecosystem.
🚀 Performance
🧠 Single-Image Understanding
🎬 Multi-Image & Video Understanding
🌐 Spatial Intelligence
✒️ Citation
If NEO series is helpful for your research, please consider star ⭐ and citation 📝 :
@article{Diao2025NEO,
title = {From Pixels to Words--Towards Native Vision-Language Primitives at Scale},
author = {Diao, Haiwen and Li, Mingxuan and Wu, Silei and Dai, Linjun and Wang, Xiaohua and Deng, Hanming and Lu, Lewei and Lin, Dahua and Liu, Ziwei},
journal = {arXiv preprint arXiv:2510.14979},
year = {2025}
}
@article{Diao2026NEOov,
title = {From Pixels to Words--Towards Native One-Vision Models at Scale},
author = {Diao, Haiwen and Wang, Jiahao and Wu, Penghao and Dong, Yuhao and Niu, Yuwei and Zhu, Yue and Cai, Zhongang and Fan, Weichen and Dai, Linjun and Wu, Silei and others},
journal = {arXiv preprint arXiv:2605.28820},
year = {2026}
}
@misc{sensenova2026neounify,
title = {NEO-unify: Building Native Multimodal Unified Models End to End},
author = {SenseNova},
journal = {Hugging Face blog},
url = {https://huggingface.co/blog/sensenova/neo-unify},
year = {2026}
}
@article{sensenova2026sensenovau1,
title = {SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture},
author = {Diao, Haiwen and Wu, Penghao and Deng, Hanming and Wang, Jiahao and Bai, Shihao and Wu, Silei and Fan, Weichen and Ye, Wenjie and Tong, Wenwen and Fan, Xiangyu and others},
journal = {arXiv preprint arXiv:2605.12500},
year = {2026}
}
@article{Fan2026NEOVLA,
title = {From Pixels to Actions--Towards Native Vision-Language-Action Policies},
author = {Fan, Weichen and Diao, Haiwen and Wu, Penghao and Hu, Siyuan and Huang, Ziqi and Liu, Tianqi and Sun, Mingze and Wang, Ziyi and Cai, Zhongang and Deng, Hanming and Wang, Quan and Yang, Lei and Lu, Lewei and Hong, Fangzhou and Chen, Zhaoxi and Lin, Dahua and Liu, Ziwei},
journal = {arXiv preprint arXiv:2611.06918},
year = {2026}
}
📄 License
The content of this project itself is licensed under LICENSE.