ShapeLLM-Omni

JAMESYJLMIT长期未更新

原生多模态大模型，把文本和图像统一到 3D 生成与理解任务中，支持文生 3D、图生 3D、3D 描述和编辑。

[NeurIPS 2025 Spotlight] A Native Multimodal LLM for 3D Generation and Understanding

暂无版本581star31fork同步于 2026-11-07 07:32 UTC

多模态大模型 3D生成文生3D 图生3D 3D编辑

对比 GitHub 官网 3d-captioning 3d-editing image-to-3d llm text-to-3d

怎么上手

由 README 提炼，执行前请确认

原文

pip install -r requirements.txt
python app.py

硬件要求按 README 说明

README未说明显存与GPU型号要求，仅建议按TRELLIS和Qwen2.5-VL的环境配置准备

下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼

没人工点评？写下你的经验

原生多模态大模型，统一做3D生成与3D理解，7B权重已开源

原生多模态大模型，把文本和图像统一到 3D 生成与理解任务中，支持文生 3D、图生 3D、3D 描述和编辑。

核心能力

原生多模态LLM同时处理3D生成与3D理解任务
开源ShapeLLM-Omni 7B与3DVQVAE预训练权重
提供50k高质量3D编辑数据对
附带Gradio可视化推理界面与任务模板
基于TRELLIS与Qwen2.5-VL构建

容易踩的坑

环境需同时满足TRELLIS与Qwen2.5-VL的依赖，直接装requirements可能不全；需从HuggingFace下载ShapeLLM-7B-omni权重才能推理；训练代码与完整3D-Alpaca数据集尚未发布

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

用户点评

登录用户的真实使用记录，可按版本和硬件筛选

共 0 条

还没有人点评。第一个说出真实体验的人最有价值。

README 原文

未翻译

在 GitHub 打开

https://github.com/user-attachments/assets/f77bb981-15ef-4546-ae1a-9baf05dc8002

Release

[6/03] 🔥🔥We released the pretrained weights for both ShapeLLM-Omni (7B) and 3DVQVAE.
[6/03] 🔥🔥We released 50k high-quality 3D edited data pairs.
[6/07] 🔥🔥We built a demo for everyone to try out.

Installation

Please set up the Python environment following TRELLIS and QWEN2.5-vl, or you can create by:

pip install -r requirements.txt

Inference

We suggest using Gradio UI for visualizing inference.

python app.py

https://github.com/user-attachments/assets/edb2b828-b65c-40f6-88da-9f5094c40b2e

For templates used for different tasks, please refer to the templates.txt

Qualitative result

https://github.com/user-attachments/assets/79a33188-3ef0-4702-9892-15b864710f2d

https://github.com/user-attachments/assets/43b7bc78-1bef-4b79-bbdb-edfc4ad2b8e1

Important Notes

Please refer to our project_page for more examples.

Todo

Release of the entire 3D-Alpaca dataset.
Release of training code.
Release of model weights featuring multi-turn dialogue and 3D editing capabilities.

Acknowledgement

Our code is based on these wonderful repos:

Also, we invite you to explore our latest work — Nano3D, a training-free 3D editing algorithm without mask constraints. Based on this algorithm, we will soon release a higher-quality 3D editing dataset — 3D-Alpaca-Editing-v2 (Nano3D-Edit-100k) — as open source.

✍️ Citation

@article{ye2025shapellm,
  title={ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding},
  author={Ye, Junliang and Wang, Zhengyi and Zhao, Ruowen and Xie, Shenghao and Zhu, Jun},
  journal={arXiv preprint arXiv:2506.01853},
  year={2025}
}