仓库本身是论文索引，不含可运行代码，需按各论文的 GitHub 链接单独获取实现；表格中的硬件栏指原论文实验所用设备，不代表本仓库要求；部分论文仅有 arXiv 链接而无开源代码
FM-AD-Survey
TUM-AVSApache-2.0活跃维护汇总自动驾驶场景生成与分析方向的基础模型论文，涵盖扩散模型、大语言模型和多模态模型，持续更新最新研究进展。
[Survey Paper] This repository collects research papers of large Foundation Models for Scenario Generation and Analysis in Autonomous Driving. The repository will be continuously updated to track the latest update.
暂无版本248star19fork同步于 2026-11-07 07:31 UTC
README 未提供安装或运行命令，仓库为论文清单，无代码安装步骤 硬件要求按 README 说明
README 未说明运行本仓库的硬件要求；论文表格中标注了各工作所用硬件，如 RTX 4090、A6000、A100、RTX 3090 或调用 API
支持 Apple Silicon
自动驾驶场景生成与分析的基础模型论文合集，持续追踪最新研究
汇总自动驾驶场景生成与分析方向的基础模型论文，涵盖扩散模型、大语言模型和多模态模型，持续更新最新研究进展。
核心能力
- 按场景生成、场景分析、数据集、仿真器等分类整理论文
- 收录 LLM、VLM、扩散模型、世界模型等基础模型工作
- 表格标注论文的硬件需求与引用数
- 按月更新并记录新增论文数量
- 提供关键词检索方法与发表时间线图
容易踩的坑
内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
暂无 release 数据（可能没有发版，或尚未同步）。
用户点评
登录用户的真实使用记录，可按版本和硬件筛选共 0 条
还没有人点评。第一个说出真实体验的人最有价值。
Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis :car:
This repository will collect research, implementations, and resources related to Foundation Models for Scenario Generation and Analysis in autonomous driving. The repository will be maintained by TUM-AVS (Professorship of Autonomous Vehicle Systems at Technical University of Munich) and will be continuously updated to track the latest work in the community.
:fire: Updates
- Aug. 2026 – Added 9 new papers on scenario generation* and 11 new papers on scenario analysis*.
- Jul. 2026 – Added 4 new papers on scenario generation* and 14 new papers on scenario analysis*.
- Jun. 2026 – Added 33 new papers on scenario analysis* (monthly backlog catch-up spanning Dec 2023 – Jun 2026).
- May 2026 – Added 7 new papers on scenario generation* and 19 new papers on scenario analysis*.
- Apr. 2026 – Added 5 new papers on scenario generation* and 13 new papers on scenario analysis*.
- Mar. 2026 – Added 10 new papers on scenario generation* and 11 new papers on scenario analysis*.
- Feb. 2026 – Added 4 new papers on scenario generation* and 19 new papers on scenario analysis*.
- Jan. 2026 – 🎉 Our survey paper is accepted by IEEE Open Journal of Intelligent Transportation Systems (OJ-ITS) and uploaded a new version to arXiv.
- Dec. 2025 – Added 3 new papers on scenario generation* and 14 new papers on scenario analysis*. Added new columns: Hardware and Citation.
- Nov. 2025 – Added 2 new papers on scenario analysis*. Added new section: Useful Resources and Links.
- Uploaded new version to arXiv. Repository now categorizes 348 papers:
- 93 on scenario generation
- 56 on scenario analysis
- 58 on datasets
- 21 on simulators
- 25 on benchmark challenges
- 95 on other related topics (e.g., FMs' implementation)
- Oct. 2025 – Added 17 new papers on scenario generation* and 2 new papers on scenario analysis*.
- Sep. 2025 – Added 3 new papers on scenario generation* and 14 new papers on scenario analysis*.
- Aug. 2025 – Added 4 new papers on scenario generation* and 4 new papers on scenario analysis*.
- Jul. 2025 – Added 9 new papers on scenario generation* and 8 new papers on scenario analysis*.
- Jun. 2025 – Released our paper on arXiv. Repository now categorizes 342 papers:
- 93 on scenario generation
- 54 on scenario analysis
- 55 on datasets
- 21 on simulators
- 25 on benchmark challenges
- 94 on other related topics
- May 2025 – Repository initialized.
🤝 Citation
Please visit Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis for more details and comprehensive information. If you find our paper and repo helpful, please consider citing it as follows:
@ARTICLE{11370877,
author={Gao, Yuan and Piccinini, Mattia and Zhang, Yuchen and Wang, Dingrui and Moller, Korbinian and Brusnicki, Roberto and Zarrouki, Baha and Gambi, Alessio and Totz, Jan Frederik and Storms, Kai and Peters, Steven and Stocco, Andrea and Alrifaee, Bassam and Pavone, Marco and Betz, Johannes},
journal={IEEE Open Journal of Intelligent Transportation Systems},
title={Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis},
year={2026},
volume={},
number={},
pages={1-1},
keywords={Autonomous vehicles;Surveys;Foundation models;Scenario generation;Frequency modulation;Visualization;Reviews;Diffusion models;Adaptation models;Cognition;Autonomous vehicles;foundation model;scenario generation;scenario analysis;scenario based testing},
doi={10.1109/OJITS.2026.3660686}}
:page_with_curl: Introduction
Foundation models are large-scale, pre-trained models that can be adapted to a wide range of downstream tasks. In the context of autonomous driving, foundation models offer a powerful approach to scenario generation and analysis, enabling more comprehensive and realistic testing, validation, and verification of autonomous driving systems. This repository aims to collect and organize research, tools, and resources in this important field.
:chart_with_upwards_trend: Publication Timeline
The following figure shows the evolution of foundation model research in autonomous driving scenario generation and analysis over time:
:mag: Search Methodology
The following list of keywords was used to search this survey's papers in the Google Scholar database. The keywords were entered either individually or in combination with other keywords in the list. The search was conducted until May 2025.
Keywords:
- Foundation Model Types: Foundation Models, Large Language Models (LLMs), Vision-Language Models (VLMs), Multimodal Large Language Models (MLLMs), Diffusion Models (DMs), World Models (WMs), Generative Models (GMs)
- Scenario Generation & Analysis: Scenario Generation, Scenario Simulation, Traffic Simulation, Scenario Testing, Scenario Understanding, Driving Scene Generation, Scene Reasoning, Risk Assessment, Safety-Critical Scenarios, Accident Prediction
- Application Context: Autonomous Driving, Self-Driving Vehicles, AV Simulation, Driving Video Generation, Traffic Datasets, Closed-Loop Simulation, Safety Assurance
🕒 Timeline of the Development of Foundation Models
The figure illustrates the evolution of foundation models. LLMs (e.g., BERT, GPT) appear at the bottom, followed by VLMs built on visual FMs (e.g., ViT, CLIP) and instruction-tuned VLMs for interactive vision–language reasoning. MLLMs are shown at the top. In parallel, the progression of visual FMs is traced through Diffusion Models (DMs) and World Models (WMs). Highlighted entries mark key conceptual milestones.
🌟 Large Language Models for Autonomous Driving
🌟 Vision-Language Models for Autonomous Driving
🌟 Multimodal Large Language Models for Autonomous Driving
| Paper | Date | Venue | Code | Hardware | Citation |
|---|---|---|---|---|---|
| Realistic Corner Case Generation for Autonomous Vehicles with Multimodal Large Language Model | 2024-11 | Tsinghua Science and Technology | - | API&RTX3090 | 12 |
| LMM-enhanced Safety-Critical Scenario Generation for Autonomous Driving System Testing From Non-Accident Traffic Videos | 2025-01 | arXiv | GitHub | RTX2080 | 6 |
| Multi-modal Traffic Scenario Generation for Autonomous Driving System Testing | 2025-06 | FSE 2025 | - | API | 1 |
| Talk2Traffic: Interactive and Editable Traffic Scenario Generation for Autonomous Driving with Multimodal Large Language Model | 2025-06 | CVPR 2025 WDFM-AD | GitHub | - | 3 |
| OmniTraffic: A Controllable Generation Pipeline and Benchmark for Spatio-Temporal Traffic Reasoning | 2026-06 | arXiv | - | - | 0 |
PythonRobotics
AtsushiSakai·Python
用 Python 实现机器人算法的示例代码与配套教材，覆盖定位、建图、路径规划、控制等模块，适合边跑可视化动画边学自动驾驶与自主导航。
路径规划+4自动驾驶入门
30888+157.4k2026-11-10 15:16 UTC
OpenPCDet
open-mmlab·Python
基于 PyTorch 的激光雷达 3D 目标检测工具箱，统一了 PointPillar、PV-RCNN 等主流点云检测模型的训练与评测流程，省去自己搭框架的麻烦。
3D目标检测+4自动驾驶
5766+41.5k2025-10-08 10:56 UTC
mrpt
MRPT·C++
一套面向移动机器人和自动驾驶的 C++ 工具库，提供 SLAM、计算机视觉、地图构建、粒子滤波和运动估计等模块，省去从零搭建底层算法。
移动机器人+4自动驾驶
2175+16623.6.22026-11-07 22:34 UTC
AutonomousVehicleControlBeginnersGuide
ShisatoYano·Python
用 Python 仿真加开源教材一步步搭建自动驾驶系统，可在同一框架里替换对比定位、规划与控制算法，适合入门学习。
自动驾驶+4自动驾驶
1939+33222026-11-08 08:38 UTC