Awesome-LLM4AD

Awesome-LLM-for-Autonomous-Driving-Resources

This is a collection of research papers about LLM-for-Autonomous-Driving(LLM4AD). The repository will be continuously updated to track the frontier of LLM4AD (Large Language Models for Autonomous Driving), which encompasses VLM4AD (Vision-Language Models for AD) and VLA4AD (Vision-Language-Action models for AD) as integral components of this unified paradigm. Maintained by SJTU-ReThinklab.

Welcome to follow and star! If you find any related materials could be helpful, feel free to contact us ([email protected] or [email protected]) or make a PR.

Citation

Our survey paper is at https://arxiv.org/abs/2311.01043 which includes more detailed discussions and will be continuously updated.

If you find our repo is helpful, please consider cite it.

@misc{yang2023survey,
      title={LLM4Drive: A Survey of Large Language Models for Autonomous Driving}, 
      author={Zhenjie Yang and Xiaosong Jia and Hongyang Li and Junchi Yan},
      year={2023},
      eprint={2311.01043},
      archivePrefix={arXiv},
      primaryClass={cs.AI}
}

Table of Contents

• Awesome LLM-for-Autonomous-Driving(LLM4AD)
  • Table of Contents
  • Overview of LLM4AD
  • Papers
  • Datasets
  • Citation
  • License

Overview of LLM4AD

LLM-for-Autonomous-Driving (LLM4AD) refers to the application of Large Language Models(LLMs) in autonomous driving. We divide existing works based on the perspective of applying LLMs: planning, perception, question answering, and generation.

Motivation of LLM4AD

The orange circle represents the ideal level of driving competence, akin to that possessed by an experienced human driver. There are two main methods to acquire such proficiency: one, through learning-based techniques within simulated environments; and two, by learning from offline data through similar methodologies. It’s important to note that due to discrepancies between simulations and the real-world, these two domains are not fully the same, i.e. sim2real gap. Concurrently, offline data serves as a subset of real-world data since it’s collected directly from actual surroundings. However, it is difficult to fully cover the distribution as well due to the notorious long-tailed nature of autonomous driving tasks. The final goal of autonomous driving is to elevate driving abilities from a basic green stage to a more advanced blue level through extensive data collection and deep learning.

Papers

format:
- [title](paper link) [links]
  - author1, author2, and author3...
  - publisher
  - task
  - keyword
  - code or project page
  - datasets or environment or simulator
  - publish date
  - summary
  - metrics

• SafeDriveVLA: Navigation-Conditioned World Model Dreaming for Conflict-Aware End-to-End Autonomous Driving

  • Shaoyuan Xie, Zihan Zhang, Jingxuan Wang, Jiashu Qu, Xiaoqing Liang, Lingdong Kong, Junchi Lu, Henrik I. Christensen, Qi Alfred Chen
  • Publish Date: 2026.10.14
  • Task: End-to-End
  • Datasets: Bench2Drive
  • Summary：
    • Proposes SafeDriveVLA, the first driving VLA that predicts a three-level navigation-compliance mode to decouple conflict reasoning from action generation.
    • Uses navigation-conditioned world model dreaming, imagining strict navigation following to inform safety prediction, and presents a benchmark suite for navigation following, conflict awareness, and adversarial robustness.
    • Achieves 88.16 Driving Score and 68.64% Success Rate on Bench2Drive, lifts closed-loop language following by 49.3%, and reduces collisions under unsafe instructions from 60 to 38 compared to the best SimLingo variant.

• DriveWLA: Unifying Future Dynamics and Scene Semantics through World-Language-Action Modeling for Autonomous Driving

  • Shuzheng Wang, Haoxuan Li, Shengzhen Li, Chenxi Wu, Yajing Sun, Lu Hou, Wenyu Liu, Xinggang Wang
  • Publish Date: 2026.10.13
  • Task: Planning
  • Datasets: NAVSIM, HUGSIM, nuScenes, Bench2Drive
  • Summary：
    • DriveWLA is a unified World-Language-Action model based on Mixture-of-Transformers (MoT) that jointly models semantic understanding, future scene dynamics, and trajectory generation, with three experts: a World Expert, a Language Expert, and an Action Expert.
    • It introduces Cross-Expert Attention to enable controlled information flow across experts and align semantic and dynamic representations, and applies Flow-GRPO reinforcement learning post-training to refine the Action Expert.
    • DriveWLA achieves 92.3 PDMS on NAVSIM v1 and 90.1 EPDMS on NAVSIM v2, and demonstrates strong zero-shot generalization to HUGSIM, nuScenes, and Bench2Drive.

• OccWAM: Occupancy World-Action Models with Joint Scene-to-Ego Flow Matching

  • Qihang Huang, Wenhui Huang, Collister Chua, Shiyu Fang, Zhan Chen, Songyan Zhang, Zhidong Wang, Daiying Tian, Chen Lv
  • Publish Date: 2026.10.13
  • Task: Planning
  • Datasets: Bench2Drive, nuScenes
  • Summary：
    • OccWAM is an occupancy-native world-action model that jointly generates dense future occupancy and continuous ego trajectories via flow matching conditioned on scene representations.
    • It introduces scene-to-ego world-action flow matching, a two-stage formulation that first predicts scene-centric occupancy and an absolute ego trajectory, then re-indexes the predicted world into future ego frames for action-aligned planning.
    • It preserves explicit semantic geometry without pixel synthesis and achieves state-of-the-art performance on Bench2Drive and nuScenes for closed-loop driving, open-loop planning, and future occupancy prediction.

• TacitVLA: Reasoning Vision-Language-Action Models Without Reasoning Latency

  • Jack Lu, Yurong You, Boyi Li, Yulong Cao, Thomas Tian, Chaowei Xiao, Xiangyu Chen, Wenjie Luo, Boris Ivanovic, Marco Pavone, Yan Wang
  • Publish Date: 2026.10.13
  • Project Page: TacitVLA
  • Task: Reasoning
  • Summary：
    • TacitVLA is a reasoning Vision-Language-Action model whose action expert never attends to the reasoning it generates for the current call, showing that actions need not wait for or condition on decoding the reasoning text to benefit from reasoning supervision, since the supervision already improves the backbone representation.
    • On long-horizon tasks where the reasoning text also serves as episodic memory, TacitVLA can generate each memory entry during action execution rather than before acting.
    • Evaluated on four benchmarks spanning manipulation and autonomous driving (RoboMME, RMBench, LIBERO, PhysicalAI-AV), it matches its explicit-reasoning counterpart at up to 3.4x lower latency and matches the latency of the same architecture trained without reasoning at up to +17.1% success, surpassing prior best results on RoboMME Counting (82.7% vs. 66.8%) and RMBench (77.5% vs. 56.5%).

• AUGUstus: Autonomous Driving VLA Model with Unified Generation and Understanding

  • Yuqi Ye, Caigao Jiang, Zhuonan Wang, Congyuan Pei, Chao Qu, Changhao Peng, Junhong Lin, Qize Yu, Zetian Song, Guangzhao Fu, Chongruo Wu, Shiyu Huang, Shuchang Zhou, Honggou Yang, Wei Gao
  • Publish Date: 2026.10.12
  • Project Page: AUGUstus
  • Task: Planning
  • Datasets: NAVSIM, nuScenes
  • Summary：
    • AUGUstus is an autonomous-driving VLA model that unifies diffusion-based action generation and autoregressive chain-of-thought understanding in one model, with AUGUstus-mode ordering action generation before CoT for low-latency inference.
    • It introduces AUGUstus-GRPO, a unified GRPO algorithm that jointly optimizes action and CoT, while joint CoT supervision preserves VLM spatial understanding for safer and more accurate trajectories.
    • Experiments show state-of-the-art planning with 94.6 PDMS on NAVSIMv1 and 90.7 EPDMS on NAVSIMv2, and on nuScenes 0.27 m average L2 error and 0.05% average collision rate, achieving 12 Hz real-vehicle operation.

• Language2Drive: Do Vision-Language-Action Models Really Follow the Instructions in Autonomous Driving?

  • Zihan You, Zehua Fan, Wenqi Lyu, Aoqi Wang, Ni Wang, Yifei Yang, Linge Zhao, Huan Li, Sining Ang, Yuguang Yang, Jueyu Chen, Xiangyu Li, Qianshan Wei, Jieyuan Pei, Yuxin Wang, Zhiwen Chen, Jiaqi Li, Wenxuan Song, Yanyuan Qiao, Benjin Zhu, Haoang Li, Yan Wang
  • Publish Date: 2026.10.12
  • Task: End-to-End
  • Summary：
    • Language2Drive is a benchmark for testing whether Vision-Language-Action models actually follow natural-language instructions in autonomous driving, using paired counterfactual units that hold the scene fixed and change only the instruction.
    • It evaluates instruction-following across closed-loop CARLA simulation, reconstructed real-world scenes, and open-loop driving benchmarks, including the released CARLA-Language2Drive simulator with purpose-built maps.
    • Results reveal a mismatch between strong driving competence and weak language control: 93.2% of successful rollouts lack a successful counterpart under the alternative feasible instruction, exposing limitations of isolated success and conventional driving scores.

• Directional Pullback Regularization for Long-Horizon Rollout of Latent World Models in Autonomous Driving

  • Shatadal Mishra, Ahmadreza Moradipari, Nejib Ammar
  • Publish Date: 2026.10.12
  • Task: Prediction
  • Summary：
    • Introduces Directional Pullback Regularization (DPR), a training-time method that defines a kinematics-aware anisotropic Riemannian metric in decoded state space and pulls it back through the decoder Jacobian to the latent manifold, preserving the original encoder-dynamics-decoder backbone.
    • DPR assigns higher cost to latent directions that decode to lateral vehicle motion, improving long-horizon latent rollout for autonomous driving.
    • Evaluated on open-loop ego-motion rollout over a common 10-second horizon with 1-, 3-, 5-, and 10-step training rollout lengths, DPR achieves lower 10-second final displacement error than both ULR and EPR, reducing error by approximately 44% for 3-step models and yielding the strongest control-relevant tradeoff at 10 steps.

• YOUDrive: Driving Style as a Composable Direction for End-to-End Autonomous Driving

  • Guantian Zheng, Lidong Yu, Tianyu Gao, Jiaxing Chen, Jiashu Li, Chen Lv
  • Publish Date: 2026.10.10
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • YOUDrive is a vision-language-action planner for continuous and composable driving-style control, distilling each teacher's behavior into a task vector that induces a style direction in the planner's representation.
    • These style directions can be scaled and combined at inference without retraining the backbone, while a flow-matching decoder models a multimodal trajectory distribution and generates kinematically feasible motion alternatives.
    • It introduces the Style Alignment Score (SAS) and, on NAVSIM navtest, demonstrates monotonic assertiveness control across six teacher-derived personas with PDMS above 0.90 and up to 0.954.

• Reflect-VLA: Internalizing Reflective Guidance via Dual-Path On-Policy Self-Distillation for Autonomous Driving

  • Yuechen Luo, He Chong, Fang Li, Shaoqing Xu, Hexin Zhang, Ruiqi Zhang, Ziying Song, Lei Yang, Zhi-Xin Yang, Fuxi Wen
  • Publish Date: 2026.10.09
  • Task: Planning
  • Datasets: NAVSIM, Bench2Drive
  • Summary：
    • Reflect-VLA internalizes Reflective Guidance as training-time privileged information via Dual-Path On-Policy Self-Distillation (Dual-OPSD), turning trajectory evaluation into explicit diagnoses and corrective instructions.
    • Dual-OPSD uses the driving score only to route student-generated trajectories, distilling dense token-level corrective distributions from the teacher for risky trajectories while applying self-imitation to preserve reliable ones.
    • The teacher is progressively constructed through Driving-Knowledge Pretraining, Planning-and-Refinement SFT, and Self-Refinement Rejection Sampling; at inference, Reflect-VLA generates trajectories in a single pass without explicit guidance or iterative refinement, reducing training time by over 90% compared with RL.

• ManeuverFlow: Maneuver Hypothesis Generation with Future Evidence for Trajectory Evaluation and Refinement in End-to-End Autonomous Driving

  • Yuhao Wei, Yong Chen, Chen Yang, Yingchen Wang, Delin Ouyang, Lingfeng Qi, Qi Lan, Zhanhua Zhang, Jie Li, Guofa Li
  • Publish Date: 2026.10.09
  • Code: ManeuverFlow
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • ManeuverFlow generates trajectories for distinct driving intentions in one step using a maneuver-routed mixture-of-experts MeanFlow field and Gaussian mixture prior.
    • It evaluates and refines candidates with current and future evidence from a world model, predicting PDM factors and local Frenet residuals for joint re-scoring.
    • With a lightweight TransFuser-style ResNet-34 backbone, it achieves 90.6 PDMS and 90.1 EPDMS on NAVSIM v1/v2 navtest, and top PDMS* scores on WorldEngine's 3DGS closed-loop benchmark.

• PlanWAM: Planning-Shaped Future Representations for End-to-End Autonomous Driving

  • Jinchang Xu, Hongda Yu, Fengwei Dong, Wenhui Huang, Xi Wei, Yongzhi Liu, Sunan Zhang, Jirao Wang, Chen Lv, Bingbing Li, Guodong Yin, Weichao Zhuang
  • Publish Date: 2026.10.08
  • Task: Planning
  • Datasets: NAVSIM, HUGSIM
  • Summary：
    • PlanWAM is a Planning-Shaped World Action Model that lets the planning task shape future-state representations so they retain the information most useful for planning.
    • It combines a Temporal Register Pyramid for recency-aware history compression, a privileged future posterior branch shaped by trajectory-planning objectives, and Hindsight-to-Foresight Distillation to train a history-only prior.
    • The predicted planning-shaped future latent serves as planning context to guide trajectory generation and selection, achieving 93.8 PDMS / 90.9 EPDMS on NAVSIM-v1/v2 navtest and 38.7 HD-Score on HUGSIM in zero-shot closed-loop evaluation.

• VGGTWorld-VLA: Intent-Conditioned 3D World Evolution for Autonomous Driving

  • Zhaoyang Liu, Kun Jiang, Ziying Song, Diange Yang
  • Publish Date: 2026.10.08
  • Task: Prediction
  • Datasets: NAVSIM
  • Summary：
    • VGGTWorld-VLA extends VGGT-World with intention-conditioned 3D world evolution for autonomous driving, enabling controllable future geometry prediction under alternative ego actions.
    • It introduces an action-semantic conditioning mechanism that injects driving semantics and ego-motion representations into the future-token stream, along with a geometry-language-action bridge for joint conditioning on historical geometry, VLA semantic features, and maneuver/trajectory representations.
    • Evaluated on NAVSIM, the method achieves competitive geometry prediction performance, and ablation studies validate the effectiveness of semantic and action conditioning.

• AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding

  • Tianhui Cai, Xinglong Sun, Chao Fang, Zhenxin Li, Rui Song, Jose M. Alvarez, Yunxiang Mao, Jiaqi Ma, Langechuan Liu
  • Publish Date: 2026.10.08
  • Project Page: AffordDrive3D
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • AffordDrive3D is an affordance- and geometry-aware world-action model that jointly learns future action-relevant regions and spatial structure for autonomous driving.
    • It forecasts drivable areas and collision-critical regions using a VLM backbone, while predicting future geometry from RGB world-model latents.
    • On NAVSIM, it achieves state-of-the-art performance with 91.3 PDMS and 89.9 EPDMS, demonstrating the effectiveness of jointly modeling future affordances and geometry for trajectory planning.

• Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving

  • Xingtai Gui, Yucheng Zhou, Dongqian Guo, Jiahao Gong, Feiyang Tan, Jianbing Shen
  • Publish Date: 2026.10.07
  • Task: Planning
  • Summary：
    • GeoCoTDrive is an explicit geometric chain-of-thought framework that grounds geometry in a planning-oriented manner for VLA-based autonomous driving.
    • It follows a "think with 2D first, drive with dedicated 3D priors" paradigm, grounding decision-critical 2D regions and retrieving localized 3D priors from a geometric foundation model for trajectory generation.
    • It introduces planning-relevant grounding and the PlanningGrounding dataset, improving safety-critical planning performance on end-to-end autonomous driving benchmarks.

• Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving

  • Mahmoud Selim, Cristina Cipriani, Karl Henrik Johansson
  • Publish Date: 2026.10.07
  • Project Page: ICDP
  • Task: Planning
  • Datasets: nuPlan
  • Summary：
    • Introduces interaction distribution shift (IDS) as a failure mode in offline reinforcement learning for interactive autonomous driving, where an ego trajectory may be marginally well supported but poorly supported jointly with surrounding-agent behavior.
    • Proposes Interaction-Constrained Drive Policy (ICDP), an offline RL framework that decomposes joint-support degradation into ego-support and residual interaction-support components, recovered via contrastive density-ratio estimation without explicit joint-density modeling, surrounding-agent prediction, or reactive rollouts.
    • Closed-loop evaluations on nuPlan, InterPlan, and real-world truck experiments show ICDP suppresses high-value yet interaction-unsupported trajectory selections and improves performance in interaction-critical driving scenarios.

• Not All Uncertainty Matters: Simulation-in-the-Loop Fast-Slow Reasoning for Decision-Critical Autonomous Driving System

  • Jiayi Chen, Shuai Wang, Guangxu Zhu, Derrick Wing Kwan Ng, Chengzhong Xu, Kaibin Huang
  • Publish Date: 2026.10.07
  • Task: Planning
  • Summary：
    • Proposes SIGMA, a simulation-in-the-loop framework for task-oriented fast-slow collaboration that decides when cloud reasoning should influence time-critical autonomous driving decisions.
    • Embeds the planner into uncertainty assessment and evaluates plausible scene realizations under semantic and geometric uncertainty to estimate the expected reduction in planning cost.
    • Introduces expected planning gain (EPG) for cloud invocation, cloud-guidance integration, and request prioritization under deadline and resource constraints; CARLA experiments reduce unnecessary cloud interactions by 50%, improve navigation success by more than 6%, and cut finish time by up to 26.2% in dynamic scenarios.

• Controllable Crowd Generation through World-Model Planning

  • JunGyu Lee, Jisu Shin, Seunghyun Shin, Hae-Gon Jeon
  • Publish Date: 2026.10.07
  • Project Page: Ctrl-CWM
  • Task: Planning
  • Summary：
    • Ctrl-CWM is a multi-agent Controllable Crowd World Model that integrates crowd generation and run-time control by adapting world-model planning with imagined futures to crowd simulation.
    • It consists of an encoder for human motion dynamics, an actor proposing pedestrian displacements, a critic evaluating imagined crowd trajectories, and a planner selecting actions using critic scores and user costs.
    • Experiments show Ctrl-CWM outperforms the state-of-the-art on most crowd realism and collision metrics and adapts crowd behaviors to user-specified objectives introduced during simulation without retraining.

• Transferable Spatial Temporal Coherence Adversarial Attack on Black-Box Vision Language Models for Autonomous Driving

  • Heyam Bin Jahlan, Areej Alhothali, Abeer Alhothali
  • Publish Date: 2026.10.06
  • Task: Perception
  • Datasets: BDD100K, nuScenes
  • Summary：
    • Introduces Spatial Temporal Coherence Adversarial Attack (STCA), a novel black-box adversarial attack targeting VLM models used for autonomous driving scenes.
    • STCA consists of three stages: modality expansion with caption-guided frame selection, spatial attack for effective perturbation, and STCA attack disrupting cross-frame temporal coherence via motion-guided masks.
    • Experiments on BDD100K and nuScenes against Video LLaVA-7B, Qwen2.5-VL-7B, and Dolphin show high attack success rates with high SSIM, revealing VLM vulnerability and the need for robust defenses.

• Retrieval-Augmented Large Language Model Decision-Making for Autonomous Driving Guided by Chinese Philosophical Wisdom

  • Xiaojun Bi, Xiaoyuan Ma, Yiwen Sun, Tianren Huang, Chaoran Liu, Bokai Huang, Hao Yang, Baichuan Mo
  • Publish Date: 2026.10.02
  • Task: Planning
  • Datasets: Highway-env
  • Summary：
    • CPW-Drive, a closed-loop retrieval-augmented generation (RAG) framework that incorporates value guidance derived from Chinese philosophy into autonomous driving decision-making.
    • Introduces Physics-aware Spatial Similarity Retrieval (PSSR) to retrieve physically relevant historical cases by comparing vehicle layouts and velocity-extrapolated states.
    • Achieves success rates of 93.0%, 86.0%, and 72.0% on Highway-env's multilane highway-driving task, outperforming the strongest baseline by 8.0, 22.5, and 25.0 percentage points, while maintaining the highest collision-free step count and low lane-change frequency.

• Localized Conformal Safety Monitoring with Vision-Language Models for Autonomous Driving

  • Luís Marques, Rong Fang, Disha Kamale, Dmitry Berenson
  • Publish Date: 2026.10.02
  • Task: Planning
  • Datasets: CARLA
  • Summary：
    • SLLCP is a post-hoc calibration layer over frozen VLMs that transforms unreliable predictions into probabilistically calibrated safety prediction sets.
    • SLLCP introduces a localized procedure that upweights relevant past experience when calculating uncertainty thresholds, providing label-conditional finite-sample distribution-free coverage under exchangeability.
    • Evaluated over 15k CARLA trajectories, SLLCP correctly flags 89.6% of collision-causing trajectories with a Qwen backbone and 88.4% with a Cosmos backbone, versus 4.6% and 39.1% for the base VLMs.

• Towards Reliable Vision-Language Models for Autonomous Driving

  • Manasa Mariam Mammen, Priyanka Mary Mammen, Zafer Kayatas, Stefan Wagner
  • Publish Date: 2026.10.01
  • Task: VQA
  • Summary：
    • Evaluates five VLMs (Qwen3.5-9B, Gemma4-E4B, LLaVA-OneVision-7B, DriveFusion/DriveFusionQA-4B, and NVIDIA Alpamayo-1.5-10B) across four driving-related QA datasets with single-frame, multi-view, multi-frame, and monocular visual input settings.
    • Shows that visual corruption effects on accuracy and confidence reliability vary across models, datasets, and input settings, with differing impacts under degraded visual conditions.
    • Applies Visual Evidence Augmentation (VEA), a recent inference-time method, finding inconsistent improvements in model reliability under degraded visual conditions.

• ReWAM: Reciprocal World Action Models for Interactive Autonomous Driving

  • Benshan Ma, Pei Liu, Ruiguo Zhong, Lang Zhang, Mingyue Feng, Yaonong Wang, Jun Ma
  • Publish Date: 2026.09.30
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • ReWAM is a game-theoretic world action modeling framework that captures the reciprocal influence between the ego agent and other agents by representing them as conditional responders whose actions are mutually influenced.
    • It instantiates a Level-k response hierarchy where role-specific ego and other action DiTs exchange compact strategy tokens through cross-agent attention while remaining grounded in a shared representation of the future driving world.
    • Evaluated on the NAVSIM dataset, ReWAM achieves state-of-the-art performance, with particularly significant improvements in interactive scenarios, validating that modeling reciprocal responses provides a more effective foundation for interaction-aware world action generation.

• Vision-Language-Action Autonomous Driving Agent with Language-based Memory

  • Kai Yan, Xiangyu Chen, Yulong Cao, Alex Naumann, Peter Karkus, Yan Wang, Jef Packer, Alex Schwing, Yuxiong Wang, Boris Ivanovic, Wenjie Luo, Marco Pavone
  • Publish Date: 2026.09.29
  • Task: End-to-End
  • Summary：
    • AD-Memo, a general-purpose VLA driving agent with language-based memory, outputs memory as an extension of its Chain-of-Thought (CoT) to record surrounding objects critical to driving; this memory becomes part of the agent's future input.
    • It curates memory-based datasets and trains VLAs with a two-stage recipe: Supervised Fine-Tuning (SFT) and Da Capo, a novel semi-closed-loop Reinforcement Learning (RL) algorithm that uses trajectory-level advantage for memory and step-level advantage for driving.
    • Across scenarios such as all-way stops and general driving, AD-Memo improves driving quality, enables better question answering on driving scenes, and produces plug-and-play memory for other models.

• PhysWAM: Physically Consistent World Action Model for Autonomous Driving

  • Dhruv Parikh, Fengcheng Yu, Quankai Gao, Jiawei Yang, Junjie Ye, Maulik Bhatt, Thang Vu, Charles Ochoa, Rowan McAllister, Igor Vasiljevic, Rajgopal Kannan, Viktor Prasanna, Vitor Guizilini, Yue Wang
  • Publish Date: 2026.09.29
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • PhysWAM is a unified world-action model for autonomous driving that co-denoises multiview video, metric depth, and ego motion within a single flow-matching transformer.
    • It introduces Coupled Point Projection (CPP), a geometric constraint that unprojects generated depth into 3D points, transforms them using generated SE(3) ego motion, and minimizes their distance to LiDAR points transformed using recorded ego motion.
    • PhysWAM achieves strong planning performance on NAVSIM v1 and v2, transfers zero-shot to unseen driving environments, and generates accurate metric depth and temporally coherent video, with CPP improving both planning and depth prediction.

• ExceptionDrive: A Planning-Oriented Counterfactual Corner-Case Benchmark for Autonomous Driving

  • Ziyi Luo, Zhe Sun, Yehao Lu, Lei Zhou, Lisheng Wu, Xuewei Li, Zequn Qin, Xi Li
  • Publish Date: 2026.09.29
  • Task: Planning
  • Datasets: nuScenes
  • Summary：
    • ExceptionDrive is a planning-oriented counterfactual corner-case benchmark that inserts safety-critical hazards into real nuScenes scenes via VLM-assisted screening, localized multi-view editing, and quality auditing.
    • It defines 21 tasks across six safety families, with hazard/conflict regions, local safety constraints, acceptable responses, and reference-free metrics UR, HCC, HPR, and CTS to evaluate edited predictions.
    • It reveals frequent hazard-region intrusions and insufficient clearance among seven representative planners, and introduces a Reminder Agent that converts visual evidence and a shared taxonomy into structured hazard records to guide a VLM-based decision agent, improving zero-shot strategy accuracy and reducing under-warning.

• Speed in the Blind Spot: An Interpretability Analysis of Dynamic Perception in VLMs for Autonomous Driving

  • Katharina Winter, Stefan Englmeier, Fabian B. Flohr
  • Publish Date: 2026.09.29
  • Task: Perception
  • Datasets: nuScenes
  • Summary：
    • This paper studies velocity understanding in VLMs for autonomous driving as a controlled diagnostic across surrounding-agent speed, current ego speed, and short-horizon future ego-speed proposal on nuScenes.
    • It evaluates open-weight general-purpose and PhysicalAI VLMs, together with the driving-oriented Alpamayo-1.5 Vision-Language-Action model, using verbal evaluation, temporal perturbations, counterfactual ego-speed hints, and linear probes of hidden representations.
    • The results reveal distinct failure modes: current ego speed is often internally accessible but poorly verbalized, multiple frames and frame order are rarely exploited, and driving specialization strengthens some motion representations without guaranteeing reliable readout or temporal grounding.

• RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rollouts

  • Hongbin Lin, Chaoda Zheng, Yiming Yang, Xiangyu Li, Shijia Chen, Jinhao Deng, Kangjie Chen, Dongbin Zhang, Jie Feng, Yu Zhang, Xianming Liu, Shuguang Cui, Boyang Wang, Zhen Li
  • Publish Date: 2026.09.29
  • Task: End-to-End
  • Datasets: nuScenes
  • Summary：
    • RoXDrive is a plug-and-play closed-loop reinforcement learning framework for end-to-end autonomous driving that identifies action-faithful world-model rollouts to avoid action-vision mismatch.
    • It uses model pre-training with an Action-Vision Faithfulness Evaluator for inverse dynamics estimation and geometry-aware auxiliary trajectory supervision, followed by action-faithful RL post-training with dense safety-aware scoring and scene-level closed-loop optimization.
    • Experiments on nuScenes and an in-house dataset with over 130K training scenarios show consistent gains across planners, reducing safety violations by 27.6% with DiffusionDrive on nuScenes and 33.7% with Qwen3-VL on internal data.

• World4Scorer: Outcome-Grounded World Modeling for Autonomous Driving

  • Jieyuan Pei, Meiyi Lu, Sining Ang, Yubo Zhao, Zhangyi Hu, Mingwei Xu, Haokai Ding, Wei Li, Zihan You, Jianwei Zheng, Li Yu, Yifeng Pan, Ji Tao, Rongjunchen Zhang, Yan Wang
  • Publish Date: 2026.09.29
  • Task: Planning
  • Datasets: NAVSIM, Bench2Drive, OGBench-Cube
  • Summary：
    • World4Scorer builds a trajectory-conditioned JEPA-style scorer for generate-and-select planners, predicting a state for each candidate and reading candidate scores from that state.
    • It supervises all candidate states with simulator outcome labels, anchors the predictor to the executed trajectory's observed future, and uses a scene-matched low-score bank plus inertial re-ranking.
    • It achieves state-of-the-art NAVSIM-v2 performance, strong adapted closed-loop Bench2Drive results, and improves manipulation planning on OGBench-Cube with LeWM.

• RefineDrive: Reliable Failure-Guided Learning for Vision-Language-Action Driving

  • Zhe Sun, Ziyi Luo, Yehao Lu, Lei Zhou, Xi Li
  • Publish Date: 2026.09.28
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • RefineDrive is a failure-guided post-training framework for Vision-Language-Action autonomous driving that learns from self-generated failures through reliable diagnosis, minimum-correction target retrieval, and safety-aware reinforcement learning.
    • It derives structured, verifiable feedback on collisions and drivable-area violations from simulator states, retrieves hard-safety-constrained corrections from a clustered human trajectory bank, and uses correction SFT followed by GRPO with a Safety-Layered Reward.
    • On NAVSIM v1, RefineDrive improves a 4B base SFT policy from 87.7 to 91.7 PDMS and achieves 89.4 EPDMS on original NAVTEST scenes under NAVSIM v2 extended metrics without additional training.

• CAR-VLA: Complexity-Aware and Risk-Adaptive Reasoning for Autonomous Driving

  • Xiaolei Chen, Zhuolin He, Yuxuan Liang, Xu Li, Haotian Chen, Shi Fan, Mengyang Zhao, Wenjuan Meng, Zisheng Chen, Zhihao Zhu, Zhounan Jin, Hengli Wang, Qingfan Wang, Jiamei Liang, Bin Li, Xiangyang Xue
  • Publish Date: 2026.09.28
  • Code: CAR-VLA
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • CAR-VLA is a unified driving VLA model that jointly considers scene complexity and dynamic risk to guide reasoning depth, urgency, and focus.
    • It maps four complexity-risk categories to three reasoning modes: Fast Intuition, Slow Thinking, and Reflex Response, with high-risk scenarios centered on the most critical hazard and immediate safe response.
    • Trained via progressive supervised learning linking scene assessment, reasoning-mode selection, and trajectory generation, followed by reasoning-augmented reinforcement learning, achieving competitive results on NAVSIM v1, NAVSIM v2, and Navhard.

• AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Driving

  • Haoran Zhu, Wancong Zhang, Yann LeCun, Anna Choromanska
  • Publish Date: 2026.09.28
  • Code: AD-E2E-JEPA
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • Systematically evaluates existing action-conditioned JEPA world models (LeWM, DINO-WM, JEPA-WM) for end-to-end autonomous driving using a goal-conditioned zero-shot planning setting.
    • Proposes AD-E2E-JEPA with a SIGReg-regularized learnable projector that reduces planning patches by 16× and embedding dimension by 4×, achieving a 100× inference speedup while retaining planning performance.
    • Achieves 67.3/72.9 EPDMS on NAVSIMv2 in goal-conditioned zero-shot planning, and the self-supervised pretrained projector improves downstream imitation learning from 80.2 to 85.4 EPDMS.

• MomWorld: Momentum-Aware Latent World Model for Long-Horizon Autonomous Driving

  • Ziying Song, Shengkai Zhang, Lei Yang, Haozhuang Chi, Yuchen Liu, Jiangtao Su, Lin Liu, Ziyang Liu, Chen Lv
  • Publish Date: 2026.09.27
  • Task: Planning
  • Datasets: NAVSIM, nuScenes, Bench2Drive
  • Summary：
    • MomWorld is a momentum-aware latent world model for long-horizon planning that extracts scene motion trends from historical-to-current observations and propagates latent momentum into future horizons.
    • It jointly predicts future configuration and momentum states with a learnable momentum persistence mechanism, scene-conditioned momentum updates, and a scene-adaptive reset gate to suppress stale momentum under abrupt changes.
    • It introduces MoFlow, a momentum-conditioned flow-matching module with horizon-aware residual fusion, improving long-horizon planning consistency and reducing the average collision rate by 12.2% relative to MomAD over a 6-second planning horizon on NAVSIM, nuScenes, and Bench2Drive.

• VehDyn: A Driving World Model Benchmark for Vehicle Dynamics

  • Tianyi Wang, Wangsheng Du, Jiazhou Chen, Tianyi Zeng, Xiangyu Li, Jiseop Byeon, Yujin Wang, Yiming Xu, Yangyang Wang, Bingzhao Gao, Sikai Chen, Zhaomiao Guo, Junfeng Jiao, Christian Claudel, Alexandre Bayen
  • Publish Date: 2026.09.27
  • Task: Evaluation
  • Summary：
    • VehDyn is a driving world model benchmark for vehicle dynamics, built on a CARLA-CarSim co-simulation platform that couples photorealistic rendering with a validated multi-body dynamics model.
    • It contains 10,080 configurations from a full factorial design over vehicle types, tire-road friction coefficients, maneuvers, target speeds, scenes, and illuminations, each paired with synchronized position, velocity, and attitude sequences.
    • It introduces a hierarchical evaluation framework for trajectory alignment, kinematic consistency, and dynamic consistency, benchmarks 12 state-of-the-art video world models, and shows that trajectory metrics are nearly saturated while dynamic consistency remains challenging and visual quality weakly correlates with vehicle-dynamics fidelity.

• RCVLA: 4D Radar-Grounded Semantic Reasoning and Trajectory Arbitration for Autonomous Driving

  • Lianqing Zheng, Xiaokai Bai, Yixuan Luo, Runwei Guan, Minghao Liu, Zhiqiang Wei, Hui-liang Shen, Xichan Zhu, Zhixiong Ma
  • Publish Date: 2026.09.26
  • Task: Planning
  • Summary：
    • RCVLA is a radar-camera vision-language-action framework with a radar-grounded semantic reasoning stage (RCVLA-Sem) and a trajectory arbitration stage (RCVLA-Phys).
    • It constructs Cap4DR with 86,016 radar-image-text samples for alignment pretraining and OmniHD-QA with 520,161 question-answer pairs for instruction tuning across scene description, key-object reasoning, occupancy understanding, and trajectory planning.
    • RCVLA-Sem improves CIDEr by 9.92 points and reduces key-object velocity error by 21.9% over OmniDrive; RCVLA-Phys reduces average L2 error from 0.348 to 0.259 m and average open-loop collision rate from 0.576% to 0.175%.

• From Scene Graphs to Answers: Selective Neuro-Symbolic Reasoning for Autonomous Driving

  • Yiyao Wang, Pei Liu, Fangzhou Liu, Jun Ma
  • Publish Date: 2026.09.26
  • Task: VQA
  • Datasets: NuScenes-QA, nuScenes
  • Summary：
    • Introduces a query-adaptive neuro-symbolic reasoning framework for autonomous-driving question answering, with a hierarchical Spatiotemporal Scene Graph (STSG) that separates persistent object identities from frame-specific states and represents spatial relations and temporal transitions as explicit directed structures.
    • Uses a symbolic executor to resolve queries through exact graph operations and invokes an LLM only when symbolic execution abstains, with query-conditioned graph retrieval and evidence filtering preserving relation direction, temporal locality, and object semantics.
    • Achieves 80.63% overall accuracy on 5,916 NuScenes-QA questions under oracle perception, improving over the LLM-only configuration by 5.48 points with GPT-5.4-mini and 6.64 points with DeepSeek-V4-Flash, with largest gains on counting questions.

• OneFixer: High-Quality and Consistent One-Step Autoregressive 3DGS Refinement for Driving Scenes

  • Boseong Jeon, Junhyeop Lee, Juhan Cha, Hayoung Kim
  • Publish Date: 2026.09.26
  • Project Page: OneFixer
  • Task: Perception
  • Datasets: Waymo
  • Summary：
    • OneFixer is a one-step autoregressive video-diffusion fixer for 3D Gaussian Splatting in driving scenes, trained in a single task-specific adaptation stage.
    • It uses a deployment-matched shared rollout where one-step predictions serve as causal context for flow matching while receiving direct pixel-space perceptual supervision, jointly learning fidelity and autoregressive robustness.
    • It exploits lane geometry and dynamic-agent states to improve geometric fidelity, achieving the lowest FVD, LPIPS, and DISTS at one step on Waymo and proprietary driving scenes, and reducing collision rate by one third in closed-loop simulation.

• CausalDriveBench: Evaluating Causal Reasoning in Vision-Language-Action Models for Autonomous Driving

  • Narendiran Chembu, Navvrat Rao, Shreedhar Shreeshail Kodate, Gayatri Srujana Banda, Arko Sarkar, Abhinav Khanna, Rajarshee Das, Umesh Kanala, Siddarth Khandelwal, Kumar Aman, Aish Dubey, Kaustubh Beedkar, Arjun Jain
  • Publish Date: 2026.09.26
  • Task: Reasoning
  • Datasets: nuScenes
  • Summary：
    • CausalDriveBench is an evaluation framework grounded in Pearl's Causal Hierarchy that tests causal reasoning in driving-specific VLAs through structured visual question answering and alternative-trajectory prediction.
    • It constructs causal scene graphs over nuScenes to distinguish causally active, dormant, and distractor entities, covering all four rungs of PCH and containing 7,285 verified causal QA pairs and 1,000 counterfactual trajectories.
    • Evaluating 10 driving-specific VLAs and 3 general-purpose VLMs shows that causal QA accuracy remains limited, driving fine-tuning incurs variable costs on causal QA, and causal QA and trajectory accuracy are statistically uncorrelated across models.

• DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution

  • Chengkai Xu, Jiaqi Liu, Yicheng Guo, Peng Hang, Jian Sun
  • Publish Date: 2026.09.25
  • Code: DriveHierarchy
  • Task: Evaluation
  • Summary：
    • DriveHierarchy is a hierarchical benchmark that organizes VLM-based autonomous driving into four ranks: perceptual grounding, contextual memory, mental reasoning, and closed-loop execution.
    • It integrates multiple open-source autonomous-driving datasets into a unified open-loop benchmark with 76,798 question-answer pairs over 84,279 frames, and develops a closed-loop simulation platform with 100 interactive scenarios.
    • Experiments on 15 VLMs show that DriveHierarchy captures structured non-redundant capability variation, relates open-loop understanding to closed-loop driving, and provides a practical basis for diagnosis and benchmark-guided optimization.

• Evaluation Is All You Need for Multi-Modal Autonomous Driving

  • Zeyu He, Shiqi Liu, Ke Chen, Yun Yan, Jinzi Wu, Dianqiao Lei, Sirui Wang, ShuRui Peng, Tao Chen, Zhuo Huang, Yu Wu, Yadong Shao, Zhichao Li, Ke Sun, Yang Guan, Keqiang Li, Shengbo Eben Li
  • Publish Date: 2026.09.25
  • Task: Evaluation
  • Datasets: NAVSIM
  • Summary：
    • iDriveVLA is a multi-modal planning framework that improves the candidate trajectory space while enabling reliable and context-aware trajectory evaluation.
    • It introduces a unified trajectory evaluator with a Safety-aware Scorer for quality and risk estimation and a VLM-guided Modulator for scene-adaptive criterion weighting.
    • An oracle-aligned progressive training strategy combines candidate imitation pretraining, candidate space refinement, and semantic ranking alignment, achieving 94.95 PDMS on NAVSIM v1 and surpassing human-expert reference.

• WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving

  • Mingkai Jia, Jiaxin Guo, Zhijian Shu, Jiawei Xu, Mingxiao Li, Jintao Cheng, Ping Tan, Wei Yin
  • Publish Date: 2026.09.24
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • WALT learns a compact generative trajectory latent space by transferring information from a frozen pretrained driving world model, aligning raw waypoints with action-relevant semantics without modifying the world model.
    • It uses a dual-branch trajectory autoencoder and systematically studies JEPA-based latent learning and REPA-style feature alignment for trajectory-only representation learning.
    • On NAVSIM, WALT improves PDMS from 89.4 to 89.8 on NAVSIMv1 and EPDMS from 87.3 to 87.9 on NAVSIMv2 while reducing trajectory planner FLOPs by 30.5%.

• Sim-to-Real Aware End-to-End Learning Environment for Micromobility

  • Shouma Amano, Takuya Azumi
  • Publish Date: 2026.09.24
  • Task: End-to-End
  • Summary：
    • Proposes a sim-to-real-aware, vehicle-specific end-to-end learning environment for the WHILL Model CR on AWSIM and ROS 2.
    • Optimizes physical parameters via Bayesian optimization using real-world data to reduce the sim-to-real gap and trajectory errors across various driving scenarios.
    • Introduces a synchronized architecture for stable training of world model-based agents, with a DreamerV3 policy achieving obstacle avoidance and direct sim-to-real transfer to the physical vehicle without fine-tuning.

• Less Language, More Latents: Annotation-Efficient VLAs for Driving

  • Alexey Zakharov, Kemal Oksuz, Puneet K. Dokania
  • Publish Date: 2026.09.23
  • Task: End-to-End
  • Datasets: Bench2Drive
  • Summary：
    • Introduces Latent Action Driving Annotations (LADA), a three-stage pipeline that transforms abundant unlabelled observation-trajectory pairs into training data for language-conditioned control.
    • Uses a latent action model with a vector-quantised bottleneck to build a compact codebook of high-level vehicle intents, and trains a vision-language translator on a small language-annotated subset.
    • Trains a driving VLA on observation-latent-action pairs over the full unlabelled corpus, achieving 87.98 Driving Score and 70.46% Success Rate on Bench2Drive with fewer than 5% language annotations.

• NAVSAFE-∞: Benchmarking Closed-Loop Driving Safety in Photorealistic Environments

  • Yuxin Bao, Hongwei Ruan, Luobin Wang, Seth Z. Zhao, Ziyang Leng, Zihan Zhang, Yu Zeng, Rowan McAllister, Henrik Christensen, Bolei Zhou
  • Publish Date: 2026.09.22
  • Project Page: NAVSAFE-∞
  • Task: Evaluation
  • Datasets: NAVSAFE-∞, NAVSIM
  • Summary:
    • NAVSAFE-∞ is a photorealistic closed-loop autonomous driving benchmark comprising 280 scenarios across 28 safety-critical event types in four categories, with event-specific success criteria and category-level ability scores.
    • Built on 3D Gaussian Splatting (3DGS) and diffusion-based rendering refinement, it enables realistic policy-environment interactions and systematic evaluation of closed-loop driving safety.
    • Evaluations of 20 E2E driving policies reveal that strong open-loop performance does not reliably translate into closed-loop safety, while passive demonstration perturbation offers limited generalization and open-loop RL fine-tuning can amplify safety failures through reward hacking.

• ForeDrive: Foresight-Guided End-to-End Autonomous Driving with a Planning-Relevant Latent World Model

  • Sinuo Wang, Zichong Gu, Yuhan Huang, Wenxin Wen, Xun Yang, Yiqing Zhang, Xingyu Zhang, Ningyu Che, Jie Ling, Qiankun Yu, Wei Liu, Jing Xu, Xinggang Wang
  • Publish Date: 2026.09.22
  • Task: End-to-End
  • Datasets: NAVSIM
  • Summary：
    • ForeDrive learns a planning-relevant latent future representation with a JEPA-style world model and couples it asymmetrically to a Diffusion Transformer (DiT) planner, so that predicted futures act as direct conditioning signals for trajectory generation rather than mere pretraining or auxiliary supervision.
    • The framework uses gated visual fusion, future-status injection, and Trajectory-Adaptive Bias (TAB) to inject future latents as guidance without overriding the current observation, addressing varying future reliability across horizons and the misalignment between BEV trajectories and image tokens.
    • Trained with pure imitation learning and using only the current front-view image as visual input at inference, ForeDrive attains 89.9 PDMS on NAVSIM v1 and 90.0 one-stage EPDMS on NAVSIM v2, without reinforcement learning or an external trajectory scorer.

• Sometimes You Gotta Run Before You Can Walk: Run-then-Walk Scheduling Strategy for VLM Autonomous Driving

  • Yuqi Ye, Shangkun Sun, Junhong Lin, Jiayi Zhao, Changhao Peng, Wei Zheng, Guoqing Liu, Tiesong Zhao, Wei Gao
  • Publish Date: 2026.09.22
  • Task: Planning
  • Datasets: NAVSIM, nuScenes
  • Summary：
    • Run-then-Walk is a simple yet effective two-stage reward scheduling strategy for GRPO that explicitly separates progress discovery from safety repair, achieving both better performance and faster convergence.
    • The Run phase focuses on progress to escape conservative bias and discover high-progress modes, while the subsequent Walk phase introduces endpoint and safety strategies to repair unsafe behaviors, overcoming the conservatism of Walk-first methods and the unsafe progress-seeking of joint optimization.
    • Validated on NAVSIMv1, NAVSIMv2, Navhard, and nuScenes with various VLM-based planners, demonstrating improved driving performance while requiring 40--50% fewer RL training epochs than baselines.

• Relationally Grounded Latent World Models for Autonomous Driving

  • Fabian Schmidt, Markus Enzweiler, Abhinav Valada
  • Publish Date: 2026.09.21
  • Task: Prediction
  • Datasets: nuScenes
  • Summary：
    • Investigates whether traffic scene graphs can serve as privileged semantic supervision for latent world representations in autonomous driving.
    • Constructs actor-centric scene graphs from nuScenes 3D annotations, encodes their serialized relational structure with a frozen text embedding model, and aligns visual latent representations with this semantic target during training.
    • Removes the supervision branch at inference and, on nuScenes, reduces average trajectory L2 error from 0.661 to 0.622 and collision rate from 0.456 to 0.217, outperforming an unstructured caption-style semantic target.

• PRIME: Perception Feedback with Situational Memory Embeddings in VLA Models

  • Erik Deinzer, Naya Baslan, Luca Paparusso, Narunas Vaskevicius, Peter Knott, Luigi Palmieri
  • Publish Date: 2026.09.18
  • Task: End-to-End
  • Datasets: Bench2Drive, Think2Drive
  • Summary：
    • PRIME introduces a learned feedback mechanism that conditions VLA perceptual queries on a novel Situational Memory, enabling intent-driven perceptual attention at minimal computational cost.
    • It aggregates latent representations of past perception, reasoning, navigation goals, and predicted behaviors across an L-step window via cross-attention, adding at most 29.7M parameters (0.41% of the 7.3B-parameter base model).
    • On Bench2Drive, PRIME achieves a state-of-the-art Driving Score of 82.47 (+4.73 over ORION) and a Success Rate of 60.00% (+5.38 pp), the highest reported among published VLAs trained on Think2Drive demonstrations.

• ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulation

  • Boni Hu, Xiong Wei, Haoming Huang, Yong Huang, Chenbo Wang, Yi Yang, Jiancheng Wang, Ruicheng Zhu, Zhimin Yang, Guanglai Liu, Qiaowan Jin, Dongzhuo Wang, Haiwei Kuang, Jiajun Fan, Yue Wu, Jiaxin Wei, Hao Sun, Feihong Yan, Yuyao Zhou, Wei Bi, Kaixuan Wang, Zichao Guo, Xiaozhi Chen
  • Publish Date: 2026.09.18
  • Task: Generation
  • Summary：
    • ZYT-World is a real-time controllable world model that natively generates four fisheye views with field of view > 180° and three pinhole views for closed-loop autonomous-driving simulation.
    • It combines projection-specific Plücker adapters, ego-motion adaptive layer normalization, and a lightweight pixel-aligned layout to control global motion and instance-level traffic participants and signals.
    • A distillation pipeline converts a 40-step bidirectional teacher into a one-step per-latent streaming generator, while TinyVAE, W8A8 quantization, and RigCritic enable over 90% PSNR/SSIM retention and 107.7× faster generation.

• Astronex-World 1.0: Real-Time Interactive World Model Foundation

  • Xin Zhou, Cong Miao
  • Publish Date: 2026.09.17
  • Task: Prediction
  • Summary：
    • Astronex-World 1.0 is an open controllable video world-model foundation that predicts future visual states from text prompts or initial observations under frame-aligned camera trajectories, continuous actions, and an embodiment identifier, and accepts text events inserted at specified rollout positions.
    • The family includes a bidirectional full-context generation model and a causal model with block-causal attention and cross-block KV caching for persistent generation, both built on the Wan2.2-TI2V-5B prior; PRoPE injects camera intrinsics/extrinsics while a 64-dimensional action stream modulates every Transformer layer.
    • A five-stage training path develops bidirectional camera and action control, converts the backbone to block-causal generation, distills a few-step student, restores mixed-domain dynamics, and applies asymmetric DMD/DMD2 distribution matching; the causal model generates 832x480 video at 24 fps and streams in real time on one NVIDIA L20, scoring 73.5 on WBench Navi and 70.0 on WBench Full.

• REARL: A Closed-loop Autonomous Driving Simulation Enhancement Framework with Real Traffic Data and Large Language Models

  • Xiaojun Bi, Jun Jiang, Yiwen Sun, Quanyi Ou, Yizhi Ma, Ke Cheng, Mingjie Bi, Yexin Li, Bowen Du
  • Publish Date: 2026.09.17
  • Task: Planning
  • Datasets: HighD
  • Summary：
    • REARL is a closed-loop simulation enhancement framework that integrates real traffic data with LLMs, using clustered real traffic data as representative scenarios and a timed sliding-window detector to monitor speed distribution and mean spacing discrepancies.
    • When a metric exceeds a threshold, the LLM adjusts vehicle decision-making; otherwise the existing controller is retained, and the LLM also selects matching real vehicle actions from a traffic snapshot to modulate simulated vehicles.
    • In a controlled HighD highway setting, REARL reduces the Hellinger distance for speed distributions to 0.3067 and the MAPE for mean spacing to 0.8371, while achieving a time headway of 22.8575 and a lane change rate of 0.0708.

• FIVE-VLA: Fast and EffectIVE Autonomous Driving with Recurrent Action Memory

  • Kemal Oksuz, Alexandru Buburuzan, Yuhan Yao, Puneet K. Dokania
  • Publisher: Five AI, University of Oxford
  • Publish Date: 2026.09.16
  • Task: Planning
  • Datasets: Bench2Drive, NVIDIA Physical AI AV Dataset
  • Summary：
    • FIVE-VLA introduces an efficient vision encoder that processes 448×896 images with only 98 tokens (over 5× fewer than existing approaches) and bypasses text generation for single-pass trajectory prediction.
    • It proposes Recurrent Action Memory (RAM), a lightweight module that conditions action prediction on previous action tokens to provide temporal context for manoeuvres such as overtaking and emergency braking.
    • With 641M parameters, FIVE-VLA achieves ~10% more routes without traffic rule infractions than the previous state-of-the-art VLA on Bench2Drive, reduces collision-violation rates on the NVIDIA Physical AI AV dataset, and runs at ~30 fps on an A100 and ~4 fps on a T4 (8–30× speedup).

• RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving

  • Dogun Kim, Yongjae Lee, Joonhee Lim, Yeina Lee, Junhyeok Park, Moogeun Park, Dongsuk Kum
  • Publish Date: 2026.09.15
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • RAF-VLA (Representation Alignment with the Future) is a VLA-based autonomous driving framework that aligns the policy's hidden states with future-frame representations from a pretrained world encoder, avoiding explicit future generation.
    • It employs Future-Aligned Supervised Fine-Tuning, a straightforward regularization that shapes planning-relevant internal representations while learning driving actions, eliminating the training burden and inference latency of world modeling.
    • Experiments on the NAVSIM benchmark show competitive planning performance against state-of-the-art VLA planners with substantially fewer training samples, while incurring only 3.8% training overhead and negligible 1 ms inference overhead.

• EditWM: Event-Decomposed World Modeling with Incremental Correction for End-to-End Autonomous Driving

  • Junjie Yang, Qingwei Zeng, Youyou Li, Zicheng Ding, Ziyi Shi, Shuqi Shen, Hongliang Lu, Hai Yang
  • Publish Date: 2026.09.15
  • Task: Planning
  • Datasets: NAVSIM
  • Summary：
    • EditWM decomposes future prediction into normal evolution and event-driven incremental correction in compact visual feature space.
    • It freezes a trajectory-conditioned normal predictor and trains a correction decoder with a learned gate to produce bounded feature updates, using corrected future features for candidate-specific trajectory scoring.
    • Evaluated on all 12,146 NAVSIM navtest scenes, it reduces future-feature MSE by 5.35% over Normal with improvements in 83.54% of scenes and achieves 91.05 EPDMS.

• GRAVA: Grounded Reasoning-to-Action Representation and Learning for Autonomous Driving

  • Xiao Liu, Haoyu Li, J