vla.cpp

A C++ inference engine for Vision-Language-Action (VLA) models, built on llama.cpp. It runs the open VLA policies under one runtime, each packaged as a single self-contained GGUF that needs no Python or PyTorch at inference time. The binaries drive robots on CPU, Apple Silicon, NVIDIA (including Jetson), AMD, Intel and Qualcomm hardware.

---

Install

Download a prebuilt release, or build from source for any other platform or backend.

Prebuilt binaries

Each release ships tarballs for Linux x86-64 and aarch64 (CPU and CUDA) and macOS arm64 (Metal):

TAG=<release tag>
curl -LO https://github.com/VinRobotics/vla.cpp/releases/download/$TAG/vla.cpp-$TAG-linux-x86_64-cpu.tar.gz
tar -xzf vla.cpp-$TAG-linux-x86_64-cpu.tar.gz

Each platform's requirements are in docs/PREBUILT.md.

Build from source

The CPU build needs CMake ≥ 3.22, a C++17 compiler (GCC 11+ or Clang 14+), ZeroMQ and protobuf. CMake fetches and pins llama.cpp itself:

sudo apt-get install -y libzmq3-dev cppzmq-dev libprotobuf-dev protobuf-compiler
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

-DVLA_BUILD_SERVER=OFF -DVLA_SPM=OFF drops the apt packages and builds only vla-cli, vla-bench and libvla; Octo then takes --tokens instead of --text.

For a GPU or NPU, follow the backend's page in docs/backend/: CUDA, ROCm, SYCL, OpenVINO, Metal, Hexagon. For containers, see docs/DOCKER.md.

---

Quickstart

Once the binaries are built, run one CPU prediction without a server or simulator:

pip install -U "huggingface_hub[cli]" transformers

# -hf fetches and caches the checkpoint (under $VLA_CACHE, default ~/.cache/vla)
./build/vla-cli -hf vrfai/smolvla-libero-gguf \
    --image assets/front.jpg --text "pick up the black bowl" --pretty

# or point at a file you already have
./build/vla-cli --ckpt models/smolvla/smolvla-libero.gguf \
    --image assets/front.jpg --text "pick up the black bowl" --pretty

With a prebuilt tarball, run vla-cli from the extracted vla.cpp-<tag>-<platform>/ directory instead of ./build/. --pretty prints one action row per line and --state sets proprioception. Prompt tokenization, -hf tags, vla-server, runtime flags and environment variables are in docs/USAGE.md.

---

Support matrix

Models (rows) against platforms (columns). Per-device latency and memory are in docs/benchmark/. Use ggml-compiler and follow the instruction to add a new model.

Model	CPU (x86-64 / ARM)	CUDA (NVIDIA)	ROCm (AMD)	SYCL (Intel)	Metal (Apple)	OpenVINO (Intel)	Hexagon (Qualcomm)
SmolVLA	Y	Y	Y	Y	Y	Y	Y
π0	Y	Y	Y	Y	Y	Y	Y
π0.5	Y	Y	Y	Y	Y	Y	Y
GR00T N1.5	Y	Y	Y	Y	Y	Y	Y
GR00T N1.6	Y	Y	Y	Y	Y	Y	Y
GR00T N1.7	Y	Y	Y	Y	Y	Y	Y
BitVLA	Y	Y	-	-	-	-	-
Evo-1	Y	Y	Y	Y	Y	Y	Y
VLA-Adapter	Y	Y	Y	Y	Y	Y	Y
OpenVLA-OFT	Y	Y	Y	Y	Y	Y	Y
VLA-JEPA	Y	Y	Y	Y	Y	Y	Y
Octo-Small	Y	Y	Y	Y	Y	-	Y
TurboVLA	Y	Y	Y	Y	Y	Y	Y
ACT	Y	Y	Y	~	Y	~	Y

Legend: Y = supported (released and benchmarked), ~ = in progress, - = planned.

---

Rollout on a real robot

The LeRobot fork adds a lerobot-vla-cpp client that drives an SO-101 arm against vla-server. It does the preprocessing for SmolVLA, π0, π0.5 and GR00T N1.5/1.6/1.7. Serve the policy on a GPU host; on a desktop CPU the action queue stalls:

./build/vla-server smolvla-so101.gguf   # binds tcp://*:5555

Installing, wiring and running the client are in the fork's rollout guide.

---

Companion Repos

Repo	Content
llama.cpp	ggml and its backends; CMake fetches it at a pinned tag (b11223)
LeRobot fork	lerobot-vla-cpp, the client that runs a real robot against vla-server
ggml-compiler	Generates ggml C++ from a policy's PyTorch implementation and verifies it, for porting a new model to vla.cpp

---

Documentation

Doc	Content
docs/PREBUILT.md	Release tarballs: layout, glibc and CUDA runtime requirements, macOS and Windows notes
docs/USAGE.md	vla-cli, vla-server, vla-bench: prompt tokenization, -hf tags, runtime flags, environment variables
docs/EVAL.md	Installing LIBERO and SimplerEnv, running the eval clients against vla-server
docs/MODELS.md	Converting safetensors checkpoints to GGUF, quantizing to Q8_0/Q4_0
docs/QUANTIZATION.md	FoldQuant W8A8 / W4A4: the GGUF contract, the integer arithmetic, converting a FoldQuantVLA quantized model
docs/DOCKER.md	Building and running the eval in containers
docs/ARCHITECTURE.md	Engine design: layers, the prediction path, backends, adding an architecture
docs/backend/	Per-backend build and run notes: CUDA, ROCm, SYCL, OpenVINO, Metal, Hexagon, Hexagon on Windows, WSL2
docs/benchmark/	Per-device latency and memory for every model, and the fastest flags per device
docs/KNOWN_ISSUES.md	Known issues and their resolutions
docs/ADOPTION.md	C ABI, Python bindings, release packaging, and what is left
docs/UPSTREAMING.md	The ggml-openvino fixes to send upstream to llama.cpp
CHANGELOG.md	Per-release changes, with the fastest configuration and success rate per model
CONTRIBUTING.md	Proving a change is numerically neutral, adding an architecture
bindings/python/	Python bindings

---

Citation

@misc{nguyen2026vlacpp,
  title         = {{vla.cpp}: A Unified Inference Runtime for Vision-Language-Action Models},
  author        = {Khanh D. Nguyen and Hung T. Ho and Chinh T. Nguyen and
                   Thanh Q. Duong and Linh D. Le and Duy M. H. Nguyen and
                   Vien A. Ngo and An T. Le},
  year          = {2026},
  eprint        = {2606.08094},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  doi           = {10.48550/arXiv.2606.08094},
  url           = {https://arxiv.org/abs/2606.08094}
}

---

License

Licensed under the Apache License, Version 2.0.

---

Acknowledgements

Supported VLA models:

• ACT - Zhao et al.
• SmolVLA - Hugging Face LeRobot team.
• π0,π0.5 - Physical Intelligence.
• BitVLA - Hongyu Wang et al.
• Evo-1 - Tao Lin et al.
• VLA-Adapter - Yihao Wang et al.
• OpenVLA-OFT - Moo Jin Kim et al.
• GR00T N1.x - NVIDIA Isaac.
• VLA-JEPA - Jingwen Sun et al.
• Octo - Octo Model Team, UC Berkeley RAIL.
• TurboVLA - Hengyi Xie et al.

Built on:

• llama.cpp - LLM inference engine in C/C++.
• LIBERO - benchmark suite for the success-rate sweeps.
• SimplerEnv - the second simulator in the eval scaffold.
• RoboCasa - RoboCasa365 kitchen simulator for the SmolVLA and GR00T N1.6 evals.