vllm-playground

vLLM Playground v0.1.9 - CI/CD Pipeline, Configurable Container Images & Remote Fixes

Release Date: September 2026

This release ships a full CI/CD pipeline (automated tests + GitHub Actions on every PR, plus a gated PyPI release process), a new Settings tab for configurable container image versions, and fixes real chat-error handling and MaaS gateway support in Remote mode.

Highlights

CI/CD Pipeline

Every pull request now runs a layered test suite automatically: lint/format, unit tests (Python 3.9/3.11/3.12), FastAPI API tests, container-manager tests against a real Podman daemon, Kubernetes-manager tests against a real kind cluster, and Playwright frontend smoke tests — all in about two minutes.

• Releases are tag-triggered (vX.Y.Z), version/changelog-verified, and re-tested against a fast safety-net subset right before publish, then shipped to PyPI via Trusted Publishing (OIDC) — no stored API token.
• While building this pipeline, a one-off container-build check caught (and this release fixes) two pre-existing bugs: both the Podman and OpenShift container images failed to start with ModuleNotFoundError: No module named 'vllm_playground', because app.py was being run as a bare script instead of as part of the installed package. Both Containerfiles are fixed and verified to boot and respond on /api/status. Building the container images is not part of the automated CI/CD pipeline itself — container/OpenShift deployment is a secondary use case, so this was a manual, ad-hoc verification rather than a standing gate.

Configurable Container Image Versions

New Settings tab lets you pick vLLM/vLLM-Omni container image versions per accelerator from a dropdown (sourced live from Docker Hub, cached, with an offline fallback list) instead of relying on hardcoded tags.

• Per-accelerator overrides persist and default back to the built-in version when unset; picking the default-tagged option resets it.
• Version bumps: vllm/vllm-openai and vllm/vllm-openai-rocm → v0.29.0, vllm/vllm-omni and vllm/vllm-omni-rocm → v0.28.0.
• CPU mode now uses the official multi-arch vllm/vllm-openai-cpu:v0.29.0 image (covers macOS ARM64 + Linux x86_64) instead of a self-built Quay image.
• AMD GPU default switched from rocm/vllm:latest to the versioned vllm/vllm-openai-rocm so the AMD dropdown has real tags to offer.

Remote Mode Fixes

• Fixed silent "No response from model" failures during streaming chat in Remote mode: upstream error responses were being serialized as invalid JSON (Python dict repr, not json.dumps), which the frontend's JSON.parse silently dropped. Errors now surface as real, readable messages.
• Added generic support for MaaS/KServe-style gateways that advertise a dedicated base URL per model (via GET /v1/models) instead of one shared OpenAI-compatible root — Remote mode now detects and uses per-model URLs automatically, with a self-healing retry if the shared root 404s.
• Fixed a token-count fallback bug that fabricated "1 out" tokens on failed requests, and distinguished a genuinely empty-but-successful response (e.g. an embedding-only model selected for chat) from a real failure.

Recipes Catalog

• Synced the recipes catalog with the latest upstream vLLM recipes (as of 2026-07-02), including the new ArceeAI/Trinity-Large-Thinking recipe.

Upgrade Notes

• If you run the OpenShift or Podman container images built from this repo's Containerfiles, upgrade to this release — prior images failed to start at all.
• No settings migration needed; existing container image tags keep working unless you explicitly pick an override in the new Settings tab.

Full test suite: pytest tests/unit tests/api tests/container tests/k8s tests/frontend — all passing in CI on Python 3.9, 3.11, and 3.12.

Release Date: April 2026

This release ships multi-instance management (registry-backed tabs and Management → Instances), improves remote and LiteLLM connectivity and model selection, and fixes benchmarking so auth and model IDs match the server you target.

Install: pip install vllm-playground==0.1.8 — https://pypi.org/project/vllm-playground/

Highlights

Multi-Instance Backends

Run and manage several vLLM servers at once — subprocess, container, and remote — each as its own Instance with isolated chat, logs, and metrics.

• Instance registry with persistence under ~/.vllm-playground/instances.json
• Per-instance tabs on the vLLM Server page; switch active instance with one click
• Management → Instances grid: search, status, connect/stop, and saved configurations
• See the Multi-Instance Guide for workflows and tips

Remote Server & LiteLLM

• Resilient remote URLs — normalization for OpenAI-compatible roots and LiteLLM-style bases; connect probing without requiring /health on every deployment
• Chat model list from /v1/models — populate the model dropdown from the active server; better alignment with served-model aliases and remote catalogs
• Context length — improved handling when gateways expose limits via model metadata (e.g. LiteLLM model/info-style data)

Benchmarking (Built-in & GuideLLM)

• Bearer Authorization for OpenAI-compatible remote targets (LiteLLM, etc.)
• Optional API key from the UI for remote benchmarks; keys are not applied to localhost/container targets (avoids breaking local vLLM with gateway tokens)
• Model ID matches the selected instance — when GuideLLM / built-in benchmark sends instance_id, the OpenAI model field (and GuideLLM --model when set) comes from that registry entry (served_model_name, then model, then saved config), fixing typical 404 model not found when the UI session was still on a different remote model

UI & Reliability

• GuideLLM target-instance dropdown stability when the instance list updates
• GPU memory utilization input normalized (e.g. fractional vs percentage)
• Mode switch clears stale remote model selection where appropriate

Upgrade

pip install --upgrade vllm-playground

From Source

git clone https://github.com/micytao/vllm-playground.git
cd vllm-playground
pip install -e .
vllm-playground

Documentation

• Changelog — Version history
• Multi-Instance Guide — Instances, tabs, and management UI
• Installation Guide — Setup instructions

Links

• PyPI: https://pypi.org/project/vllm-playground/
• vLLM Workshop: https://micytao.github.io/vllm-workshop/

vLLM Playground v0.1.7 - Hotfix & Tutorials

Release Date: March 2026

This release fixes six bugs and adds an embedded Tutorials page.

Note: v0.1.6 has been yanked from PyPI due to the container mode bug below. Users should upgrade to v0.1.7.

Bug Fixes

Container Mode Switching Broken

The getConfig() function in app.js was missing an explicit branch for the container radio button. Additionally, the fallback default was changed from 'container' (v0.1.5) to 'remote' (v0.1.6), which meant selecting container mode silently sent run_mode: 'remote' to the backend. The server would then attempt to connect to a remote URL (typically empty), breaking the vLLM server entirely.

Fix: Restored the default to 'container' (matching v0.1.5) and added explicit handling for all three modes (subprocess, container, remote).

Logprobs Tooltip Clipped on First Line

The logprobs probability tooltip (shown when hovering over tokens with logprobs enabled) was clipped when appearing on the first line of a chat response. The .message-content container had overflow: hidden, which prevented the absolutely-positioned tooltip from extending above the message bubble.

Fix: Changed .message-content from overflow: hidden to overflow: visible.

Logprobs Crash on Metal/CPU

When logprobs were enabled with Metal or CPU compute mode, vLLM hit an IndexError: list index out of range in _create_chat_logprobs in both streaming and non-streaming modes. This is an upstream vLLM bug.

Fix: Two-layer workaround based on compute mode (not run mode):

1. Force non-streaming mode when logprobs are enabled with Metal or CPU compute mode selected
2. If vLLM still returns a 500 error, transparently retry the request without logprobs and show a notification: "Logprobs not available — vLLM backend does not support logprobs for this model/platform."

GPU compute mode and remote mode are unaffected and continue to use streaming with logprobs. The workaround correctly handles the edge case where remote mode hides (but retains) the compute mode radio state — remote mode always allows streaming regardless of the hidden compute selection.

Mode Switch Config Leak

When switching from container or subprocess mode to remote mode, form fields like served_model_name, enable_tool_calling, and tool_call_parser retained their stale values from the previous mode. Since these fields are part of the hidden "local server settings" section, they were invisible to the user but still read by getConfig(). If served_model_name was set (e.g., "my-coder"), it overrode the auto-discovered remote model name, causing chat requests to target a nonexistent model and returning "no response from model."

Fix: Two layers of protection:

1. toggleRunMode() auto-clears served_model_name, enable_tool_calling, and tool_call_parser when switching to remote mode
2. getConfig() explicitly skips these fields for remote mode as a safety net

Stale Observability Data After Mode Switch

After stopping a server (container, subprocess, or remote) and starting a new one in a different mode, the observability dashboard continued to display metrics from the previous session. The MetricStore and legacy metrics were not cleared on server stop.

Fix: start_server() now clears all metrics data (MetricStore.latest, MetricStore.history, latest_vllm_metrics, metrics_history) when a new server starts, so the observability dashboard begins fresh. Old metrics remain visible after stopping for post-mortem analysis.

Claude Code Not Detected on VPS/Linux

The Claude Code CLI installed via the official curl -fsSL https://claude.ai/install.sh | bash method installs to ~/.local/bin/claude, which is often not in the $PATH of the Python process running vLLM Playground. Similarly, npm global installs may go to ~/.npm-global/bin/.

Fix: `find_cl

vLLM Playground v0.1.6 - Observability Dashboard & Context Insights

Release Date: March 2026

This release adds a comprehensive Observability Dashboard with real-time vLLM server metrics, a PagedAttention Visualizer for KV cache insight, Token Counter & Logprobs Visualizer for per-token analysis, a Speculative Decoding Dashboard, and persistent user settings.

Real-time Observability Dashboard with auto-discovered vLLM metrics, category filtering, and threshold alerts.

Highlights

• Observability Dashboard - Full-page metrics dashboard with auto-discovery, time-series charts, and threshold alerts
• PagedAttention Visualizer - Real-time KV cache utilization heatmap with eviction alerts
• Token Counter & Logprobs - Live token estimation, per-token probability heatmap, and alternative token tooltips
• Speculative Decoding Dashboard - Acceptance rate, speedup factor, and token count tracking
• Settings Persistence - User preferences saved to ~/.vllm-playground/settings.json

Observability Dashboard

A dedicated full-page dashboard for monitoring vLLM server health and performance in real time. Access it from the Observability navigation item.

Overview Tab

Categorized metric cards with color-coded threshold indicators (normal / warning / danger):

All Metrics Tab

Searchable, sortable table of every metric exposed by the vLLM /metrics endpoint. Unregistered metrics auto-appear with sensible defaults -- no configuration needed.

Time Series Tab

Interactive time-series charts powered by uPlot:

• Select any combination of metrics to plot
• Configurable time windows (1m, 5m, 15m, 1h)
• History persists across tab switches and page navigations
• Export metric data as JSON

Alerts

Configurable threshold alerts with visual and historical tracking:

• Per-metric warning and danger thresholds
• Alert history log with timestamps
• Alerts badge in the navigation sidebar

Generic Metrics Architecture

A new modular architecture powers the dashboard:

Adding a new curated metric requires a single entry in the registry. The dashboard, sidebar, and alerts pick it up automatically.

PagedAttention Visualizer

Real-time visualization of vLLM's PagedAttention KV cache, displayed as a collapsible panel in the main chat view.

Features

• Block Heatmap - Canvas-rendered heatmap showing cache block utilization over time (up to 40 data points)
• Usage Gauge - Circular progress indicator for current KV cache usage percentage
• Prefix Cache Stats - Hit rate, total hits, and total queries
• Eviction Alerts - Three-level warning system:
  • Normal (< 90% usage)
  • Warning (90-95% usage)
  • Critical (> 95% usage, preemptions detected)

Data is sourced from the shared metrics poller, so enabling the visualizer adds no extra API calls.

Token Counter, Logprobs & Speculative Decoding

Live Token Counter

A real-time token estimator displayed alongside the chat input:

• Input estimation - Approximates token count as you type (using server-side /tokenize when av