ome

OME 1.2.1

1.2.1 is a small patch release about onboarding a new model family and removing a duplicated source of truth. ThinkingMachines Inkling becomes a servable model family end to end — a new audio-text capability, a capability rule that survives the vision short-circuit, and an NVFP4 runtime — while the GPU instance-type map collapses from two hand-synced copies into one.

✨ Highlights

ThinkingMachines Inkling Support (#672, #674)

Inkling is multimodal in a way the capability pipeline couldn't previously express: it takes audio and text and returns text. This release adds that capability and the runtime to serve it:

• New AUDIO_TEXT_TO_TEXT capability — added to the ModelCapability enum, the modelconfig capability set, and the modelparser → OME capability mapping, so a parsed Inkling model surfaces the capability on its BaseModel status (#672)
• Inkling capability rule — models whose type contains inkling_mm_model resolve to both TEXT_TO_TEXT and AUDIO_TEXT_TO_TEXT; the rule is ordered ahead of the vision rule because the vision short-circuit would otherwise return early and silently drop the audio-text output (#672)
• model_max_length from nested configs — the nested LLM config is now consulted for model_max_length, so checkpoints that only declare it on the inner config no longer parse as 0 (#672)
• tokenspeed-inkling-nvfp4 runtime — a ServingRuntime for InklingForConditionalGeneration safetensors checkpoints with nvfp4 quantization, autoSelect enabled at priority 1, and a router + ome-container topology requesting 4 GPUs per replica (#674)

One Instance-Type Map Instead of Two (#673)

The instance-type → short-name mapping was maintained in two places that had to agree: the model-agent ConfigMap and the enigma init container's environment. Adding a GPU shape meant editing both, and forgetting one was silent.

• Single source of truth — the mapping now lives only in modelAgent.instanceTypeMap in values.yaml, rendered by a new ome.instanceTypeMap helper into both the model-agent ConfigMap (instance-type-map) and the enigma init container's INSTANCE_TYPE_MAP env var
• Onboarding a GPU shape is one edit — add the shape to values.yaml and both consumers pick it up
• Broader shape coverage — OCI, AWS, Azure, GCP, CoreWeave, and Nebius shapes are enumerated in one place, including B200 and B300

⚠️ Upgrading:

• Apply the updated CRDs before using AUDIO_TEXT_TO_TEXT. It is a new value in a +kubebuilder:validation:Enum, so a BaseModel carrying it is rejected at admission until the new CRD is applied (#672).
• Move any instance-type-map override into values.yaml. If you patched the mapping directly in the model-agent ConfigMap, that edit is now overwritten on upgrade and does not reach the enigma init container — set modelAgent.instanceTypeMap instead so both consumers stay in sync (#673).

🚀 Features & Enhancements

• AUDIO_TEXT_TO_TEXT model capability, wired through the ModelCapability enum, modelconfig, and the modelparser capability mapping (#672)
• Inkling capability rule resolving inkling_mm_model to TEXT_TO_TEXT + AUDIO_TEXT_TO_TEXT, ordered ahead of the vision rule so the vision short-circuit no longer drops the audio-text output (#672)
• model_max_length is now merged from nested LLM configs during model config parsing (#672)
• New tokenspeed-inkling-nvfp4 ServingRuntime for NVFP4 InklingForConditionalGeneration checkpoints, auto-selected at priority 1 (#674)
• Instance-type → short-name map single-sourced in modelAgent.instanceTypeMap and rendered into both the model-agent ConfigMap and the enigma init container via the ome.instanceTypeMap helper (#673)

♻️ Maintenance

• Bump bundled image references to v1.2.0 across the Helm chart, the InferenceService ConfigMap, and the kustomize manager / model-agent overlays (#671)

📦 Container Images

The following container

OME 1.2.0

1.2.0 is about where model weights come from and what's still in the API. Pre-provisioned and air-gapped weights on shared storage become a first-class model source, the long-deprecated predictor field is finally removed, and the model-agent's task handling gets meaningfully sturdier under deletion, reuse, and churn.

✨ Highlights

PVC-Backed BaseModels (#653, #654, #655, #656, #657, #658, #659, #668)

Until now a pvc:// BaseModel could be parsed but never acted on — nothing is downloaded for a PVC model, so it sat in Importing/In_Transit forever while the controller waited on per-node status ConfigMaps that never arrived. This release closes that gap and makes PVC a first-class, Ready-reaching model source:

• Serve from shared storage — point a BaseModel or ClusterBaseModel at pvc://… (CSI, NFS, Lustre) and it reaches Ready with no per-node download and no egress — ideal for pre-provisioned or air-gapped weights
• On-demand metadata — a short-lived ome-agent model-metadata Job parses the model on the claim and publishes a per-model status ConfigMap the controller reconciles, instead of writing the CR directly (#668)
• Mounted read-only into serving pods — the claim is mounted at the model path with the correct sub-path; the per-node model-ready affinity is skipped since PVCs aren't node-scoped (#656)
• Validated at admission — a validating webhook rejects malformed or wrongly-scoped pvc:// URIs at kubectl apply time instead of surfacing later as a Failed status (#657, #658)
• Condition-driven status — model status gains Conditions, ObservedGeneration, LastReconcileTime, and SourceReachable/Ready conditions with PVC-lifecycle reasons (PVCNotBound, MetadataExtracting, Validated, ExtractionFailed, …) (#653)
• Backend registry seam — the BaseModel controller is now a thin dispatcher over pluggable Backends, so the PVC path stays fully isolated from the per-node download path (#654)

Namespacing: a namespaced BaseModel uses pvc://{claim}/{sub-path} (its own namespace is implied); a ClusterBaseModel must scope the claim explicitly with pvc://{namespace}:{claim}/{sub-path}.

The Deprecated predictor Field Is Removed (#661)

• spec.predictor and the entire PredictorSpec type are gone — serving is now configured exclusively via spec.engine + spec.model (plus spec.decoder / spec.router for disaggregated setups)
• Removes ~22k lines, the bulk being the predictor schema stripped from the InferenceService CRD and the regenerated OpenAPI / deepcopy / defaults
• Breaking: applying any manifest that still contains predictor now fails (unknown field "spec.predictor"); GitOps/CI syncs of stale manifests hard-fail until predictor is removed from source
• No disruption to running services — a stale predictor: {} left on an existing object is pruned automatically on the next write (a reconcile, kubectl edit, even kubectl annotate); no manual object cleanup required

Sturdier Model-Agent Task Handling (#650, #660, #663)

• Deletion and reuse no longer wait behind downloads — a dedicated high-priority worker path runs lightweight delete and same-path reuse/wait work so it isn't blocked behind long-running download/validation; deletion state is re-checked before starting work and before writing Ready, so a stale queued download can't mark an already-deleted model Ready (#650)
• Metadata-only spec updates stop forcing re-downloads — parser-populated fields (model type, framework, format, capabilities) no longer trigger a DownloadOverride; storage, label/annotation, and TensorRT-LLM routing changes still do (#660)
• No more reconcile churn on modelConfiguration — a sorted-key vs declaration-order byte mismatch made the agent rewrite the BaseModel spec on every reconcile; the field is now preserve-if-unset like every other field (#663)

⚠️ **Upgra

OME 1.1.0

This release is about controlling when runtime changes reach your services: InferenceServices can now pin to immutable ServingRuntime snapshots with explicit roll-forward and rollback, explicit runtime selection no longer hard-fails on compatibility, and model downloads got faster and more resilient.

✨ Highlights

Runtime Revision Pinning (#639, #640)

Until now, every InferenceService tracked its ServingRuntime live — editing a shared runtime rippled into every service referencing it, immediately, with no staging and no way back. This release makes that coupling opt-out:

• Pin — set spec.runtime.autoSync: false and the service freezes on an immutable, content-addressed snapshot of its runtime, backed by ControllerRevisions
• Roll forward — bump the ome.io/runtime-sync annotation to re-sync to the latest runtime on your own schedule
• Roll back — set spec.runtime.revision to return to any retained snapshot
• See drift — a RuntimeDrifted status condition surfaces when a pinned service has fallen behind the live runtime
• Stay clean — stale snapshots are garbage-collected automatically, with tunable retention and grace-period flags
• No surprises — live tracking remains the default; nothing changes unless you opt in

Explicit Runtime Compatibility Is Now Advisory (#643)

• Naming a runtime explicitly with a declared model-format or architecture mismatch no longer hard-fails the InferenceService
• The service proceeds and emits a warning event instead, matching the admission webhook's existing behavior
• Generic runtimes can now serve architectures they don't enumerate — you stay informed without being blocked

Faster, More Robust Model Downloads (#629, #644, #645)

• OCI Object Storage downloads are skipped when a same-path artifact is already Ready — nodes stop re-pulling weights they already have (#629)
• The same-path shortcut is bypassed when a download override is set, so forced re-downloads still behave as expected (#645)
• HuggingFace downloads in the replica agent now have timeouts — stalled transfers fail fast instead of hanging (#644)

⚠️ Upgrading: runtime revision pinning requires the controller-manager to have create on controllerrevisions.apps. The bundled charts and manifests include this grant (#646) — if you manage RBAC out of band, add it before pinning any service.

🚀 Features & Enhancements

• Content-addressed runtime snapshot machinery, backed by ControllerRevisions (#639)
• Runtime revision controls on InferenceService: spec.runtime.autoSync, spec.runtime.revision, the RuntimeDrifted condition, and snapshot GC flags (#640)
• Explicit-runtime compatibility mismatches downgraded from a hard failure to a warning event (#643)
• HuggingFace download timeout for the replica agent (#644)
• Skip OCI Object Storage downloads for Ready same-path artifacts (#629); bypass the reuse when a download override is set (#645)

🐛 Bug Fixes

• Grant the controller-manager create on controllerrevisions.apps, required for runtime revision pinning (#646)

📚 Documentation

• Docs backfill (#648):
  • Runtime revisions guide
  • Controller-manager configuration
  • Quantization, including compressed-tensors
  • Annotations & labels reference
  • Rewritten fine-tuned-weights guide
  • Regenerated API reference
• Fixed the GitHub Pages build on Hugo 0.157.0 / Docsy v0.15.0 (#642); bumped autoprefixer 10.4.23 → 10.5.0

♻️ Maintenance

• Hardened the xet cargo build against transient network failures (#641)

📦 Container Images

The following container images are available:

# OME Manager
docker pull ghcr.io/moirai-internal/ome-manager:v1.1.0

# Model Agent
docker pull ghcr.io/moirai-internal/model-agent:v1.1.0

# OME Agent
docker pull ghcr.io/moirai-internal/ome-agent:v1.1.0

# Multinode Prober
docker pull ghcr.io/moirai-internal/multinode-prober:v1.1.

OME 1.0.0

What's NEW

⚠️ Breaking Changes Module path renamed from github.com/sgl-project/ome to sigs.k8s.io/ome (#620). Update all imports — anything depending on OME as a Go module must change its import paths accordingly. 🚀 Features Add DeepSeek v4 runtimes — new DeepSeek-V4-Flash and DeepSeek-V4-Pro model + vLLM runtime configs (#598) AUDIO_TO_TEXT capability detection for ASR models (#581) Expose container startup failures in InferenceService status, surfacing crash/startup errors to users (#616) Recognize compressed-tensors as a quantization method in the model parser (#637) 🐛 Bug Fixes Return pod webhook errors instead of panicking (#619) Fix InferenceService pod status lookup for long raw-deployment names (#618) Trigger model redownload on spec/label/annotation changes (#552) Override data parallel size for accelerator configs (#611) Fix mixed-case check against lowercased normalizedModelType (#591) ♻️ Refactors Refactor model parser (#635) Refactor AcceleratorClass cheapest-selection iteration (#548) Add multi-modal model fallback in generic model config parsing (#589) 📚 Docs Update documentation site URLs (#605) 🔧 Maintenance / CI Add pre-commit hooks and wire them into PR validation, with failure annotations (#562, #572, #573, #571) Fix static-analysis / staticcheck findings across pkg/ and the model agent (#558, #561, #554) Repo hygiene: end-of-file newlines, trailing whitespace, spelling fixes (#565, #563, #571) Update outdated HF model config testdata (#586) Helm: bump default image and remaining references to v0.1.5 (#582, #592) Bump actions/deploy-pages 4 → 5 (#551)

📦 Container Images

The following container images are available:

# OME Manager
docker pull ghcr.io/moirai-internal/ome-manager:v1.0.0

# Model Agent
docker pull ghcr.io/moirai-internal/model-agent:v1.0.0

# OME Agent
docker pull ghcr.io/moirai-internal/ome-agent:v1.0.0

# Multinode Prober
docker pull ghcr.io/moirai-internal/multinode-prober:v1.0.0

⎈ Helm Installation

Option 1: OCI Registry (Recommended)

# Install directly from OCI registry
helm install ome-crd oci://ghcr.io/moirai-internal/charts/ome-crd --version 1.0.0 --namespace ome --create-namespace
helm install ome oci://ghcr.io/moirai-internal/charts/ome-resources --version 1.0.0 --namespace ome

Option 2: GitHub Releases

# Add the OME Helm repository
helm repo add ome https://github.com/ome-projects/ome/releases/download/v1.0.0
helm repo update

# Install OME
helm install ome-crd ome/ome-crd --namespace ome --create-namespace
helm install ome ome/ome-resources --namespace ome

📋 Changelog

https://github.com/ome-projects/ome/compare/v0.1.5...v1.0.0

📄 Software Bill of Materials

SBOMs are available in both SPDX and CycloneDX formats for:

• Source code
• All container images

🔐 Signatures

All container images are signed with cosign. Verify with:

cosign verify ghcr.io/moirai-internal/ome-manager:v1.0.0   --certificate-identity-regexp=https://github.com/ome-projects/ome/.github/workflows/release.yaml@refs/tags/.*   --certificate-oidc-issuer=https://token.actions.githubusercontent.com

What's Changed

• [Core]refactor acceleratorclass cheapest selection iteration by @pallasathena92 in https://github.com/ome-projects/ome/pull/548
• [Misc]Fix static analysis issues in model agent by @pallasathena92 in https://github.com/ome-projects/ome/pull/554
• Bump actions/deploy-pages from 4 to 5 by @dependabot[bot] in https://github.com/ome-projects/ome/pull/551
• [Misc] Fix staticcheck findings under pkg/ by @heymrbox in https://github.com/ome-projects/ome/pull/558
• [Misc]add pre-commit hooks by @pallasathena92 in https://github.com/ome-projects/ome/pull/562
• [Misc]Fix static analysis issues by @pallasathena92 in https://github.com/ome-projects/ome/pull/561
• [Misc]trail whitespace in repo by @pallasathena92 in https://git