runanywhere-sdks

RunanywhereAI活跃维护

一套能在手机和边缘设备上本地跑大模型、扩散模型和多模态推理的生产级 SDK,覆盖 Android、iOS、Flutter,省去自己搭推理链路的麻烦。

Production ready toolkit to run AI locally

cpp-desktop-v0.20.43发布于 2026-10-28 05:16 UTC10,339star +2 / 7 天414fork同步于 2026-11-10 11:29 UTC

怎么上手

由 README 提炼,执行前请确认
pip install runanywhere
硬件要求按 README 说明

README 未给出具体显存或内存门槛;引擎按设备选择:Snapdragon Hexagon NPU 走 QHexRT,Apple 芯片走 MLX/Metal,NVIDIA 走 CUDA(需 opt-in 构建),浏览器走 WebGPU,其余回落到 llama.cpp 的 CPU。

纯 CPU 可运行支持 Apple Silicon需要 CUDA
下载:源码 zip·GitHub Releases(安装包)本站不镜像安装包

项目说明

由 README 自动翻译提炼

一套 SDK 在手机、浏览器、桌面和服务器上本地运行 LLM、语音、视觉与 RAG

一套能在手机和边缘设备上本地跑大模型、扩散模型和多模态推理的生产级 SDK,覆盖 Android、iOS、Flutter,省去自己搭推理链路的麻烦。

核心能力

  • 用能力注册表按设备自动路由到最优推理引擎
  • 覆盖 LLM 对话、VLM、STT、TTS、嵌入、RAG 与图像生成
  • 支持 token 流式输出、多轮历史和 LoRA 适配器
  • 提供 schema 校验的结构化 JSON 输出与本地工具调用
  • 语音代理把 VAD、STT、LLM、TTS 串成一条离线管线

容易踩的坑

首次调用会自动下载模型,需联网;CUDA 后端是 opt-in 构建,默认不启用;LiteRT 与 ExecuTorch 只是预留枚举值,尚未集成运行时;唤醒词检测未实现。

内容由模型根据 README 生成,可能不完整或不准确,以官方文档为准。看原文 README

最近发版

5 个版本cpp-desktop-v0.20.43
  1. cpp-desktop-v0.20.43cpp-desktop kit v0.20.43最新正式版2026-10-28 · 2026-10-28 05:16 UTC

    Desktop kits only. SDK release stays 0.20.37. Built from siddhesh/mlx-local-headers (LFM3 Omni and d1, llama.cpp runanywhere-b10453.10).

  2. cpp-desktop-v0.20.42cpp-desktop kit v0.20.422026-10-21 · 2026-10-21 21:09 UTC
    查看更新说明

    Kit-only release from runanywhere-sdks#1150: Recursive Flint GGUF on llama.cpp (runanywhere-b10453.9, flint arch). No SDK release; main stays at 0.20.37.

  3. cpp-desktop-v0.20.41cpp-desktop kit v0.20.412026-10-19 · 2026-10-19 22:40 UTC
    查看更新说明

    Kit-only release from runanywhere-sdks#1145: pplx-decider v1.2 GGUF decisions on llama.cpp, and Clef GGUF decisions restored on runanywhere-b10453.8. No SDK release; main stays at 0.20.37.

  4. cpp-desktop-v0.20.40cpp-desktop kits v0.20.402026-10-17 · 2026-10-17 09:51 UTC
    查看更新说明

    Desktop kits for 0.20.40. pplx-decider v1.1 and v1.2 on MLX; decision controls and images in the contract (IDL 1.6.0, plugin API 14).

  5. cpp-desktop-v0.20.39cpp-desktop kits v0.20.392026-10-09 · 2026-10-09 02:53 UTC
    查看更新说明

    Desktop kits for 0.20.39. D1 and GLiNER text decisions.

用户点评

登录用户的真实使用记录,可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译

    What you can build

    Every capability below runs fully on-device behind one semantic API across the eight SDKs. Call RunAnywhere.capabilities() (v4) to discover what the current package and device can actually execute — enum presence alone does not mean an engine is installed.

    • LLM chat: Llama, Qwen, Gemma, Phi, LFM, SmolLM, DeepSeek, and more, with token streaming, multi-turn history, and LoRA adapters
    • Structured output: schema-validated JSON; constrained decoding where the engine supports it (generateStructured + enforcement mode)
    • Tool calling: local function tools with stable call IDs and an agent loop (parallel calls when the engine/capability reports support)
    • Vision (VLM): image understanding, live camera description, and photo Q&A
    • Computer-use action parser (CUA): parse Fara1.5-style action strings into viewport-scaled coordinates — not a full autonomous agent framework
    • Speech-to-Text: Whisper and Moonshine transcription, batch and live frame streams
    • Text-to-Speech: neural voices from Piper, Kokoro, Kitten, MeloTTS, and Magpie
    • Voice agents: VAD, STT, LLM, and TTS in one pipeline with SpeechHandle-scoped playback (wake-word detection is not implemented)
    • Embeddings: L2-normalized vectors for search and retrieval
    • RAG: local document ingestion and retrieval-augmented answers, with streaming
    • Image generation: Stable Diffusion on Core ML, plus inpainting on the Hexagon NPU (platform/backend gated)

    Your code rarely picks hardware. Engines register what they can run, and the highest-priority engine that fits the device wins: QHexRT on the Snapdragon Hexagon NPU, MLX on Apple silicon, llama.cpp everywhere (Metal on Apple, CUDA on NVIDIA as an opt-in build, WebGPU in the browser), sherpa + ONNX for speech and embeddings, and Core ML for diffusion. LiteRT and ExecuTorch are reserved framework values only — they are not integrated runtimes yet.


    See it in action


    Quick start

    The fastest way to feel it. Install, load, generate, all local:

    pip install runanywhere
    
    import runanywhere as ra
    from runanywhere import LlmOptions
    
    ra.initialize()
    # downloads on first use
    print(ra.llm.generate("Explain on-device AI in one sentence.",
                          LlmOptions(model="qwen2.5-0.5b")).text)
    

    Prefer a terminal? Install the CLI from RunanywhereAI/wally — it consumes the C++ desktop kit this repo publishes:

    curl -fsSL https://raw.githubusercontent.com/RunanywhereAI/wally/main/install.sh | sh
    wally run qwen3 "Explain on-device AI in one sentence."
    

    Building for mobile, web, or desktop? Every platform below speaks the same API.

    import RunAnywhere
    import LlamaCPPRuntime
    
    // 1. Initialize
    LlamaCPP.register()
    try RunAnywhere.initialize()
    
    // 2. Load a model
    var load = RAModelLoadRequest()
    load.modelID = "smollm2-360m"
    load.category = .language
    load.framework = .llamaCpp
    _ = await RunAnywhere.loadModel(load)
    
    // 3. Generate
    var req = RALLMGenerateRequest()
    req.prompt = "What is the capital of France?"
    let result = try await RunAnywhere.generate(req)
    print(result.text) // "Paris is the capital of France."
    

    Add the MLX backend (import RunAnywhereMLX; MLX.register()) for Apple-native LLM, VLM, STT, TTS, and embeddings on Apple silicon.

    Install via Swift Package Manager:

    https://github.com/RunanywhereAI/runanywhere-sdks
    

    Documentation · Source

    import ai.runanywhere.proto.v1.ModelCategory
    import ai.runanywhere.proto.v1.SDKEnvironment
    import com.runanywhere.sdk.llm.llamacpp.LlamaCPP
    import com.runanywhere.sdk.public.RunAnywhere
    import com.runanywhere.sdk.public.extensions.*
    import com.runanywhere.sdk.public.types.RAModelInfo
    import com.runanywhere.sdk.public.types.RAModelLoadRequest
    
    // 1. Initialize (in a coroutine scope)
    LlamaCPP.register()
    RunAnywhere.initialize(
        context = this,
        environment = SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
    )
    
    // 2. Download and load a model
    val modelId = "smollm2-360m-instruct-q8_0"
    RunAnywhere.downloadModelStream(RAModelInfo(id = modelId)).collect { /* progress */ }
    RunAnywhere.loadModel(
        RAModelLoadRequest(model_id = modelId, category = ModelCategory.MODEL_CATEGORY_LANGUAGE),
    )
    
    // 3. Generate
    val result = RunAnywhere.generate("What is the capital of France?")
    println(result.text) // "Paris is the capital of France."
    

    Install via Gradle (Maven Central):

    dependencies {
        implementation("io.github.sanchitmonga22:runanywhere-sdk:0.20.27")
        implementation("io.github.sanchitmonga22:runanywhere-llamacpp:0.20.27")
        // Optional: STT / TTS / VAD
        // implementation("io.github.sanchitmonga22:runanywhere-onnx:0.20.27")
    }
    

    Documentation · Source

    import 'package:runanywhere/runanywhere.dart';
    import 'package:runanywhere_llamacpp/runanywhere_llamacpp.dart';
    
    // 1. Initialize
    LlamaCpp.register();
    await RunAnywhere.initialize();
    
    // 2. Download and load a model
    await RunAnywhere.downloadModel('smollm2-360m');
    await RunAnywhere.llm.load('smollm2-360m');
    
    // 3. Generate
    final response = await RunAnywhere.llm.chat('What is the capital of France?');
    print(response); // "Paris is the capital of France."
    

    Install via pub.dev:

    dependencies:
      runanywhere: ^0.20.27
      runanywhere_llamacpp: ^0.20.27  # LLM/VLM text generation
      # runanywhere_onnx: ^0.20.27    # STT, TTS, VAD, voice agent
      # runanywhere_mlx: ^0.20.27     # Apple-native LLM/VLM/STT/TTS/embeddings
      # runanywhere_qhexrt: ^0.20.27  # Snapdragon Hexagon NPU
    

    Documentation · Source

    import { RunAnywhere, SDKEnvironment } from '@runanywhere/core';
    import { LlamaCPP } from '@runanywhere/llamacpp';
    
    // 1. Initialize
    await RunAnywhere.initialize({ environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT });
    LlamaCPP.register();
    
    // 2. Download and load a model
    await RunAnywhere.downloadModel('smollm2-360m');
    await RunAnywhere.loadModel('smollm2-360m');
    
    // 3. Generate
    const result = await RunAnywhere.generate('What is the capital of France?');
    console.log(result.text); // "Paris is the capital of France."
    

    Install via npm:

    npm install @runanywhere/[email protected] @runanywhere/[email protected]
    # optional backends: @runanywhere/onnx @runanywhere/mlx @runanywhere/qhexrt
    

    Documentation · Source

    import { RunAnywhere, SDKEnvironment } from '@runanywhere/web';
    import { LlamaCPP } from '@runanywhere/web-llamacpp';
    
    // 1. Initialize
    await RunAnywhere.initialize({ environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT });
    await LlamaCPP.register({ acceleration: 'auto' }); // WebGPU when available, WASM otherwise
    await RunAnywhere.completeServicesInitialization();
    
    // 2. Load a model
    await RunAnywhere.loadModel({ modelId: 'qwen2.5-0.5b' });
    
    // 3. Generate
    const result = await RunAnywhere.generate({
      prompt: 'What is the capital of France?',
    });
    console.log(result.text); // "Paris is the capital of France."
    

    Install via npm:

    npm install @runanywhere/[email protected] @runanywhere/[email protected]
    # @runanywhere/web-onnx for STT/TTS/VAD/embeddings in the browser
    

    Source · Web starter app

    const { RunAnywhere } = require('@runanywhere/electron');
    
    // 1. Initialize
    RunAnywhere.initialize();
    
    // 2. Load a model (catalog id or a local path)
    const llm = await RunAnywhere.loadLLM('qwen2.5-0.5b');
    
    // 3. Generate (streaming)
    for await (const token of llm.generate('What is the capital of France?')) {
      process.stdout.write(token);
    }
    
    llm.unload();
    RunAnywhere.shutdown();
    

    A native N-API addon over the C core. Inference runs in an isolated Electron utility process and streams to the renderer over a MessagePort. LLM, VLM, STT, TTS, embeddings, RAG, structured output, tool calling, and a voice pipeline, with a prebuilt win32-x64 addon. CUDA is available as an opt-in source build.

    Install: build from source (Windows x64 preview), see the SDK README for steps.

    import runanywhere as ra
    from runanywhere import LlmOptions
    
    # 1. One call brings the SDK up
    ra.initialize()
    
    # 2. Stream tokens (the model auto-downloads and auto-loads)
    for event in ra.llm.generate_stream("What is the capital of France?",
                                        LlmOptions(model="qwen2.5-0.5b")):
        if event.is_token:
            print(event.text, end="", flush=True)
    
    # 2b. Or async
    # async for event in ra.llm.agenerate_stream("..."):
    #     ...
    
    # 3. Or grab the whole result, metrics included
    result = ra.llm.generate("Capital of France? One word.")
    print(result.text, result.tokens_per_second)   # "Paris" 41.2
    

    Namespaces per modality (llm, vlm, stt, tts, vad, embeddings, rag, models), an a-prefixed async twin for every blocking verb, structured output and tool calling, with prebuilt wheels that bundle the native runtime. CUDA is available as an opt-in source build.

    Install via pip:

    pip install runanywhere==0.20.27
    

    Source

    $ wally models pull qwen3
    pulling qwen3-0.6b ▕████████████▏ 100%  639 MB/639 MB  32 MB/s
    $ wally run qwen3 "Reply with exactly: RCLI WORKS" --no-think
    RCLI WORKS
    $ wally serve qwen3        # OpenAI-compatible API on :8080
    

    Also: --json on every command.

    Install (macOS Apple Silicon, Linux x86_64):

    curl -fsSL https://raw.githubusercontent.com/RunanywhereAI/wally/main/install.sh | sh
    

    Windows (x64):

    irm https://raw.githubusercontent.com/RunanywhereAI/wally/main/install.ps1 | iex
    

    CLI README


    SDKs

    SDKPlatformsStatusInstallDocs
    SwiftiOS 17.5+, macOS 14.5+StableSwift Package Managerdocs.runanywhere.ai/swift
    KotlinAndroid API 24+StableGradle (io.github.sanchitmonga22:runanywhere-sdk)docs.runanywhere.ai/kotlin
    FlutteriOS, AndroidBetapub.dev (runanywhere)docs.runanywhere.ai/flutter
    React NativeiOS, AndroidBetanpm (@runanywhere/core)docs.runanywhere.ai/react-native
    WebChromium, Safari, FirefoxBetanpm (@runanywhere/web)SDK README
    ElectronWindows x64 desktopPreviewBuild from sourceSDK README
    PythonWindows, macOS, LinuxAlphapip (runanywhere)SDK README
    wallymacOS, Linux, WindowsStableinstall scriptwally

    All SDKs ship on one version line, currently 0.20.37, from a single C++ core. Pin the same version across the core package and its backends. See Releases for what is published today.


    Features

    FeatureSwiftKotlinFlutterRNWebElectronPythonwally
    LLM generation + streamingYesYesYesYesYesYesYesYes
    Vision language models (VLM)YesYesYesYesYesYesYesYes
    Computer-use agent (CUA)YesYesYesYesAPI onlyn/an/an/a
    Speech-to-TextYesYesYesYesYesYesYesYes
    Text-to-SpeechYesYesYesYesYesYesYesYes
    Voice activity detectionYesYesYesYesYesYesYesYes
    Voice agent pipelineYesYesYesYesYesYesStubYes
    Wake wordNoNoNoNoNoNoNoNo
    EmbeddingsYesYesYesYesYesYesYesYes
    RAG (with streaming)YesYesYesYesYes*YesYesn/a
    Structured output (JSON)YesYesYesYesYesYesYesn/a
    Tool callingYesYesYesYesYesYesYesn/a
    Image generation (diffusion)YesYesYesYesn/an/aStubYes
    LoRA adaptersYesYesYesYesYesPartialStubYes
    Diarization (standalone)YesYesGatedYesYesn/aStubn/a
    SegmentationYesYesGatedYesYesn/aStubn/a
    capabilities() discoveryYesYesYesYesYesPartialYesn/a

    * Web RAG may be limited to one session per process — check capabilities().rag.multiSession. Stub / Gated / Partial mean the verb is absent, preflight-fails, or only partially wired; call capabilities() for the installed build. | Hexagon NPU (QHexRT) | n/a | Yes | Yes | Yes | n/a | n/a | n/a | n/a | | MLX (Apple silicon) | Yes | n/a | Yes | Yes | n/a | n/a | n/a | Yes | | OpenAI-compatible server | n/a | n/a | n/a | n/a | n/a | n/a | Yes | Yes | | Model download + progress | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | | Connect (LAN host/client) | Host (macOS) / Client (iOS, iPadOS) | Client | — | — | — | — | — | — |

    Connect (trusted LAN)

    Connect lets a macOS Swift app host a loaded language model on the local network so iOS, iPadOS, and Android clients can discover it and stream generation without downloading that model. It is app-scoped (lives with the host app process), not an OS daemon.

    RoleSupported todayNot in this release
    HostmacOS (Swift example / SDK)Windows, Electron, Web, RN, Flutter
    ClientiOS, iPadOS (Swift), Android (Kotlin)React Native, Flutter, Web, Electron
    • Discovery: Bonjour / NSD service type _runanywhere-connect._tcp
    • Transport: framed TCP on the LAN; commons owns protocol version, role policy, session accounting, and generation validation (idl/connect.proto, rac_connect_*)
    • Lifecycle: the host app selects and loads the model, starts hosting, and supplies generation; stopping the host disconnects clients
    • Threat model: trusted LAN only — no TLS, pairing PIN, or mutual auth in this release. Do not expose Connect across untrusted networks. Future work may add TLS/pairing, Windows hosting, or a daemon; those change lifecycle and security and are out of scope here
    • Electron note: RunAnywhereMain.connect() is local MessagePort / utility-process IPC inside one Electron app. It is unrelated to LAN Connect

    CUA on Web is "API only": the prompt/parse scaffold ships, but the catalogued Fara1.5-4B does not fit the 4 GB WASM32 heap, so no CUA model is seeded there.


    Inference engines

    Every SDK is a thin binding over runanywhere-commons, a single C++ core behind a pure C ABI. Engines plug into a capability registry and declare, per modality, what they can run. At inference time the highest-priority engine that serves the modality on the current device wins. Same code, different silicon, no branching in your app.

    EngineModalitiesRuns onNotes
    QHexRTLLM, VLM, STT, TTS, embeddings, inpaintingSnapdragon Hexagon NPU (v75 / v79 / v81)RunAnywhere's own NPU runtime, details below
    MLXLLM, VLM, STT, TTS, embeddingsApple siliconApple-native inference via mlx-swift, safetensors models
    llama.cppLLM, VLMEverywhere: Metal on Apple, CUDA opt-in on Windows/Linux, WebGPU + WASM in the browser, CPU with NEON/AVXGGUF models
    sherpa + ONNXSTT, TTS, VAD, embeddingsAll platformssherpa-onnx for speech, ONNX Runtime for embeddings and RAG
    Core MLImage generation (diffusion)iOS, macOSCore ML dispatches each layer across CPU, GPU, and the Apple Neural Engine
    PlatformApple Foundation Models, system TTSiOS, macOS, AndroidNative OS capabilities behind the same API
    CloudHybrid STTAll platformsOptional confidence-cascade routing to hosted providers

    MetalRT, RunAnywhere's proprietary GPU inference engine for Apple silicon, powers RCLI, our on-device voice assistant for macOS with local RAG and 40+ system actions at sub-200 ms latency. Signed binaries live at metalrt-binaries.


    Hexagon NPU acceleration (QHexRT)

    QHexRT is RunAnywhere's inference runtime for the Qualcomm Hexagon NPU. It runs LLM, vision, speech, and text-to-speech models directly on the Snapdragon NPU (Hexagon v75 / v79 / v81) and ships as a built-in accelerator: your app calls the same loadModel and generate, and it uses the NPU automatically on supported devices.

    • Runs LLM, VLM, speech-to-text, and text-to-speech on the NPU, including text-to-speech, which other runtimes run on the CPU.
    • Runs Mixture-of-Experts and hybrid-attention models on the NPU (Phi-tiny-MoE, Qwen3.5), plus the 1-bit Bonsai family up to Bonsai-27B (Hexagon v81).
    • Runs NVIDIA's Cosmos3-Edge and Magpie-TTS Multilingual, and handles embeddings and image inpainting (LaMa) on the NPU as well.
    • Hybrid streaming voice agents: LLM on the NPU, STT and TTS on the CPU, with sentence-by-sentence streaming playback.
    • Fast prefill and low time-to-first-token, with context that extends past the compiled window.
    • Prebuilt model bundles published on Hugging Face; the SDK downloads the one matching the device.

    Measured on a Samsung Galaxy S25 (Snapdragon 8 Elite, Hexagon v79):

    ModelTaskParamsDecodeTime to first token
    LFM2.5-230MLLM0.23 B164 tok/s32 ms
    Qwen3-0.6BLLM0.6 B33 tok/s (prefill up to 3,692 tok/s)127 ms
    Llama-3.2-1BLLM1.2 B16.3 tok/s56 ms
    Phi-tiny-MoEMoE LLM3.8 B (1.1 B active)5-7 tok/s~2.5 s
    InternVL3.5-1BVLM1 B37 tok/s290 ms
    Whisper baseASR74 M~5x real-timen/a
    MeloTTS-ENTTSn/a~4.5x real-timen/a

    Available on the Kotlin, Flutter, and React Native SDKs. Snapdragon (Android arm64) only.


    OpenAI-compatible server

    The Python SDK and wally both expose the local runtime as a drop-in OpenAI API, so anything that speaks the OpenAI client works against models running on your machine:

    pip install "runanywhere[server]"
    runanywhere serve        # http://127.0.0.1:8000
    
    from openai import OpenAI
    
    client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
    reply = client.chat.completions.create(
        model="qwen2.5-0.5b",
        messages=[{"role": "user", "content": "Hello from the edge."}],
    )
    

    Endpoints: /v1/chat/completions (streaming and non-streaming, text and vision), /v1/completions, /v1/embeddings, /v1/audio/transcriptions, /v1/audio/speech, and /v1/models. wally serve is LLM-only: it serves one model on port 8080 with /v1/chat/completions and /v1/models.


    RunAnywhere Console

    The Console is the control plane for on-device AI fleets. SDKs authenticate with an API key, register the device with its hardware profile, pull their assigned models, and report per-modality telemetry.

    • Deploy models over the air: assign catalog or bring-your-own models to an API key, and devices fetch them on their next sync. New models without an app release.
    • Device fleet: every registered device with its chip, memory, and NPU capability.
    • Analytics: usage, latency, and errors per modality across the fleet.
    • Benchmarks: compare local models against hosted providers.

    The Console is optional. Every SDK runs fully offline without an API key, and telemetry is scoped per modality when enabled.


    Models

    Hexagon NPU (QHexRT)

    Prebuilt bundles published on Hugging Face; the SDK downloads the one matching the device.

    ModelTaskParamsBundle
    Llama-3.2-1BLLM1.2 Bllama3_2_1b_HNPU
    LFM2.5-230M / 350MLLM0.23 / 0.35 Blfm2_5_230m_HNPU · lfm2_5_350m_HNPU
    LFM2.5-2.6BLLM2.6 Blfm2_5_2_6b_HNPU
    Qwen3.5-0.8B / 2B / 4BLLM0.8-4 Bqwen3_5_0_8b_HNPU · 2b · 4b
    Bonsai 1-bit familyLLM1.7 / 4 / 8 / 27 B1-bit and ternary builds; Bonsai-27B runs on Hexagon v81
    Gemma-4-E2B / E4BLLM + VLM~2 / 4 Bgemma4_e2b_HNPU · gemma4_e4b_HNPU
    Phi-tiny-MoEMoE LLM3.8 Bphi_tiny_moe_HNPU
    DeepSeek-R1-Distill-QwenLLM1.5 / 7 B1.5b · 7b
    Cosmos3-EdgeLLMedgeNVIDIA model family, Hexagon v79
    Qwen3-VL-2BVLM2 Bqwen3_vl_HNPU
    InternVL3.5-1BVLM1 Binternvl3_5_1b_HNPU
    Whisper base / smallASR74 / 244 Mwhisper_base_HNPU · whisper_small_HNPU
    Moonshine tiny / baseASRn/amoonshine_base_HNPU
    MeloTTS-ENTTSn/amelotts_en_HNPU
    Magpie-TTS MultilingualTTS357 Mmagpie_tts_357m_HNPU
    Kitten TTS mini / microTTSn/aHexagon v75
    EmbeddingGemma-300MEmbeddings300 Membeddinggemma_300m_HNPU

    Browse all models on Hugging Face

    Cross-platform

    TypeModelsEngine
    LLMSmolLM2, Qwen 3 / 2.5, Llama 3.2, LFM2, Mistral 7B (GGUF)llama.cpp
    LLM / VLM (Apple)Qwen3, SmolVLM2, and other mlx-community safetensors modelsMLX
    VLMSmolVLM2, LFM2-VL, Qwen2-VL (GGUF + mmproj)llama.cpp
    Speech-to-TextWhisper Tiny / Base, Moonshinesherpa + ONNX
    Text-to-SpeechPiper voices, Kokoro, Kitten TTSsherpa + ONNX
    VADSilero VADsherpa + ONNX
    EmbeddingsMiniLM, EmbeddingGemmaONNX Runtime
    Image generationStable DiffusionCore ML

    Anything not in the catalog can be pulled straight from Hugging Face or a direct URL; the core infers format, framework, and category from the artifact.


    Example apps

    Full consumer-assistant apps, one per platform, all built on the SDK. The iOS, Android, Web, and Electron apps live in their own repositories; the Flutter and React Native ones are still in this tree.

    PlatformSourceGet it
    iOSRunanywhereAI/runanywhere-iosApp Store
    AndroidRunanywhereAI/runanywhere-androidGoogle Play
    WebRunanywhereAI/runanywhere-webBuild from source
    ElectronRunanywhereAI/runanywhere-electronBuild from source (Windows)
    React Nativebindings/react-native/exampleBuild from source
    Flutterbindings/flutter/exampleBuild from source

    The Android, Flutter, and React Native apps include an NPU section that detects the device's Hexagon arch and runs LLM, vision, speech, and text-to-speech on the NPU.

    Minimal examples, the in-repo harnesses SDK contributors use to check a change end to end. Each builds the SDK from local source, so an edit shows up without a publish step:

    SDKHarnessRun it
    Swiftbindings/swift/example./run example ios run
    Kotlinbindings/kotlin/example./run example android install
    Webbindings/web/example./run example web dev

    Starters, minimal projects to copy from: Swift · Kotlin · Flutter · React Native · Web

    Real projects built on the stack:

    • RCLI: on-device voice assistant for macOS with local RAG and 40+ system actions, powered by MetalRT

    Repository layout

    Business logic lives in the C++ core, so one fix lands on all eight SDKs at once.

    runanywhere-sdks/
    ├── core/                           # Shared C/C++ core behind a C ABI — all business logic
    │
    ├── bindings/                       # Thin language bindings over core/
    │   ├── swift/                      # iOS/macOS SDK (XCFramework) + example/
    │   ├── kotlin/                     # Android SDK (JNI) + example/
    │   ├── flutter/                    # Flutter SDK (Dart FFI) + example/
    │   ├── react-native/               # React Native SDK (Nitro/JSI) + example/
    │   ├── web/                        # Web SDK (WebAssembly / WebGPU) + example/
    │   ├── electron/                   # Electron SDK (N-API addon) + example/
    │   ├── python/                     # Python SDK (pybind11) + example/
    │   ├── proto-ts/                   # @runanywhere/proto-ts, the IDL's TypeScript binding
    │   └── shared-apple/               # Apple transport shared by the RN + Flutter bindings
    │                                   # (iOS/Android/Web/Electron consumer apps live in their own repos)
    │
    ├── engines/                        # llamacpp, mlx, sherpa, onnx, neurt, qhexrt, cloud
    ├── runtimes/                       # cpu, coreml, onnxrt compute adapters
    ├── idl/                            # Protobuf schemas, generated bindings per language
    └── docs/                           # Documentation
    

    Requirements

    PlatformMinimum
    iOS17.5+
    macOS14.5+
    AndroidAPI 24 (7.0), arm64 recommended
    WebChrome 96+ / Edge 96+, Chrome 120+ for WebGPU
    React Native0.83.1+, 0.85+ recommended (Node.js 22.12+)
    Flutter3.44+ (Dart 3.12+)
    ElectronWindows x64 (preview)
    Python3.9+ on Windows, macOS, Linux (3.12+ recommended)
    wallymacOS arm64, Linux x86_64, Windows x86_64

    Hexagon NPU: Snapdragon with Hexagon v75 / v79 / v81, Android arm64. MLX: Apple silicon, physical devices. Memory: 2 GB minimum, 4 GB+ recommended for larger models.


    Contributing

    We welcome contributions. See the Contributing Guide for setup and conventions.

    git clone https://github.com/RunanywhereAI/runanywhere-sdks.git
    cd runanywhere-sdks
    
    # Doctor / setup helpers
    ./run doctor
    ./run setup
    
    # Build the native XCFrameworks into bindings/swift/Binaries/.
    # Required for local Swift development.
    ./bindings/swift/scripts/build-core-xcframework.sh
    
    # Stream one completion through the minimal Swift harness
    cd bindings/swift/example
    RUNANYWHERE_USE_LOCAL_NATIVES=1 swift run
    

    Community


    License

    RunAnywhere License (Apache 2.0 based, with additional commercial-use terms). See LICENSE for details.