大模型推理服务

Plugin for JADX to integrate MCP server

High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

한국 문서를 읽고·고치고·새로 쓰는 AI 문서 도구 — HWP·HWPX·PDF·오피스·스캔(OCR)을 Markdown으로, 신구대조·서식 보존 편집·Markdown→HWPX 작성·HWP→HWPX 변환·양식 채우기·공문서 검수까지. 라이브러리·CLI·MCP | Read, diff, edit, write and convert Korean documents (HWP, HWPX, PDF, Office, scans) — library, CLI and MCP server

TokenSpeed is a speed-of-light LLM inference engine.

Use LLMs and LLM Vision (OCR) to handle paperless-ngx - Document Digitalization powered by AI

The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

Turn one topic into a finished Vox-style paper-collage explainer/ad video — automated end to end on Atlas Cloud + ffmpeg. An agent skill.

Use Hugging Face with JavaScript

Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

Turn any computer or edge device into a command center for your computer vision projects.

Open source implementation and extension of Google Research’s PaperBanana for automated academic figures, diagrams, and research visuals, expanded to new domains like slide generation.

ASR/STT subtitle generator. Uses Qwen3-ASR, local LLM, Whisper, TEN-VAD. Noise-robust for JAV

Reconstruct and validate C/C++ code from compiled programs with AI.

Let Claude (or any LLM) actually watch a video — scene-aware, deduplicated frames + transcript, from a URL or local file. Runs locally, MIT.

Instant, controllable, local pre-trained AI models in Rust

Run a 1-billion parameter LLM on a $10 board with 256MB RAM

Vim plugin for LLM-assisted code/text completion

A local inference engine for Apple silicon, built around the model.

The Agentic Framework of the PHP ecosystem to build production-ready AI driven applications. Connect components (LLMs, Tools, vector DBs, memory) to agents that interact with your data and UI.

Free, high-quality text-to-speech API endpoint to replace OpenAI, Azure, or ElevenLabs

MCP Router — development, support and security updates ended 2026-09-18. Historical source and final release for existing users.

Inference Llama 2 in one file of pure 🔥

Realtime Voice AI with 100+ Models on Arduino ESP32 with Secure Websockets and Edge Functions for AI Companions, and Devices

Completely free, private, UI based Tech Documentation MCP server. Designed for coders and software developers in mind. Easily integrate into Cursor, Windsurf, Cline, Roo Code, Claude Desktop App

llama and other large language models on iOS and MacOS offline using GGML library.

A curated list for Efficient Large Language Models

Your Cheat Sheet For AI Engineering Interviews at Top AI Companies - Questions and Answers.

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

Run LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.

Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

A community-driven collection of RAG (Retrieval-Augmented Generation) frameworks, projects, and resources. Contribute and explore the evolving RAG ecosystem.

Easy token price estimates for 400+ LLMs. TokenOps.

Community maintained hardware plugin for vLLM on Apple Silicon

中文羊驼大模型三期项目 (Chinese Llama-3 LLMs) developed from Meta Llama 3

The non profit fostering AI in India through credits, grants, resources

OpenAI API client for Kotlin with multiplatform and coroutines capabilities.

Make your own story. User-friendly software for LLM roleplaying

Control panel for VLLM, Sglang, llama.cpp, exllamav3

Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ.

A modern and easy-to-use client for Ollama

A股 AI 投研

High-performance, privacy-first on-device AI inference library for React Native, powered by PyTorch's ExecuTorch runtime

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

The React library for LLMs

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

Create characters in Unity with LLMs!

LLM Inference Engine for Metal, CUDA and Vulkan.

Awesome-Jailbreak-on-LLMs is a collection of state-of-the-art, novel, exciting jailbreak methods on LLMs. It contains papers, codes, datasets, evaluations, and analyses.

A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ultra-low latency.

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.