大模型推理服务

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

Fast Multimodal LLM on Mobile Devices

Amica is an open source interface for interactive communication with 3D characters with voice synthesis and speech recognition.

A high-performance inference engine for LLM, VLM and DiT models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.

Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request)

Open-source AI computer builds, from 2-GPU desktops to 8-GPU servers: CAD, BOM, BIOS and build guides. Get the case kit with PCIe risers, or ask us for a free fit check.

VS Code extension for LLM-assisted code/text completion

The LLM Anti-Framework

Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs

List of software that allows searching the web with the assistance of AI: https://hf.co/spaces/felladrin/awesome-ai-web-search

Chat with your favourite LLaMA models in a native macOS app

Real-time two-way speech translation for bilingual meetings — auto-detects the spoken language and translates both directions, cloud or fully offline on-device. Desktop (Windows · macOS · Linux) + browser extension (Chrome · Edge) for Zoom, Meet, Teams & any app.

Run local LLMs like llama, deepseek-distill, kokoro and more inside your browser

日本語LLMまとめ - Overview of Japanese LLMs

Parallax is a distributed model serving framework that lets you build your own AI cluster anywhere

A curated collection of top-tier penetration testing tools and productivity utilities across multiple domains. Join us to explore, contribute, and enhance your hacking toolkit!

🦙 Integrating LLMs into structured NLP pipelines

📎 Clippy, now with some AI

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

LLMs as Copilots for Theorem Proving in Lean

An elegent pytorch implement of transformers

Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀

Gp.nvim (GPT prompt) Neovim AI plugin: ChatGPT sessions & Instructable text/code operations & Speech to text [OpenAI, Ollama, Anthropic, ..]

Jlama is a modern LLM inference engine for Java

Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

提取抖音无水印视频链接，视频文案，douyin-mcp-server，mcp，claude skill，支持龙虾

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

Fully Open Framework for Democratized Multimodal Training

🥤 RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with DuckDB or PostgreSQL

Official Inference Code of EAGLE-4

A practical lab for building, testing, and evaluating apps with Apple's Foundation Models framework.

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

识屏 Shiping：截屏 + AI，看懂屏幕上的任何内容。代码、报错、题目、图表、外文资料，按快捷键截下图就有解答。可作为桌面工具常驻后台，随时框选识别。对话模式实时听懂问题、给出回答要点。可用作桌面工具、面试助手、考试助手、刷题助手。

Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable

Streamlines and simplifies prompt design for both developers and non-technical users with a low code approach.

Starter templates for Obsidian

Unify Efficient Fine-tuning of RAG Retrieval, including Embedding, ColBERT, ReRanker.

The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on AI applications.

MAC (Multi-Agent CAD): A decoupled multi-agent framework for text-to-CAD generation via constrained test-time compute

Dynamiq is an orchestration framework for agentic AI and LLM applications

Run Laya, the open-source Jev-compatible System-1 decision model, from Node.js / TypeScript via ONNX Runtime

A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking.

RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.

UniRL is a Framework for Unified Multimodal Model Reinforcement Learning

Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking

“AI-Compass”将为社区指引在 AI 技术海洋中航行的方向，无论你是初学者还是进阶开发者，都能在这里找到通往 AI 各大方向的路径。旨在帮助开发者系统性地了解 AI 的核心概念、主流技术、前沿趋势，并通过实践掌握从理论到落地的全过程。

A throughput-oriented high-performance serving framework for LLMs

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

List of awesome hosting sorted by minimal plan price