大模型推理服务

Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM

Zero-modification Human-in-the-Loop adapter for OpenOPC's agentic DAG runtime.

Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets

An autonomous, context-aware AI desktop companion. Built with Python, featuring real-time screen vision, custom ONNX voice synthesis, active window tracking, and a dynamic floating UI with reactive facial expressions.

cube studio开源云原生一站式机器学习/深度学习/大模型AI平台，mlops算法链路全流程，算力租赁平台，notebook在线开发，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务VGPU虚拟化，边缘计算，标注平台自动化标注，deepseek等大模型sft微调/奖励模型/强化学习训练，vllm/ollama/mindie大模型多机推理，私有知识库，AI模型市场，支持国产cpu/gpu/npu 昇腾生态，支持RDMA，支持pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano等分布式

Curated list of the best truly open-source AI projects, models, tools, and infrastructure. Daily updated.

MockServer is an HTTP(S) mock server and proxy for testing that lets you mock APIs, inspect and modify live traffic, and inject failures. It supports HTTP/1.1, HTTP/2, gRPC, WebSockets, TCP and more on a single port, with additional support for HTTP/3, message brokers, and AI/LLM APIs.

Repo for BenCao [original name: HuaTuo (华驼)], Instruction-tuning Large Language Models with Chinese Medical Knowledge. 本草（原名：华驼）模型仓库，基于中文医学知识的大语言模型指令微调

Fast inference engine for Transformer models

RuVector provides High Performance, Real-Time decisions and agent memory , Self-Learning Ai, Vector GNN DB built in Rust.

StarVector is a foundation model for SVG generation that transforms vectorization into a code generation task. Using a vision-language modeling architecture, StarVector processes both visual and textual inputs to produce high-quality SVG code with remarkable precision.

AI Skills, MCP Tools, and CLI for Unity Engine. Full AI develop and test loop. Use cli for quick setup. Efficient token usage, advanced tools. Any C# method may be turned into a tool by a single line. Works with Claude Code, Gemini, Copilot, Cursor and any other absolutely for free.

TypeDB: Built for systems, not records

🤖 A visualization mcp & skills contains 25+ visual charts using @antvis. Using for chart generation and data analysis.

Optimizing inference proxy for LLMs

An open-source model family for long-form speech, dialogue synthesis, voice design, sound effects, and real-time streaming TTS

Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

Everything you need to know to build your own RAG application

State of the Art Natural Language Processing

Harness LLMs with Multi-Agent Programming

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.

The missing DevTools for Claude Code — inspect session logs, tool calls, token usage, subagents, and context window in a visual UI. Free, open source.

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

A股多Agent投研框架 — 适配A股数据源(龙虎榜/游资/解禁等)，7位分析师基于A股规则的辩论决策，基于TradingAgents深度改造，适配大A。A-share multi-agent investment research framework — 7 AI analysts, bull/bear debate, risk assessment。

Cascading runtime for AI agents. Optimize cost, latency, quality, and policy decisions inside the agent loop.

中文nlp解决方案(大模型、数据、模型、训练、推理)

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

Token conscious database MCP server for Postgres, MySQL, SQL Server, Oracle, MariaDB, SQLite.

A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

The official repo of MiniMax-Text-01 and MiniMax-VL-01, large-language-model & vision-language-model based on Linear Attention

总结Prompt&LLM论文，开源数据&模型，AIGC应用

AI 基础知识 - GPU 架构、CUDA 编程、大模型基础及AI Agent 相关知识。

[CVPR 2023] DepGraph: Towards Any Structural Pruning; LLMs, Vision Foundation Models, etc.

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

A high-performance inference engine for AI models

The Security Toolkit for LLM Interactions

Document (PDF, Word, PPTX ...) extraction and parse API using state of the art modern OCRs + Ollama supported models. Anonymize documents. Remove PII. Convert any document or picture to structured JSON or Markdown

Skywork-R1V is an advanced multimodal AI model series developed by Skywork AI, specializing in vision-language reasoning.

Sparsity-aware deep learning inference runtime for CPUs

Add a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query, search, and LLM-inference engine in Rust for data-grounded AI apps and agents.

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

Community maintained hardware plugin for vLLM on Huawei Ascend

Jupyter Notebooks to help you get hands-on with Pinecone vector databases

PyTorch native quantization for training and inference

Enhances Tesseract OCR output using LLMs (local or API) for error correction, smart chunking, and markdown formatting of scanned PDFs