标签：GGUF

仅需Python基础，从0构建大语言模型；从0逐步构建GLM4\Llama3\RWKV6， 深入理解大模型原理

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

Build a modern LLM from scratch. Every line commented. Explained like we are five.

Harness LLMs with Multi-Agent Programming

Llama3-中文后训练版

中文nlp解决方案(大模型、数据、模型、训练、推理)

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

Eagle: Frontier Vision-Language Models with Data-Centric Strategies

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.

The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.

A native macOS app that allows users to chat with a local LLM that can respond with information from files, folders and websites on your Mac without installing any other software. Powered by llama.cpp.

AI Chatbots in terminal for free

AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.

Data framework for your LLM applications. Focus on server side solution

PyTorch native quantization for training and inference

LLM Finetuning with peft

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

Run GLM-5.3 (744B), GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- and Anthropic-compatible API.

Maid is a free and open source application for interfacing with llama.cpp models locally, and with Anthropic, DeepSeek, Ollama, Mistral and OpenAI models remotely.

Local AI API Platform

Professional AI research tool for Zotero: chat with PDFs, whole-library search, Literature Radar, AutoPilot. 30+ LLMs or local models; built-in MCP server.

Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

[EMNLP-2024] Build multimodal language agents for fast prototype and production

LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite interfaces, such as o1,ollama, gemini, grok, qwen, GLM, deepseek, kimi,doubao. Adapted to local llms, vlm, gguf such as llama-3.3 Janus-Pro, Linkage graphRAG

Instant, controllable, local pre-trained AI models in Rust

A secure low code deception runtime framework, leveraging AI for System Virtualization.

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

Vim plugin for LLM-assisted code/text completion

Inference Llama 2 in one file of pure 🔥

Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-V supported. Accelerated by XNNPACK. Python, C# and JS(WASM) bindings available.

llama and other large language models on iOS and MacOS offline using GGML library.

Run LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.

Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

中文羊驼大模型三期项目 (Chinese Llama-3 LLMs) developed from Meta Llama 3

The leading open-source AI copilot for JetBrains. Connect to any model in any environment, and customize your coding experience in any way you like.

Make your own story. User-friendly software for LLM roleplaying

A curated list of awesome LLM and AI Agent Skills, resources and tools for customising AI Agent workflows - that works with Claude Code, Codex, Gemini CLI and your custom AI Agents

Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ.

A modern and easy-to-use client for Ollama

Create chatbots with ease

The React library for LLMs

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

Create characters in Unity with LLMs!

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率，量化感知训练的精度

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

Fast Multimodal LLM on Mobile Devices

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

Fun-ASR speech recognition models, with native Hugging Face Transformers support for Fun-ASR-Nano and separate FunASR, vLLM and llama.cpp deployment paths.