标签：CUDA

本地跑完整语音流水线，替掉 ElevenLabs 的订阅和上传环节

单卡也能压榨出高吞吐的 LLM 推理引擎，PagedAttention 基本成了行业标准

本地语音工作室，把声音克隆、多引擎 TTS 和全局听写合并进一个桌面应用

前缀复用做得最好的推理框架，RAG 与多轮对话场景吞吐明显占优

把 NeRF/SDF 训练从小时级压到秒级的多分辨率哈希编码参考实现，同时是一套可交互的神经图形实验台

用 Rust 写训练代码，同一份代码直接跑推理，省掉导出模型这一步

传统 HMM-GMM / TDNN 语音识别流水线的参考实现，用来复现论文级 baseline 或给新模型做对齐和打分

NVIDIA 官方 LLM 推理引擎，用预编译 engine 和专用 kernel 替代通用 PyTorch 推理路径

点云与网格处理的基础设施层，替代自己拿 PCL 或手写 KD-tree、ICP、泊松重建

把 vLLM 这类引擎的 KV 缓存抽成独立进程，跨实例复用 prefill 结果，专治长上下文首 token 慢

把模型代码和依赖声明直接编译成可推 registry 的推理镜像，替掉手写 Dockerfile 和 CUDA 版本对照表

在 Go 服务里直接调 OpenCV，省掉 CGO 桥接层和 Python 推理进程

用 NVML 直读替代 nvidia-smi 文本解析的交互式 GPU 进程查看器，定位是 GPU 版的 htop

LLM 推理服务里的 attention/GEMM/MoE 内核层，顶替 vLLM、SGLang 里手写或第三方 CUDA 算子

把散落的 GPU 机器收成一个可调度的推理集群，顶替手写 docker run 加 vLLM 启动脚本那套

Optimized primitives for collective multi-GPU communication

A PyTorch Library for Accelerating 3D Deep Learning Research

Fast inference engine for Transformer models

Lightning fast C++/CUDA neural network framework

A retargetable MLIR-based machine learning compiler and runtime toolkit.

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

Train, inspect, edit, automate, and export 3D Gaussian Splatting scenes from a single native application.

Self-hosted, local only NVR and AI Computer Vision software. With features such as object detection, motion detection, face recognition and more, it gives you the power to keep an eye on your home, office or any other place you want to monitor.

AI 基础知识 - GPU 架构、CUDA 编程、大模型基础及AI Agent 相关知识。

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

PyTorch native quantization for training and inference

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

Run GLM-5.3 (744B), GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- and Anthropic-compatible API.

CV-CUDA™ is an open-source, GPU accelerated library for cloud-scale image processing and computer vision.

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

computer vision projects | 计算机视觉相关好玩的AI项目（Python、C++、embedded system）

LUPINE is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.

Cross-vendor 3D Gaussian Splatting trainer - video to splat to mesh, Vulkan or CUDA.

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

NVIDIA DeepStream SDK 8.0 / 7.1 / 7.0 / 6.4 / 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 / 5.1 implementation for YOLO models

Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

OneDiff: An out-of-the-box acceleration library for diffusion models.

Real-time dense visual SLAM system

Large-scale LLM inference engine

Ultrafast serverless GPU inference, sandboxes, and background jobs

High Performance Serving Runtime For Interactive Super Intelligence System

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

🔥 Real-time NVIDIA GPU dashboard

Self-host the powerful Chatterbox TTS model. This server offers a user-friendly Web UI, flexible API endpoints (incl. OpenAI compatible), predefined voices, voice cloning, and large audiobook-scale text processing. Runs accelerated on NVIDIA (CUDA), AMD (ROCm), and CPU.

🔥🔥🔥TensorRT for YOLO26 (det/pose/seg/depth/obb/cls), YOLOv11,YOLOv10 YOLOv9,YOLOv8, ... 🚀🚀🚀 CUDA IS ALL YOU NEED. 🍎🍎🍎

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.

Efficient Deep Learning Systems course materials

Cross-platform, customizable multimedia/video processing framework. With strong GPU acceleration, heterogeneous design, multi-language support, easy to use, multi-framework compatible and high performance, the framework is ideal for transcoding, AI inference, algorithm integration, live video streaming, and more.