标签：CUDA

VoiceStudio is the open-source, fully-local ElevenLabs alternative — voice cloning, voice design, video dubbing, dictation, transcription & audiobook creation in 646 languages.

单卡也能压榨出高吞吐的 LLM 推理引擎，PagedAttention 基本成了行业标准

The open-source AI voice studio. Clone, dictate, create.

前缀复用做得最好的推理框架，RAG 与多轮对话场景吞吐明显占优

Instant neural graphics primitives: lightning fast NeRF and more

Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.

kaldi-asr/kaldi is the official location of the Kaldi project.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

Open3D: A Modern Library for 3D Data Processing

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

Containers for machine learning

Go package for computer vision using OpenCV 4 and beyond. Includes support for DNN, CUDA, OpenCV Contrib, and OpenVINO.

An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.

FlashInfer: Kernel Library for LLM Serving

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

Optimized primitives for collective multi-GPU communication

A PyTorch Library for Accelerating 3D Deep Learning Research

Fast inference engine for Transformer models

Lightning fast C++/CUDA neural network framework

A retargetable MLIR-based machine learning compiler and runtime toolkit.

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

Train, inspect, edit, automate, and export 3D Gaussian Splatting scenes from a single native application.

Self-hosted, local only NVR and AI Computer Vision software. With features such as object detection, motion detection, face recognition and more, it gives you the power to keep an eye on your home, office or any other place you want to monitor.

AI 基础知识 - GPU 架构、CUDA 编程、大模型基础及AI Agent 相关知识。

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

PyTorch native quantization for training and inference

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

Run GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- and Anthropic-compatible API.

CV-CUDA™ is an open-source, GPU accelerated library for cloud-scale image processing and computer vision.

computer vision projects | 计算机视觉相关好玩的AI项目（Python、C++、embedded system）

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

LUPINE is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.

Cross-vendor 3D Gaussian Splatting trainer - video to splat to mesh, Vulkan or CUDA.

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

NVIDIA DeepStream SDK 8.0 / 7.1 / 7.0 / 6.4 / 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 / 5.1 implementation for YOLO models

Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

OneDiff: An out-of-the-box acceleration library for diffusion models.

Real-time dense visual SLAM system

Large-scale LLM inference engine

Ultrafast serverless GPU inference, sandboxes, and background jobs

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

High Performance Serving Runtime For Interactive Super Intelligence System

🔥 Real-time NVIDIA GPU dashboard

Self-host the powerful Chatterbox TTS model. This server offers a user-friendly Web UI, flexible API endpoints (incl. OpenAI compatible), predefined voices, voice cloning, and large audiobook-scale text processing. Runs accelerated on NVIDIA (CUDA), AMD (ROCm), and CPU.

🔥🔥🔥TensorRT for YOLO26 (det/pose/seg/depth/obb/cls), YOLOv11,YOLOv10 YOLOv9,YOLOv8, ... 🚀🚀🚀 CUDA IS ALL YOU NEED. 🍎🍎🍎

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.

Efficient Deep Learning Systems course materials

Cross-platform, customizable multimedia/video processing framework. With strong GPU acceleration, heterogeneous design, multi-language support, easy to use, multi-framework compatible and high performance, the framework is ideal for transcoding, AI inference, algorithm integration, live video streaming, and more.