登录

awesome-cuda-and-hpc

coderonion长期未更新

收录 CUDA、cuBLAS、CUTLASS、TensorRT-LLM、Triton、TVM、MLIR 等 GPU 加速与高性能计算开源项目的精选清单，方便按方向找底层算子、编译器和推理框架。

🚀🚀🚀 This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.

暂无版本524star50fork同步于 2026-11-07 07:31 UTC
CUDA高性能计算GPU加速张量编译器推理框架
对比 GitHub 官网awesomeblascublascudacudnn

怎么上手

由 README 提炼，执行前请确认
原文 
README 未说明
硬件要求按 README 说明

README 未说明

需要 CUDA支持 ROCm / AMD多卡 / 张量并行
下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

一份 CUDA 与 HPC 生态的精选资源索引，帮你快速找到框架、学习资料和博客

收录 CUDA、cuBLAS、CUTLASS、TensorRT-LLM、Triton、TVM、MLIR 等 GPU 加速与高性能计算开源项目的精选清单，方便按方向找底层算子、编译器和推理框架。

核心能力

  • 按官方库、学习资源、框架、应用、博客、视频、面试分类整理
  • 覆盖 CUDA、cuBLAS、cuDNN、CUTLASS、TensorRT、Triton、TVM、MLIR、PTX 等
  • 框架按语言细分 C++、Python、Rust、Julia、Mojo、Zig、Go 实现
  • 收录 LLM 推理引擎、高性能 kernel 库与多 GPU 分布式框架
  • 汇总 CUDA、TensorRT、Triton、TVM、MLIR、HPC 的学习资料与博客

容易踩的坑

这是资源清单仓库，不含可安装的代码或工具；条目多为外部链接，可用性需自行核实

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

0 个版本
全部 release

暂无 release 数据（可能没有发版，或尚未同步）。

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    Awesome-CUDA-and-HPC

    Awesome

    🚀🚀🚀 This repository lists some awesome public CUDA, cuda-python, cuBLAS, CUTLASS, cuDNN, NCCL, TensorRT, TensorRT-LLM, Triton, TileLang, TVM, MLIR, LLVM, PTX and High Performance Computing (HPC) projects.

    Contents

    Official Version

    • CUDA : CUDA is a parallel computing platform and programming model developed by NVIDIA for general computing on graphical processing units (GPUs).

    • NVIDIA/cuda-python : CUDA Python: Performance meets Productivity. nvidia.github.io/cuda-python/

    • cuBLAS : Basic Linear Algebra on NVIDIA GPUs. NVIDIA cuBLAS is a GPU-accelerated library for accelerating AI and HPC applications. It includes several API extensions for providing drop-in industry standard BLAS APIs and GEMM APIs with support for fusions that are highly optimized for NVIDIA GPUs. The cuBLAS library also contains extensions for batched operations, execution across multiple GPUs, and mixed- and low-precision execution with additional tuning for the best performance.

    • cuDNN : The NVIDIA CUDA Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, attention, matmul, pooling, and normalization.

    • CUTLASS : CUDA Templates for Linear Algebra Subroutines. CUTLASS is a collection of CUDA C++ template abstractions for implementing high-performance matrix-matrix multiplication (GEMM) and related computations at all levels and scales within CUDA. It incorporates strategies for hierarchical decomposition and data movement similar to those used to implement cuBLAS and cuDNN.

    • TensorRT : NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. developer.nvidia.com/tensorrt

    • TensorRT-LLM : TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines. nvidia.github.io/TensorRT-LLM

    • Triton : Development repository for the Triton language and compiler. This is the development repository of Triton, a language and compiler for writing highly efficient custom Deep-Learning primitives. The aim of Triton is to provide an open-source environment to write fast code at higher productivity than CUDA, but also with higher flexibility than other existing DSLs. triton-lang.org/

    • TileLang : Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels. Tile Language (tile-lang) is a concise domain-specific language designed to streamline the development of high-performance GPU/CPU kernels (e.g., GEMM, Dequant GEMM, FlashAttention, LinearAttention). By employing a Pythonic syntax with an underlying compiler infrastructure on top of TVM, tile-lang allows developers to focus on productivity without sacrificing the low-level optimizations necessary for state-of-the-art performance. tilelang.com/

    • TVM : Open deep learning compiler stack for cpu, gpu and specialized accelerators. tvm.apache.org/

    • MLIR : Multi-Level Intermediate Representation Compiler Framework. The MLIR project is a novel approach to building reusable and extensible compiler infrastructure. MLIR aims to address software fragmentation, improve compilation for heterogeneous hardware, significantly reduce the cost of building domain specific compilers, and aid in connecting existing compilers together.

    • LLVM : The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.

    • PTX : PTX, a low-level parallel thread execution virtual machine and instruction set architecture (ISA).

    Awesome List

    Learning Resources

    Frameworks

    • CUDA Frameworks

      • GPU Interface

        GPU接口
        • CPP Version

          • CCCL : CUDA C++ Core Libraries. The concept for the CUDA C++ Core Libraries (CCCL) grew organically out of the Thrust, CUB, and libcudacxx projects that were developed independently over the years with a similar goal: to provide high-quality, high-performance, and easy-to-use C++ abstractions for CUDA developers.

          • HIP : HIP: C++ Heterogeneous-Compute Interface for Portability. HIP is a C++ Runtime API and Kernel Language that allows developers to create portable applications for AMD and NVIDIA GPUs from single source code. rocmdocs.amd.com/projects/HIP/

        • Python Version
        • Rust Version
        • Julia Version

      • Performance Benchmark

        • NVIDIA/nvbench : CUDA Kernel Benchmarking Library.

        • FlagPerf : FlagPerf is an open-source software platform for benchmarking AI chips. FlagPerf是智源研究院联合AI硬件厂商共建的一体化AI硬件评测引擎，旨在建立以产业实践为导向的指标体系，评测AI硬件在软件栈组合（模型+框架+编译器）下的实际能力。

        • te42kyfo/gpu-benches : collection of benchmarks to measure basic GPU capabilities.

      • Scientific Computing Framework

        科学计算框架

        • cuBLAS : Basic Linear Algebra on NVIDIA GPUs. NVIDIA cuBLAS is a GPU-accelerated library for accelerating AI and HPC applications. It includes several API extensions for providing drop-in industry standard BLAS APIs and GEMM APIs with support for fusions that are highly optimized for NVIDIA GPUs. The cuBLAS library also contains extensions for batched operations, execution across multiple GPUs, and mixed- and low-precision execution with additional tuning for the best performance.

        • CUTLASS : CUDA Templates for Linear Algebra Subroutines.

        • MatX : MatX - GPU-Accelerated Numerical Computing in Modern C++. An efficient C++17 GPU numerical computing library with Python-like syntax. nvidia.github.io/MatX

        • DeepGEMM : DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling.

        • MUTLASS : MUSA Templates for Linear Algebra Subroutines.

        • CuPy : CuPy : NumPy & SciPy for GPU. cupy.dev

        • GenericLinearAlgebra.jl : Generic numerical linear algebra in Julia.

        • custos-math : This crate provides CUDA, OpenCL, CPU (and Stack) based matrix operations using custos.

      • Attention and Transformer Framework

        • FlashAttention : Fast and memory-efficient exact attention. "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness". (arXiv 2022).

        • SageAttention : Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models. "SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration". (arXiv 2024). "SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization". (arXiv 2024).

        • fla-org/flash-linear-attention : 🚀 Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton.

        • 66RING/tiny-flash-attention : flash attention tutorial written in python, triton, cuda, cutlass.

        • weishengying/tiny-flash-attention : 使用 cutlass 实现 flash-attention 精简版，具有教学意义。

        • jepeake/tiny-flash-attention : flash attention in ~20 lines.

      • Machine Learning Framework

        • cuDNN : The NVIDIA CUDA® Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, attention, matmul, pooling, and normalization.

        • PyTorch : Tensors and Dynamic neural networks in Python with strong GPU acceleration. pytorch.org

        • MooreThreads/torch_musa : torch_musa is an open source repository based on PyTorch, which can make full use of the super computing power of MooreThreads graphics cards.

        • PaddlePaddle : PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）. www.paddlepaddle.org/

        • flashlight/flashlight : A C++ standalone library for machine learning. fl.readthedocs.io/en/latest/

        • yhwang-hub/dl_model_infer : his is a c++ version of the AI reasoning library. Currently, it only supports the reasoning of the tensorrt model. The follow-up plan supports the c++ reasoning of frameworks such as Openvino, NCNN, and MNN. There are two versions for pre- and post-processing, c++ version and cuda version. It is recommended to use the cuda version., This repository provides accelerated deployment cases of deep learning CV popular models, and cuda c supports dynamic-batch image process, infer, decode, NMS.

        • NVlabs/tiny-cuda-nn : Lightning fast C++/CUDA neural network framework.

        • zjhellofss/KuiperLLama : 《动手自制大模型推理框架》。KuiperLLama 动手自制大模型推理框架，支持LLama2/3和Qwen2.5。校招、秋招、春招、实习好项目，带你从零动手实现支持LLama2/3和Qwen2.5的大模型推理框架。

        • zjhellofss/KuiperInfer : 校招、秋招、春招、实习好项目！带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step。

        • zjhellofss/kuiperdatawhale : 从零自制深度学习推理框架。

        • MarioSieg/magnetron : (WIP) A small but powerful, homemade PyTorch from scratch. Minimalistic homemade PyTorch alternative, written in C99 and Python.

        • lucasdelimanogueira/PyNorch : Recreating PyTorch from scratch (C/C++, CUDA, NCCL and Python, with multi-GPU support and automatic differentiation!)

      • AI Inference Framework

        AI推理框架
        • LLM Inference and Serving Engine

          • TensorRT : NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. developer.nvidia.com/tensorrt

          • TensorRT-LLM : TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines. nvidia.github.io/TensorRT-LLM

          • NVIDIA/TensorRT-Model-Optimizer : TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization, pruning, distillation, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs. nvidia.github.io/TensorRT-Model-Optimizer

          • Ollama : Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 2, and other large language models. ollama.com

          • vLLM : A high-throughput and memory-efficient inference and serving engine for LLMs. docs.vllm.ai

          • Nano-vLLM : A lightweight vLLM implementation built from scratch.

          • SGLang : SGLang is a fast serving framework for large language models and vision language models. docs.sglang.ai/

          • sgl-project/sgl-learning-materials : Materials for learning SGLang.

          • llama.cpp : LLM inference in C/C++.

          • FlashInfer : FlashInfer: Kernel Library for LLM Serving . flashinfer.ai

          • Chitu（赤兔） : High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability. Chitu (赤兔) 是一个专注于效率、灵活性和可用性的高性能大语言模型推理框架。

          • ztxz16/fastllm : fastllm是c++实现自有算子替代Pytorch的高性能全功能大模型推理库，可以推理Qwen, Llama, Phi等稠密模型，以及DeepSeek, Qwen-moe等moe模型。fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tps，多并发可达60+。

          • MLC LLM : Universal LLM Deployment Engine with ML Compilation. llm.mlc.ai/

          • KTransformers : A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations. kvcache-ai.github.io/ktransformers/

          • Aphrodite : Large-scale LLM inference engine. aphrodite.pygmalion.chat

          • GPUStack : GPUStack is an open-source GPU cluster manager for running AI models. Manage GPU clusters for running AI models. gpustack.ai

          • Lamini : The Official Python Client for Lamini's API. lamini.ai/

          • triton-inference-server/tensorrtllm_backend : The Triton TensorRT-LLM Backend.

          • datawhalechina/self-llm : 《开源大模型食用指南》基于Linux环境快速部署开源大模型，更适合中国宝宝的部署教程。

          • ninehills/llm-inference-benchmark : LLM Inference benchmar

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    netron
    lutzroeder·JavaScript

    把 ONNX、PyTorch、Keras、CoreML 等格式的神经网络模型直接拖进浏览器就能看结构图，排查模型转换和层连接问题很方便。

    模型可视化+4格式转换
    33676+123.2kv9.3.52026-11-09 12:35 UTC
    onnxruntime
    microsoft·C++

    跨平台高性能推理与训练加速引擎，可直接加载 ONNX 模型，兼容 PyTorch、TensorFlow 等框架导出，解决模型在不同硬件上高效部署的问题。

    推理引擎+4格式转换
    22275+154.4kv1.30.12026-11-10 17:16 UTC
    onnx
    onnx·Python

    ONNX 是开放的机器学习模型交换格式标准，让 PyTorch、TensorFlow、Keras 等框架训练出的模型能互相转换、跨平台部署。

    模型交换格式+4格式转换
    21669+64.1kv1.23.22026-11-10 15:40 UTC
    magika
    google·Rust

    用深度学习快速准确地识别文件真实类型，不依赖扩展名，支持 MIME 检测，Rust 实现，可嵌入各类应用做内容安全与文件分类。

    文件类型检测+4格式转换
    18787+111.2kcli/v1.1.02026-11-10 16:30 UTC