登录

tt-metal

tenstorrentApache-2.0活跃维护

面向 Tenstorrent 加速卡的 TT-NN 算子库和 TT-Metalium 底层内核编程模型，让大模型和图像生成模型能在非 GPU 硬件上高效推理。

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

v0.83.0-dev20261108发布于 2026-11-08 01:47 UTC1,747star833fork同步于 2026-11-08 23:43 UTC
算子库内核编程加速卡大模型推理图像生成
对比 GitHub 官网acceleratoraicudadeepseekgpu

怎么上手

由 README 提炼，执行前请确认
原文 
README 未给出安装命令，仅指向 INSTALLING.md 安装文档
硬件要求按 README 说明

面向 Tenstorrent 自研 AI 加速卡，README 中出现的型号包括 Wormhole（n150、n300、QuietBox、Galaxy）与 Blackhole（p150），Blackhole 软件优化仍在开发中；README 未说明显存容量与 CPU 运行方式。

多卡 / 张量并行
下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

Tenstorrent 芯片上的神经网络算子库与底层内核编程模型，用于在 Wormhole/Blackhole 硬件上跑模型

面向 Tenstorrent 加速卡的 TT-NN 算子库和 TT-Metalium 底层内核编程模型，让大模型和图像生成模型能在非 GPU 硬件上高效推理。

核心能力

  • 提供 Python 与 C++ 的 TT-NN 神经网络算子库
  • 用 TT-Metalium 编写底层 kernel 直接驱动硬件
  • 内置 Llama、Qwen、Mixtral、Whisper 等模型 demo
  • 支持多设备张量并行与 Scale-Out 多 mesh 编程
  • 配套 TT-NN Visualizer、TT-SMI 等调试与监控工具

容易踩的坑

安装步骤不在 README 正文，需另看 INSTALLING.md；模型 demo 依赖对应版本的 TT-Metalium release，版本不匹配可能跑不通；vLLM 部署需用 tenstorrent/vllm-tt-plugin，旧的 vLLM TT fork 已废弃

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本v0.83.0-dev20261108
全部 release
  1. v0.83.0-dev20261108pre-release最新正式版2026-11-08 · 2026-11-08 01:47 UTC

    [!NOTE]

    If you are installing from a release, please refer to the README, INSTALLATION instructions, and any other documentation packaged with the release, not on the main branch. There may be differences between the latest main and the previous release.

    The changelog will now follow, showing the changes from last release.

    This release was generated by the CI workflow https://github.com/tenstorrent/tt-metal/actions/runs/42429824651

    LLK (low-level kernels)

    • support all tile shapes in unpack tilize PR 63787
    • feat: release the emulator when a gtest is interrupted or the dispatch context is dead PR 64002
    • WH SFPU elu/selu/celu: expm1_cw constants in Prgm regs PR 62782
    • WH SFPU comp eqz/nez/ltz/gtz/gez/lez: predicates on raw Dest bits PR 62852
    • WH SFPU power_iterative: walk exponent once per row pair PR 62801

    Metalium (tt-metal core)

    • Keep the Tensor prefetcher signal table out of Wormhole firmware PR 64397
    • MeshBuffer: return built device buffers without the lazy-build lock PR 64583
    • Watcher-checked multicast into DFBs and semaphores PR 63667
    • Add SRAMArena to support custom ephemeral allocation PR 60652
    • Split custom-Alignment APIs & distributed utilities out of experimental/tensor PR 51656
    • feat: release the emulator when a gtest is interrupted or the dispatch context is dead PR 64002

    TT-NN

    • ttnn.sort: stop the go-signal multicast from writing outside the program PR 64705
    • Fix DiT norm, qkv decode and RoPE defects (#57686) PR 59273
    • Read ttnn buffer properties without going through Tensor::buffer() PR 63855
    • Kimi-K3 recurrent scan, affine prefix and output projection at HiFi3 PR 62897
    • dit_fused_distributed_rmsnorm: row_capacity (runtime rows on a fixed worker split) PR 62184
    • nlp_create_qkv_heads, nlp_concat_heads, all_gather_async: opt-in runtime row count on the interleaved paths PR 62186
    • Split custom-Alignment APIs & distributed utilities out of experimental/tensor PR 51656
    • Match docstring argument names to the nanobind bindings (part 2) PR 63687
    • Run attention_softmax_ and sharded rms_norm cases with watcher again (closed #37098, #37259) PR 63670
    • gather: reject a preallocated out= whose layout differs from the input PR 63664
    • moreh_layer_norm, moreh_group_norm: reject gamma and beta of the wrong shape PR 63658
    • moe: reject inputs that are not BFLOAT16 (FLOAT32 hangs the device) PR 63657
    • test_linear: drop the stale #16599 xfail on the Mx1xN vector case PR 63616
    • embedding + untilize: opt-in runtime rows for the tilized fused embedding and the multi-core interleaved untilize PR 62187

    tt-train

  2. v0.83.0-dev20261107pre-release2026-11-07 · 2026-11-07 02:58 UTC
    查看更新说明

    [!NOTE]

    If you are installing from a release, please refer to the README, INSTALLATION instructions, and any other documentation packaged with the release, not on the main branch. There may be differences between the latest main and the previous release.

    The changelog will now follow, showing the changes from last release.

    This release was generated by the CI workflow https://github.com/tenstorrent/tt-metal/actions/runs/42317870829

    LLK (low-level kernels)

    • Use each Quasar variant's real L1 size PR 64015
    • Unpack SrcB from its own face shape in unpack_AB and reduce PR 63552
    • Extend Quasar tt-llk tests for Dest depth, matmul K, reduce and formats PR 63906
    • Update TT/TTI macros anticipating mod/dst/src metadata PR 63845
    • WH heaviside: restore the SFPLZ zero test PR 64328
    • ci: raise Quasar llk test parallelization from 32 to 40 tests in parallel PR 64470
    • ci: update exalens version to 0.4.3 PR 64441
    • Remove L1_CONGESTION from EVERYWHERE PR 60571
    • llk wormhole: keep the init paths straight PR 63819
    • Typecast SFPLOADMACRO templates use the documented SFPSWAP max mode PR 64426
    • perf harness: drive the wormhole isolate stand-ins through the MOP expander PR 63820
    • Balance the smoke and perf legs by test durations from CI artifacts PR 64461
    • Audit a Tensix config write that a later RISC store can overtake PR 64282

    Metalium (tt-metal core)

    • Prune links leaving the solved subset in topology mapping PR 64446
    • Use each Quasar variant's real L1 size PR 64015
    • Unpack SrcB from its own face shape in unpack_AB and reduce PR 63552
    • HAL codegen: emit generated header includes in sorted order PR 64520
    • Fix order-dependent DRAM-to-L1 multicast tests and a fmt/ranges.h unity conflict PR 64511
    • genfiles: emit KERNEL_BOUND_DFB_MASK only for Quasar PR 64606
    • Quasar: fence after noc_semaphore_set so a following multicast sends the new value PR 64642
    • Quasar watcher: MPSC ring buffer on dispatch engines PR 64330
    • Fix ColMajorShardingAndReplication shard size on 8-row meshes PR 64024
    • PAUSE in the idle waits of BRISC, NCRISC and the TRISCs PR 63886
    • Deprecate MeshBuffer::get_backing_buffer() PR 64032
    • Migrate tests off deprecated MeshDevice::get_optimal_dram_bank_to_logical_worker_assignment(noc) PR 63447
    • Add TTNN command list recording, build, and replay PR 62999
    • SFPI 7.95.0 1101 PR 64313
    • Host transfer stats: report host <-> device transfer time and volume at teardown PR 63267
    • Quasar DFB: finish DFBs in firmware at kernel
  3. v0.83.0-dev20261106pre-release2026-11-06 · 2026-11-06 13:50 UTC
    查看更新说明

    [!NOTE]

    If you are installing from a release, please refer to the README, INSTALLATION instructions, and any other documentation packaged with the release, not on the main branch. There may be differences between the latest main and the previous release.

    The changelog will now follow, showing the changes from last release.

    This release was generated by the CI workflow https://github.com/tenstorrent/tt-metal/actions/runs/42136711855

    LLK (low-level kernels)

    • SFPU lgamma: keep the inf fixup in bf16 Dest mode and let the pole win over the overflowed Stirling term PR 60983
    • perf report: keep the rows of every chunk a worker runs PR 63816
    • llk blackhole: no fast tile end drain while the packer accumulates in L1 PR 60894
    • #51866: bf16 softplus keeps the negative tail PR 58101
    • Add Quasar matmul test sweeping direct indexing (EN_DI) PR 59885
    • custom_mm: support N = 16 (half-width in1 / output tiles) on Blackhole PR 63514
    • profiler: write both zone records after the closing clock read PR 63817
    • perf harness: end each launch on the TRISC PC buffers PR 63818

    Metalium (tt-metal core)

    • llk blackhole: no fast tile end drain while the packer accumulates in L1 PR 60894
    • #51866: bf16 softplus keeps the negative tail PR 58101
    • Let a PrefetcherPipe kernel also run on nodes outside the pipe PR 60363
    • Automated UMD Bump 04.11.2026 PR 64271
    • custom_mm: support N = 16 (half-width in1 / output tiles) on Blackhole PR 63514
    • Bump ttsim version to v1.16.0 PR 64373
    • Add relax_tensor_alignment to Metal 2.0 TensorSpecRelaxations PR 64290

    TT-NN

    • SFPU lgamma: keep the inf fixup in bf16 Dest mode and let the pole win over the overflowed Stirling term PR 60983
    • Reject out-of-range update cache positions PR 64072
    • ttnn.where: run an inferred sharded output on its own sub-device (#64037) PR 64221
    • eltwise: squared_difference golden keys off the tensor operand; comparison-mode test grades scalar-first div (#57213, #57212) PR 63701
    • #51866: bf16 softplus keeps the negative tail PR 58101
    • KDA: fused output projection + TP reduce-scatter, op staging in circular buffers PR 64213
    • Cut test_matmul_2d_config_bitexact down to one small shape: it timed out sim CI PR 64307
    • Rebalance reduction coverage in the ttnn matmul fused reduction ops merge-gate job PR 60780
    • Check every roll and tensor round-trip result PR 64210
    • Fix reshape docstring PR 64266
    • Let a PrefetcherPipe kernel also run on nodes outside the pipe PR 60363
    • ring_joint_sdpa: pass tensor addresses as common runtime args PR 63466
    • mo
  4. v0.83.0-dev20261105pre-release2026-11-05 · 2026-11-05 04:17 UTC
    查看更新说明

    [!NOTE]

    If you are installing from a release, please refer to the README, INSTALLATION instructions, and any other documentation packaged with the release, not on the main branch. There may be differences between the latest main and the previous release.

    The changelog will now follow, showing the changes from last release.

    This release was generated by the CI workflow https://github.com/tenstorrent/tt-metal/actions/runs/41941124257

    LLK (low-level kernels)

    • top32_rm: size the L1 chunk offset by the source format, not the Dest format PR 61113
    • Bracket every LLK replay-buffer record for instruction-gathering builds PR 63715
    • Unify sfpu binop scalar functional and perf PR 63745
    • test(llk): pack Int32 typecast stimuli as two's complement PR 58952
    • #62165: Unify eltwise bcast col custom tests PR 63743
    • Share sfpu ternary kernel PR 63771
    • Unify sfpu ema functional and perf PR 63767
    • Fix stale tt-llk docs and agent configuration PR 63485
    • WH SFPU shift: branch-free arithmetic right shift PR 62749

    Metalium (tt-metal core)

    • Quasar TRISC: retry a skipped mret in the HW error handler PR 63403
    • Fabric: round ethernet packet sends up to whole 16B words PR 61546
    • Write preprocess-and-ship .ii and .d files atomically PR 63873
    • Bracket every LLK replay-buffer record for instruction-gathering builds PR 63715
    • tracy: report correct core in FW boundary errors PR 64057
    • Fabric: show mesh and chip IDs in ControlPlane warnings PR 64053
    • Fabric: Resolve the program-scoped fabric connection helpers through a MeshDevice PR 61509
    • PrefetcherPipe: allocate Tensor prefetcher pipe L1 from the L1 allocator PR 63417
    • Remove MetalContext dependency from Reader/Writer DataMovementConfig PR 63901
    • Split buffer_types.hpp into host-only and device-only headers PR 62849
    • Remove unused includes for tt_metal and its tests PR 62821
    • Release Tensor prefetcher waits from device ops, re-armed on every trace replay PR 60169
    • noc_async_read_barrier_with_trid can return before the data lands (stale reads on Blackhole): fix and reproducer PR 59623
    • SFPI 7.94.0 1099 PR 63971
    • Multicast into TensorAccessor destinations, Watcher-checked for sharded tensors PR 63433
    • fabric docs: fix broken links in the MGD and PGD READMEs PR 64059
    • Take ttsim arch from UMD instead of a soc_descriptor.yaml PR 62711
    • Fix source links in the programming example walkthroughs PR 64061

    TT-NN

  5. v0.82.0-rc2pre-release2026-11-05 · 2026-11-05 04:00 UTC
    查看更新说明

    [!NOTE]

    If you are installing from a release, please refer to the README, INSTALLATION instructions, and any other documentation packaged with the release, not on the main branch. There may be differences between the latest main and the previous release.

    The changelog will now follow, showing the changes from last release.

    This release was generated by the CI workflow https://github.com/tenstorrent/tt-metal/actions/runs/41941333525

    Models

    • Gemma 4 release to stable: MTP speculative decoding (#62806) and the sliding SDPA perf leg (#63934) PR 64268

    Infrastructure & CI

    • Gemma 4 release to stable: MTP speculative decoding (#62806) and the sliding SDPA perf leg (#63934) PR 64268

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译
    在 GitHub 打开

    tt-metal CI Ask DeepWiki

    Hardware | Install | Discord | Join Us | Bounty $

    TT-NN is a Python & C++ Neural Network OP library.

    API Reference | Model Demos

    Featured Models

    The Models team is focused on developing the following models, optimizing them for performance, accuracy, and compatibility. Follow each model link for more details.

    [!IMPORTANT] For a full model list see the Model Matrix, or visit the Developer Hub.

    [!NOTE] Performance Metrics:

    • Time to First Token (TTFT) measures the time (in milliseconds) it takes to generate the first output token after input is received.
    • T/S/U (Tokens per Second per User): Represents the throughput of first-token generation after prefill. It is calculated as 1 / inter-token latency.
    • T/S (Tokens per Second): Represents total token throughput, calculated as T/S = T/S/U x batch size.
    • TP (Tensor Parallel) and DP (Data Parallel): Indicate the parallelization factors across multiple devices.
    • Reported LLM Performance: Based on an input sequence length of 128 tokens for all models.
    • Performance Data Source: Metrics were collected using the tt-metal model demos (linked above). Results may vary when using other runtimes such as the vLLM inference server.

    Llama 3.3 70B (TP=32)

    BatchHardwareTTFT (MS)T/S/UTargetT/S/UT/STT-Metalium ReleasevLLM Tenstorrent Repo Release
    32Galaxy (Wormhole)5372.5802268.8v0.65.0-rc759be953

    Qwen 2.5 7B (TP=2)

    BatchHardwareTTFT (MS)T/S/UTargetT/S/UT/STT-Metalium ReleasevLLM Tenstorrent Repo Release
    32n300 (Wormhole)10922.130707.2v0.62.0-rc35ced0161

    Qwen 2.5 72B (TP=8)

    BatchHardwareTTFT (MS)T/S/UTargetT/S/UT/STT-Metalium ReleasevLLM Tenstorrent Repo Release
    32QuietBox (Wormhole)22315.420492.8v0.62.0-rc25e7c329b

    Whisper (distil-large-v3)

    BatchHardwareTTFT (MS)T/S/UTargetT/S/UT/STT-Metalium Release
    1n150 (Wormhole)163105.045105.0v0.65.0-dev20251208
    1p150 (Blackhole)63263.4263.4v0.65.0-dev20251208

    Mixtral 8x7B (TP=8)

    BatchHardwareTTFT (MS)T/S/UTargetT/S/UT/STT-Metalium Release
    32QuietBox (Wormhole)12224.933796.8v0.62.0-dev20251015

    Blackhole software optimization is under active development. Please join us in shaping the future of open source AI! [Discord] [Developer Hub]

    For more information regarding vLLM installation and environment creation visit the Tenstorrent vLLM TT plugin README. Note that the previously maintained vLLM TT fork is deprecated.

    Model Updates

    For the latest model updates and features, please see MODEL_UPDATES.md

    Model Bring-Up and Testing

    For information on initial model procedures, please see Model Bring-Up and Testing

    TT-NN Tech Reports

    Benchmarks

    TT-Metalium is our low-level programming model, enabling kernel development for Tenstorrent hardware.

    Programming Guide | API Reference

    Getting started

    Get started with simple kernels.

    TT-Metalium Tech Reports

    Scaleout Tech Reports

    TT-Metalium Programming Examples

    Hello World

    Add Integers

    Simple Tensor Manipulation

    DRAM Data Movement

    Eltwise

    Matmul

    Tools and Instruments

    TT-NN Visualizer

    A comprehensive tool for visualizing and analyzing model execution, offering interactive graphs, memory plots, tensor details, buffer overviews, operation flow graphs, and multi-instance support with file or SSH-based report loading.

    TT-Exalens

    The TT-Exalens repository describes TT-Lensium, a low-level debugging tool for Tenstorrent hardware. It allows developers to access and communicate with Wormhole and Blackhole devices.

    TT-SMI

    The TT-SMI repository describes the Tenstorrent System Management Interface. This command line utility can interact with Tenstorrent devices on host. TT-SMI provides an easy to use interface displaying device, telemetry, and firmware information.

    Model Explorer

    The Model Explorer is an intuitive and hierarchical visualization tool using model graphs. It organizes model operations into nested layers and provides features for model exploration and debugging.

    Tracy Profiler

    The Tracy Profiler is a real-time nanosecond resolution, remote telemetry, hybrid frame, and sampling tool. Tracy supports profiling CPU, GPU, memory allocation, locks, context switches, and more.

    Device Print Debug

    DPRINT can print variables, addresses, and circular buffer data from kernels to the host terminal or log file. This feature is useful for debugging issues with kernels.

    Watcher

    Watcher monitors firmware and kernels for common programming errors, and overall device status. If an error or hang occurs, Watcher displays log data of that occurrence.

    Inspector

    Inspector provides insights into host runtime. It logs necessary data for investigation and allows queries to host runtime data.

    Related Tenstorrent Projects

    Latest Releases

    ReleaseRelease DateFW VersionKMD VersionSMI Version
    0.68.0ETA Apr 30, 202619.2.02.5.03.0.38
    0.67.4Mar 30, 202619.2.02.5.03.0.38
    0.67.0Mar 25, 202619.2.02.5.03.0.38
    0.66.0Feb 18, 202619.2.02.5.03.0.38
    0.65.1Jan 12, 202619.2.02.5.03.0.38
    0.65.0Dec 15, 202519.2.02.5.03.0.38
    0.64.5Dec 1, 202518.12.02.4.13.0.32
    0.64.4Nov 24, 202518.12.02.4.13.0.32
    0.64.3Nov 14, 202518.12.02.4.13.0.32
    0.64.0Oct 29, 202518.12.02.4.13.0.32
    0.63.0Sep 22, 202518.8.02.3.03.0.28
    0.62.2Aug 20, 202518.6.02.0.03.0.20
    0.61.0Skipped---
    0.60.1Jul 22, 202518.6.02.0.03.0.20
    0.59.0Jun 18, 2025---
    0.58.0May 13, 2025---
    0.57.0Apr 15, 2025---
    0.56.0Mar 7, 2025---

    Visit the releases folder for details on releases, release notes, and estimated release dates.

    Tenstorrent Bounty Program Terms and Conditions

    This repo is a part of Tenstorrent’s bounty program. If you are interested in helping to improve tt-metal, please make sure to read the Tenstorrent Bounty Program Terms and Conditions before heading to the issues tab. Look for the issues that are tagged with both “bounty” and difficulty level!

    License

    TT-Metalium and TTNN are licensed under the Apache 2.0 License, as detailed in LICENSE and LICENSE_understanding.txt.

    Some distributable forms of this project—such as manylinux-compliant wheels—may need to bundle additional libraries beyond the standard Linux system libraries. For example:

    • libnuma
    • libhwloc
    • openmpi (when built with multihost support)
    • libevent (when built with multihost support)

    These libraries are bound by their own license terms.

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    mlx
    ml-explore·C++

    面向 Apple silicon 的数组计算框架，让机器学习模型能直接在 Mac 芯片上高效训练和推理，省去折腾 CUDA 的麻烦。

    Apple silicon+4硬件加速适配进阶
    29130+432.4kv0.32.42026-11-06 23:36 UTC
    burn
    tracel-ai·Rust

    Rust 写的张量库和深度学习框架，兼顾灵活性与效率，支持自动微分、跨平台后端和内核融合，可训练和部署神经网络。

    深度学习框架+4硬件加速适配硬核
    16266+161.1kv0.22.12026-11-11 00:16 UTC
    rio
    raphamorim·Rust

    用 Rust 写的 GPU 加速终端模拟器，基于 Metal/Vulkan 渲染，桌面和浏览器里都能跑，适合追求流畅终端体验的开发者。

    终端模拟器+4硬件加速适配进阶
    7726+16382v0.5.292026-11-08 15:57 UTC
    mlx-lm
    ml-explore·Python

    在 Apple Silicon 上通过 MLX 框架运行大语言模型，支持加载、推理和微调，让 Mac 本地跑 LLM 更省心。

    MLX+4硬件加速适配进阶
    7618+311.2kv0.32.22026-11-06 11:23 UTC