登录

bitsandbytes

bitsandbytes-foundationMIT活跃维护

让大模型在普通显卡上跑起来的 PyTorch 量化库，支持 8-bit 和 4-bit 权重压缩，也是 QLoRA 微调背后的核心依赖。

Accessible large language models via k-bit quantization for PyTorch.

0.50.2发布于 2026-08-27 00:13 UTC8,536star942fork同步于 2026-11-10 05:31 UTC
量化PyTorch大模型QLoRA低显存
对比 GitHub 官网llmmachine-learningpytorchqloraquantization

怎么上手

由 README 提炼，执行前请确认
原文 
pip install bitsandbytes
硬件要求按 README 说明

需 Python 3.10+ 和 PyTorch 2.4+。CPU 需 AVX2（推荐 AVX512F/AVX512BF16）；NVIDIA GPU 最低 SM60、推荐 SM75+；另支持 AMD（CDNA/RDNA 系列）、Intel GPU（Max/Arc A/B 系列）、Intel Gaudi2/3，以及 macOS 14+ 的 Apple M1+（CPU 与 MPS）。README 未给出具体显存数值。

支持 Apple Silicon
下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼
没人工点评？写下你的经验

PyTorch 的 k-bit 量化库，用 8-bit 优化器、LLM.int8() 和 QLoRA 大幅降低大模型显存占用

让大模型在普通显卡上跑起来的 PyTorch 量化库，支持 8-bit 和 4-bit 权重压缩，也是 QLoRA 微调背后的核心依赖。

核心能力

  • 用分块量化实现 8-bit 优化器，显存远低于 32-bit 且保持性能
  • LLM.int8() 用向量级量化加 16-bit 离群值处理，推理显存减半
  • QLoRA 将模型量化到 4-bit 并插入 LoRA 权重以低显存训练
  • 提供 Linear8bitLt、Linear4bit 量化层与 bitsandbytes.optim 优化器模块

容易踩的坑

PyTorch 需 2.4+ 且推荐用最新版；Linux 需 glibc >= 2.24；macOS 上 8-bit 优化器在 MPS 后端仍为计划中，部分标记功能性能优化不足

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README
这个条目还没有人工点评。如果它对你很有用，欢迎到 提交仓库 页写下你的使用经验。

用户点评

登录用户的真实使用记录，可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    最近发版

    5 个版本0.50.2
    全部 release
    1. continuous-release_mainLatest `main` wheelpre-release2026-10-27 · 2026-10-27 21:28 UTC
      查看更新说明

      Latest main pre-release wheel

      This pre-release contains the latest development wheels for all supported platforms, rebuilt automatically on every commit to the main branch.

      How to install: Pick the correct command for your platform and run it in your terminal:

      macOS 14+ (arm64)

      pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-macosx_14_0_arm64.whl

      Linux (aarch64)

      pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-manylinux_2_24_aarch64.whl

      Linux (x86_64)

      pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-manylinux_2_24_x86_64.whl

      Windows (x86_64)

      pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-win_amd64.whl

      Windows (arm64)

      pip install --force-reinstall https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-win_arm64.whl

      Custom PyTorch builds (Intel XPU, ROCm, etc.): The --force-reinstall flag causes pip to re-resolve all dependencies from PyPI, which may replace your custom PyTorch build with the default CUDA variant. To avoid this, add --no-deps:

      pip install --force-reinstall --no-deps https://github.com/bitsandbytes-foundation/bitsandbytes/releases/download/continuous-release_main/bitsandbytes-1.33.7.preview-py3-none-manylinux_2_24_x86_64.whl

      Note: These wheels are updated automatically with every commit to main and become available as soon as the python-package.yml workflow finishes.

      The version number is replaced with 1.33.7-preview in order to keep the link stable, this however does not affect the installed version at all:

      > pip install https://.../bitsandbytes-1.33.7-preview-py3-none-manylinux_2_24_x86_64.whl
Collecting bitsandbytes==1.33.7rc0
...
Successfully installed bitsandbytes-0.49.0.dev0
    2. 0.50.2最新正式版2026-08-27 · 2026-08-27 00:13 UTC

      Highlights

      • ROCm 10.0 build is now included for both Linux and Windows.
      • Additional AMD GPU targets are included in our prebuilt wheels.
      • Performance improvement for certain 4bit GEMM shapes on NVIDIA GB10.

      What's Changed

      New Contributors

      Full Changelog: https://github.com/bitsandbytes-foundation/bitsandbytes/compare/0.50.1...0.50.2

    3. 0.50.10.50.1: RTX Spark Support2026-08-13 · 2026-08-13 17:19 UTC
      查看更新说明

      This release adds support for additional hardware platforms and improves performance on RTX Spark / DGX Spark.

      Highlights

      • Support added for NVIDIA RTX Spark products running Windows on ARM64 (#2038, #2037, #2032)
      • Improved 4bit GEMM dispatch heuristics for NVIDIA GB10 (#2030)
      • Support for AMD CDNA5 hardware (e.g. MI455X)

      What's Changed

      New Contributors

      Full Changelog: https://github.com/bitsandbytes-foundation/bitsandbytes/compare/0.50.0...0.50.1

    4. 0.50.00.50.0: Faster 4-bit inference, CPU/ROCm/Apple Silicon improvements, and more!2026-07-25 · 2026-07-25 01:42 UTC
      查看更新说明

      This release brings a new fused 4-bit GEMM for inference on CUDA and ROCm, faster CPU ops on x86-64 and ARM64, reduced host-side overhead, and a much improved Apple Silicon backend. We've also added Windows on ARM CPU support, ROCm builds for Windows, additional ROCm and CUDA build variants, and new optimizer support on CPU and Intel XPU.

      Highlights

      🚀 New 4-bit GEMM kernels for inference (CUDA)

      New fused 4-bit dequantize + GEMM kernels replace the old GEMV and dequantize + F.linear paths for small-to-medium batch sizes. 4-bit inference is up to 4x faster at batch sizes of 2 through 64 across Turing through Blackwell, with wins at batch size 1 in many cases too. Nested (double) quantization and bias are fused in as well, so nested quant sees an additional benefit. Kernel selection happens automatically at runtime based on shape, GPU architecture, and SM count. See #1949 for benchmarks and details.

      🚀 Fused 4-bit SIMT GEMM on ROCm

      The SIMT version of the new 4-bit GEMM has been ported to ROCm and wired into the same dispatch for small inference batches, validated on gfx1100, gfx1201, and gfx1151 (#1979).

      AMD ROCm improvements

      Stability and performance improvements bring AMD ROCm support out of preview; it is now considered stable.

      • ROCm wheels for Windows are now published for ROCm 7.2 and 7.14 (#1915, #2007).
      • Added ROCm 7.14 and dropped the ROCm 6.2 and 6.3 builds (#2007).
      • Expanded GPU target coverage to add RDNA2 consumer GPUs, additional RDNA3 and RDNA3.5 GPUs, and CDNA1 (gfx908) data center GPUs (#1886, #2007).

      Apple Silicon (MPS)

      Apple Silicon support is improved. The MPS backend added optimized Metal kernels from the Hub (#1875) and was substantially improved (#1960, #1983, #1994) so that all 4-bit and LLM.int8() configurations now work on MPS. On macOS 26+, install the kernels package for the best performance, which enables the optimized Metal kernels; otherwise a naive fallback is used. The MPS path requires torch >= 2.9.

      The last remaining feature for support parity is the 8bit optimizers, which will land in a future release.

      CPU performance on x86-64 and ARM64

      Blockwise quantization and dequantization on CPU are considerably faster, mostly from better SIMD usage plus some compile flag tuning. Improvements range from 1.1x to over 20x depending on op, dtype, and hardware, with the largest gains on fp16 and on x86-64 CPUs without AVX-512. The LLM.int8() matmul on CPU was improved as well. See #1968 for benchmarks.

      Reduced host-side CPU overhead

      Reduced Python dispatch overhead, especially on the CUDA/ROCm backend (#1953).

      Windows on ARM CPU support

      Windows ARM64 CPU wheels are now built with NEON-optimized kernels (#1959), with nightly test coverage added (#1962).

      New optimizer support

      • CPU optimizers are now supported for both 32-bit and 8-bit blockwise (#1901).
      • Intel XPU gained paged optimizer support (#1898) and support for the LARS and LAMB optimizers (#1902).
      • Fixed Lion to use decoupled weight decay (#1993, #2001).
      • Other optimizer fixes (#1869, #1998, #1855).

      NVIDIA CUDA

      • Added CUDA 13.2 build wheels and nightly test coverage (#1925).
      • Smaller wheels: reduced the CUDA build matrix and added a closest-version fallback for library loading (#1980).

      Intel XPU

      • Added a oneAPI 2026 build; wheels now package both the 2025 and 2026 variants and pick the right one at runtime based on the PyTorch XPU version (#2002).
      • Fixed gemv_4bit bf16 correctness on Intel Arc A-series (Alchemist) GPUs (#1942).

      🚨 Breaking Changes

      • Minimum PyTorch version is now 2.4 (#1926).
      • Removed deprecated APIs: the research module, non-blockwise (block_wise=False) optimizers, and legacy dynamic quantization functions, along with their CUDA/HIP kernels (#1871, #1880).
      • Removed legacy sparse functionality (spmm_coo, spmm_coo_very_sparse) and dropped the cusparse/hipsparse dependencies (#1881).

      Deprecations

    5. 0.49.22026-02-16 · 2026-02-16 21:29 UTC
      查看更新说明

      Highlights

      • The default blocksize of 64 for 4bit quantization is now supported on ROCm. Previously the default was 128, which was a mismatch from the default for other devices.
      • ROCm 7.2 build is now included.

      What's Changed

      New Contributors

      Full Changelog: https://github.com/bitsandbytes-foundation/bitsandbytes/compare/0.49.1...0.49.2

    README 原文

    未翻译
    在 GitHub 打开

    bitsandbytes enables accessible large language models via k-bit quantization for PyTorch. We provide three main features for dramatically reducing memory consumption for inference and training:

    • 8-bit optimizers uses block-wise quantization to maintain 32-bit performance at a small fraction of the memory cost.
    • LLM.int8() or 8-bit quantization enables large language model inference with only half the required memory and without any performance degradation. This method is based on vector-wise quantization to quantize most features to 8-bits and separately treating outliers with 16-bit matrix multiplication.
    • QLoRA or 4-bit quantization enables large language model training with several memory-saving techniques that don't compromise performance. This method quantizes a model to 4-bits and inserts a small set of trainable low-rank adaptation (LoRA) weights to allow training.

    The library includes quantization primitives for 8-bit & 4-bit operations, through bitsandbytes.nn.Linear8bitLt and bitsandbytes.nn.Linear4bit and 8-bit optimizers through bitsandbytes.optim module.

    System Requirements

    bitsandbytes has the following minimum requirements for all platforms:

    • Python 3.10+
    • PyTorch 2.4+
      • Note: While we aim to provide wide backwards compatibility, we recommend using the latest version of PyTorch for the best experience.
    Accelerator support:

    Note: this table reflects the status of the current development branch. For the latest stable release, see the document in the 0.50.0 tag.

    Legend:

    🚧 = Planned | 〰️ = Partially Supported | ✅ = Supported | ❌ = Not Supported

    :book: Documentation

    :heart: Sponsors

    The continued maintenance and development of bitsandbytes is made possible thanks to the generous support of our sponsors. Their contributions help ensure that we can keep improving the project and delivering valuable updates to the community.

    License

    bitsandbytes is MIT licensed.

    How to cite us

    If you found this library useful, please consider citing our work:

    QLoRA

    @article{dettmers2023qlora,
  title={Qlora: Efficient finetuning of quantized llms},
  author={Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},
  journal={arXiv preprint arXiv:2305.14314},
  year={2023}
}

    LLM.int8()

    @article{dettmers2022llmint8,
  title={LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale},
  author={Dettmers, Tim and Lewis, Mike and Belkada, Younes and Zettlemoyer, Luke},
  journal={arXiv preprint arXiv:2208.07339},
  year={2022}
}

    8-bit Optimizers

    @article{dettmers2022optimizers,
  title={8-bit Optimizers via Block-wise Quantization},
  author={Dettmers, Tim and Lewis, Mike and Shleifer, Sam and Zettlemoyer, Luke},
  journal={9th International Conference on Learning Representations, ICLR},
  year={2022}
}

    相关项目

    同分类里 star 最高的项目
    对比全部 →
    headroom
    headroomlabs-ai·Python

    在工具输出、日志、文件和 RAG 片段进入 LLM 前先做压缩，编码 Agent 省约 20% token，JSON 省 60-95%，答案不变。提供库、代理和 MCP 服务器三种用法。

    上下文压缩+4量化
    79264+2196.2kv0.43.12026-11-10 05:09 UTC
    llmfit
    AlexsJones·Rust

    一条命令就能从数百个模型和供应商里筛出适合你硬件配置的本地大模型，省去逐个试跑和查显存的麻烦。

    本地推理+4量化
    38868+1412.5kv1.1.202026-11-09 21:31 UTC
    PaddleNLP
    PaddlePaddle·Python

    基于飞桨的 NLP 与大模型开发库，提供丰富模型库，覆盖 BERT、ERNIE、LLaMA 等，支持分布式训练、压缩和文档智能等任务。

    大模型开发库+4量化
    12987+13.0krl-v1.0.02026-05-23 02:54 UTC
    shimmy
    Michael-A-Kuykendall·Rust

    纯 Rust 实现的 WebGPU 推理引擎，原生支持 GGUF 模型，兼容 OpenAI API，无需 Python 和 llama.cpp，单个二进制文件即可在任意 GPU 上跑本地大模型。

    推理引擎+4量化
    6015+3587v2.6.62026-11-09 18:48 UTC