GPTQ 是 ICLR 2023 论文的官方实现，能在训练后把 GPT 类大模型量化到 3/4 比特，几乎不掉精度，大幅降低显存占用和推理成本。
面向大模型量化的压缩工具包，支持 NVIDIA、AMD、Intel、华为昇腾等多硬件加速，可无缝接入 HF、vLLM 和 SGLang 推理。
面向大语言模型的高吞吐、低显存推理与服务引擎，通过 PagedAttention 等优化显著提升并发吞吐并降低显存占用，适合部署在线推理服务。
面向 PyTorch、TensorFlow 和 ONNX Runtime 的模型压缩工具，支持 INT8/FP8/INT4 等低比特量化和稀疏化，帮大模型在低精度下保持精度并降低显存与推理成本。
…-GPU runs actually use it. - Checkpointing: Added gptqmodel.reshard() for per-layer safetensors sharding with one global shard, byte-for-byte tensor copies, preserved met
…0) launch tables to every tuned Triton pipeline — GPTQ GEMV/GEMM/dequant, ParoQuant GEMM, AWQ GEMM/GEMV, GGUF Triton, and bitsandbytes 4-bit — enforced by a hardware-free
…(previously ~9s and 6767 modules). Vectorized the GPTQ/QQQ MSE shrink-grid and AWQ clip-grid evaluation; grid workspace budgets moved from environment variables to a new
…VX512-BF16 int4 qzero group scaled cpu kernel for GPTQ/AWQ group sizes 16/32/64/128/256/512 and prioritized it ahead of ATen when supported. Added packed/low-precision CP
…y @ZX-ModelCloud in https://github.com/ModelCloud/GPTQModel/pull/3085 Fix Qwen3.5-MoE multimodal calibration routing by @ZX-ModelCloud in https://github.com/ModelCloud/GP
…huggingface/optimum/pull/2384 Fully deprecate AutoGPTQ for GPT-QModel by @ZX-ModelCloud in https://github.com/huggingface/optimum/pull/2385 Transformers v5 pre release su