Awesome-Model-Quantization

Awesome Model Quantization

Awesome Model Quantization is a curated, continuously updated collection of papers, benchmarks, surveys, and open-source implementations on neural network and model quantization. It spans binary and ternary networks, post-training quantization, quantization-aware training, vector and lattice quantization, low-bit LLMs, multimodal and generative models, KV-cache quantization, low-precision training, and hardware-efficient deployment. The project was initiated by Haotong Qin. Thanks to all contributors for helping grow and maintain this collection.

Quick Navigation

• Research Landscape
• Representative Works
• Benchmarks · Survey Papers
• Papers by Year 2026 · 2025 · 2024 · 2023 · 2022 · 2021 · 2020 · 2019 · 2018 · 2017 · 2016 · 2015 · 2014
• Books · Related Repositories · Researcher Homepages · Contributing / Scope

Research Landscape

Model quantization can be organized along five dimensions:

• Optimization: post-training quantization (PTQ), quantization-aware training (QAT), quantized fine-tuning, data-free methods, and low-precision training.
• Representation: scalar, vector/codebook, lattice, binary-coded, binary/ternary, and mixed-precision quantization.
• Error reduction: rotations, outlier smoothing, residual reconstruction, error compensation, and sensitivity-aware methods.
• Quantized tensors: weights, activations, KV caches, training states, gradients, and communication.
• Models and deployment: vision, language, multimodal, generative, state space, and graph models, alongside edge and hardware systems.

Methods often combine several dimensions, such as PTQ with rotations and vector codebooks.

Optimization paradigm

Post-Training Quantization (PTQ) converts a pretrained model, often with calibration: GPTQ, SmoothQuant, AWQ, OmniQuant, QuaRot, SpinQuant, FlatQuant, BiLLM. Quantization-Aware Training (QAT) models quantization during optimization: PACT, LSQ, IR-Net. Quantized Fine-Tuning / Parameter-Efficient Fine-Tuning (PEFT) adapts low-bit models: QLoRA, QA-LoRA, LoftQ, IR-QLoRA, L4Q. Data-Free / Zero-Shot Quantization avoids original training data, using model statistics or synthetic samples: ZeroQ, Qimera. Low-Precision Training also reduces precision in training computation or stored states: INT8/FP8 training, 8-bit Optimizers.

Representation / coding structure

Scalar quantization codes individual values; non-uniform, logarithmic, and floating-point quantization change the available levels (AdaLog, LLM-FP4). Vector quantization jointly codes tuples; codebook quantization stores reusable representatives; product / grouped vector quantization partitions vectors into groups (GPTVQ, VPTQ, EPQuant). Lattice quantization uses structured geometric codebooks (QuIP#, NestQuant, grouped lattice vector quantizers). Binary-coded quantization combines binary bases (AnyBCQ); binary / ternary quantization constrains values to two / three levels (IR-Net, BiBERT, PT²-LLM). Mixed precision allocates different bit widths or formats across tensors or groups (HAWQ, SliM-LLM).

Transformation / error handling

Rotation / orthogonal transforms redistribute coordinates (QuaRot, SpinQuant); outlier smoothing / redistribution balances quantization difficulty (SmoothQuant, AWQ). Residual / low-rank reconstruction models remaining errors or outliers (LQER, SVDQuant); error compensation corrects quantization effects (GPTQ, First-Order Error Matters). Saliency-aware / Hessian-aware quantization uses importance or curvature to guide precision, reconstruction, or rounding (HAWQ, GPTQ, BiLLM). These techniques can accompany scalar or structured coding.

Quantized object

Weights (GPTQ, AWQ); activations (PACT); weight + activation (SmoothQuant, BiBERT); KV cache (KIVI, KVQuant, ZipCache, PM-KVQ); training states / optimizer states (ActNN, 8-bit Optimizers); gradients / communication (DoReFa-Net, SDP4Bit). Weight bit width does not imply the same activation, accumulator, or cache precision.

Model family / deployment

CNNs / classical vision (XNOR-Net, BRECQ); Vision Transformers (PTQ4ViT); Large Language Models (GPTQ, QLoRA); multimodal / VLM / VLA (Q-VLM, MQuant, AutoQVLA); diffusion / generative models (PTQ4DM, Q-Diffusion, PTQD, ViDiT-Q, SVDQuant, BinaryDM, Q-VDiT, S²Q-VDiT, QuantSparse); Mamba / state space models (Quamba2, SSDi8); graph / point cloud models (EPQuant, BiPointNet); edge / embedded / hardware-oriented systems (HAQ, FINN, LUT-GEMM).

For the structured-coding lineage, QuIP introduces incoherence processing for low-bit LLM quantization; QuIP# connects this direction to lattice codebooks, while QTIP uses trellis coding. GPTVQ, VPTQ, and NestQuant explore vector or lattice representations. TurboQuant and RaBitQ are also retained for their vector-quantization methodology; RaBitQ targets approximate nearest-neighbor search rather than LLM weight quantization.

Bit width alone does not specify storage overhead, arithmetic precision, or deployment speed.

Representative Works

Selected works grouped by technical approach. Click a category to expand its papers, authors and method summaries. Each paper has one primary home here; the yearly collection includes the wider literature.

• Foundations: Classical / QAT · Data-free · Binary / ternary · Vector / codebook
• Language models: Transformer / LLM · Quantized fine-tuning · KV cache
• Models and systems: Generative models · Multimodal / state space · Vision / hardware · Floating-point formats · Training

Reading the links: Scholar opens a title search on Google Scholar, where citation counts can be viewed. Star badges show the linked GitHub repository’s stars, which may cover several papers. These are discovery aids, not rankings.

From binary weights and learned codebooks to integer arithmetic, learned quantizers and reconstruction-based PTQ.

• BinaryConnect: Training Deep Neural Networks with binary weights during propagations Matthieu Courbariaux, Yoshua Bengio, Jean-Pierre David NeurIPS 2015 · Neural Networks QAT Binary Weights · Paper · Code · Scholar Trains neural networks with binary weights during forward and backward propagation.

• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, Ali Farhadi ECCV 2016 · CNN Binary Weight + Activation · Paper · Code · Scholar Approximates convolutions with binary weights and inputs for efficient CNN inference.

• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han, Huizi Mao, William J. Dally ICLR 2016 · CNN Weight Sharing Codebook · Paper · Scholar Combines pruning, trained weight sharing and Huffman coding, connecting learned quantization to compressed model storage.

• PACT: Parameterized Clipping Activation for Quantized Neural Networks Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, Kailash Gopalakrishnan ICLR 2018 · CNN QAT Activations · Paper · Scholar Learns activation clipping thresholds to support low-bit network training.

• Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, Dmitry Kalenichenko CVPR 2018 · QAT INT8 Integer-Only Inference · Paper · Scholar Co-designs quantization-aware training and integer arithmetic for mobile inference, including scale and zero-point handling.

• HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision Zhen Dong, Zhewei Yao, Amir Gholami, Michael W. Mahoney, Kurt Keutzer ICCV 2019 · Mixed Precision Hessian-Aware · Paper · Code · Scholar Uses Hessian information to guide mixed-precision neural network quantization.

• Learned Step Size Quantization Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, Dharmendra S. Modha ICLR 2020 · QAT Low-Bit · Paper · Scholar Learns quantizer step sizes alongside network parameters.

• Up or Down? Adaptive Rounding for Post-Training Quantization Markus Nagel, Rana Ali Amjad, Mart van Baalen, Christos Louizos, Tijmen Blankevoort ICML 2020 · PTQ Rounding · Paper · Scholar Optimizes rounding decisions when converting pretrained weights to low precision.

• HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks Zhen Dong, Zhewei Yao, Daiyaan Arfeen, Amir Gholami, Michael Mahoney, Kurt Keutzer NeurIPS 2020 · Mixed Precision Hessian-Aware · Paper · Code · Scholar Develops trace-weighted Hessian sensitivity for mixed-precision allocation.

• BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, Shi Gu ICLR 2021 · CNN PTQ Reconstruction · Paper · Code · Scholar Uses block reconstruction to reduce post-training quantization error.

• QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization Xiuying Wei, Ruihao Gong, Yuhang Li, Xianglong Liu, Fengwei Yu ICLR 2022 · PTQ Activations Reconstruction · Paper · Code · Scholar Randomly bypasses activation quantization during reconstruction to improve low-bit generalization beyond the calibration data.

These methods replace access to the original dataset with model statistics or generated samples; they may still require calibration or optimization.

• Data-Free Quantization Through Weight Equalization and Bias Correction Markus Nagel, Mart van Baalen, Tijmen Blankevoort, Max Welling ICCV 2019 · CNN PTQ Data-Free · Paper · Scholar Equalizes channel ranges and corrects quantization-induced bias using model parameters and statistics.

• ZeroQ: A Novel Zero Shot Quantization Framework Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gholami, Michael W. Mahoney, Kurt Keutzer CVPR 2020 · CNN Data-Free Mixed Precision · Paper · Code · Scholar Synthesizes calibration inputs from batch-normalization statistics to quantize without the original training dataset.

• Diversifying Sample Generation for Accurate Data-Free Quantization Xiangguo Zhang, Haotong Qin, Yifu Ding, Ruihao Gong, Qinghua Yan, Renshuai Tao, Yuhang Li, Fengwei Yu, Xianglong Liu CVPR 2021 · Oral · CNN Data-Free Synthetic Data PTQ · Paper · Scholar Relaxes batch-normalization statistic matching and varies layer-wise emphasis to diversify synthetic calibration samples for data-free quantization.

• Diverse Sample Generation: Pushing the Limit of Generative Data-Free Quantization Haotong Qin, Yifu Ding, Xiangguo Zhang, Jiakai Wang, Xianglong Liu, Jiwen Lu IEEE TPAMI 2023 · CNN Data-Free PTQ + QAT Sample Diversity · Paper · Code · Scholar Extends the CVPR 2021 DSG method with theoretical analysis and inter-sample decorrelation, improving synthetic-data generation for both PTQ and QAT.

• LLM-QAT: Data-Free Quantization Aware Training for Large Language Models Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, Vikas Chandra ACL Findings 2024 · LLM QAT Data-Free KV Cache · Paper · Code · Scholar Uses the pretrained model’s generated text for distillation-based QAT of weights, activations and KV caches.

Weight-only compression, weight–activation quantization and QAT address different deployment needs. Sparse outlier handling and rotations offer complementary ways to control error.

• LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale Tim Dettmers, Mike Lewis, Younes Belkada, Luke Zettlemoyer NeurIPS 2022 · Transformer INT8 Mixed Precision · Paper · Code · Scholar Enables 8-bit matrix multiplication at transformer scale while handling outlier features in higher precision.

• GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers Elias Frantar, Saleh Ashkboos, Torsten Hoefler, Dan Alistarh ICLR 2023 · LLM PTQ Weights · Paper · Code · Scholar Uses approximate second-order information and error compensation for low-bit weight quantization.

• SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, Song Han ICML 2023 · LLM PTQ Weight + Activation · Paper · Code · Scholar Redistributes activation outlier difficulty into weights to enable low-precision matrix multiplication.

• AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, Song Han MLSys 2024 · LLM PTQ Weights Saliency-Aware · Paper · Code · Scholar Uses activation information to guide weight quantization for on-device compression and acceleration.

• OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, Ping Luo ICLR 2024 · LLM PTQ Calibration · Paper · Code · Scholar Optimizes clipping and equivalent transformations to calibrate low-bit LLMs.

• QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, James Hensman NeurIPS 2024 · LLM PTQ 4-Bit Rotation · Paper · Code · Scholar Uses rotations to suppress outliers and enable 4-bit inference.

• SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, Dan Alistarh ICLR 2024 · LLM PTQ Weights Sparse Outliers · Paper · Code · Scholar Separates sensitive outlier weights into a sparse higher-precision component while quantizing the remaining weights.

• SqueezeLLM: Dense-and-Sparse Quantization Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W. Mahoney, Kurt Keutzer ICML 2024 · LLM PTQ Non-uniform Sparse Outliers · Paper · Code · Scholar Combines sensitivity-weighted non-uniform scalar quantization with a sparse component for outlier weights.

• SpinQuant: LLM Quantization with Learned Rotations Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, Tijmen Blankevoort ICLR 2025 · LLM PTQ Learned Rotation · Paper · Code · Scholar Learns rotations to make LLM representations more amenable to quantization.

• FlatQuant: Flatness Matters for LLM Quantization Yuxuan Sun, Ruikang Liu, Haoli Bai, Han Bao, Kang Zhao, Yuening Li, Jiaxin Hu, Xianzhi Yu, Lu Hou, Chun Yuan, Xin Jiang, Wulong Liu, Jun Yao ICML 2025 · LLM PTQ Transformation · Paper · Code · Scholar Targets distribution flatness to improve LLM quantization.

• EfficientQAT: Efficient Quantization-Aware Training for Large Language Models Mengzhao Chen, Wenqi Shao, Peng Xu, Jiahao Wang, Peng Gao, Kaipeng Zhang, Ping Luo ACL 2025 · LLM QAT Low-Bit · Paper · Code · Scholar Trains block parameters first, then quantization parameters end to end, to reduce the cost of LLM QAT.

QLoRA and related methods adapt low-bit bases with low-rank updates; PV-Tuning also optimizes discrete compressed representations.

• QLoRA: Efficient Finetuning of Quantized LLMs Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer NeurIPS 2023 · LLM PEFT 4-Bit · Paper · Code · Scholar Fine-tunes low-rank adapters through a frozen 4-bit quantized base model.

• QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models Yuhui Xu, Lingxi Xie, Xiaotao Gu, Xin Chen, Heng Chang, Hengheng Zhang, Zhengsu Chen, Xiaopeng Zhang, Qi Tian ICLR 2024 · LLM PEFT Quantization-Aware · Paper · Code · Scholar Combines quantization-aware optimization with low-rank adaptation.

• LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, Tuo Zhao ICLR 2024 · LLM PEFT Low-Bit · Paper · Code · Scholar Aligns quantization with LoRA initialization to reduce the error encountered during adaptation.

• Accurate LoRA-Finetuning Quantization of LLMs via Information Retention Haotong Qin, Xudong Ma, Xingyu Zheng, Xiaoyang Li, Yang Zhang, Shouda Liu, Jie Luo, Xianglong Liu, Michele Magno ICML 2024 · LLM PEFT Information-Aware · Paper · Code · Scholar Uses information retention to improve low-bit quantization and LoRA adaptation.

• PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Burlachenko, Kai Yi, Dan Alistarh, Peter Richtarik NeurIPS 2024 · LLM Quantized Fine-Tuning Discrete Optimization · Paper · Code · Scholar Alternates continuous and discrete optimization to fine-tune extremely compressed models, including additive-codebook representations.

• L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models Hyesung Jeon, Yulhwa Kim, Jae-Joon Kim ACL 2025 · LLM PEFT QAT · Paper · Scholar Combines parameter-efficient fine-tuning with quantization-aware training.

Binary CNNs and transformers, post-training binarization, and native ternary pretraining have different training costs and arithmetic requirements.

• Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm Zechun Liu, Baoyuan Wu, Wenhan Luo, Xin Yang, Wei Liu, Kwang-Ting Cheng ECCV 2018 · CNN Binary QAT · Paper · Code · Scholar Connects real-valued intermediate activations through shortcuts to improve information flow in 1-bit CNNs.

• Forward and Backward Information Retention for Accurate Binary Neural Networks Haotong Qin, Ruihao Gong, Xianglong Liu, Mingzhu Shen, Ziran Wei, Fengwei Yu, Jingkuan Song CVPR 2020 · CNN QAT Binary 1-Bit · Paper · Code · Scholar Retains information in both forward activations and backward gradients when training binary neural networks.

• ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions Zechun Liu, Zhiqiang Shen, Marios Savvides, Kwang-Ting Cheng ECCV 2020 · CNN Binary QAT · Paper · Code · Scholar Learns activation shifts and reshaping functions to reduce the accuracy gap between binary and real-valued networks.

• BiBERT: Accurate Fully Binarized BERT Haotong Qin, Yifu Ding, Mingyuan Zhang, Qinghua Yan, Aishan Liu, Qingqing Dang, Ziwei Liu, Xianglong Liu ICLR 2022 · Transformer NLP Binary Weight + Activation · Paper · Code · Scholar Targets fully binarized BERT, extending binary networks to transformer language models.

• BiLLM: Pushing the Limit of Post-Training Quantization for LLMs Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, Xiaojuan Qi ICML 2024 · LLM PTQ Binary Extreme Low-Bit · Paper · Code · Scholar Uses saliency-aware binarization to push pretrained LLM weights into the extreme low-bit regime.

• DB-LLM: Accurate Dual-Binarization for Efficient LLMs Hong Chen, Chengtao Lv, Liang Ding, Haotong Qin, Xiabin Zhou, Yifu Ding, Xuebo Liu, Min Zhang, Jinyang Guo, Xianglong Liu, Dacheng Tao ACL Findings 2024 · LLM Dual Binarization Extreme Low-Bit · Paper · Code · Scholar Uses dual binarization to compress LLMs while retaining accuracy.

• The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, Furu Wei arXiv 2024 · LLM QAT Ternary Weights 8-Bit Activations · Paper · Scholar Extends BitNet’s quantization-aware pretraining to ternary weights; this is a training recipe, distinct from post-training binarization.

• ARB-LLM: Alternating Refined Binarizations for Large Language Models Zhiteng Li, Xianglong Yan, Tianao Zhang, Haotong Qin, Dong Xie, Jiang Tian, Zhongchao Shi, Linghe Kong, Yulun Zhang, Xiaokang Yang ICLR 2025 · LLM Binary Extreme Low-Bit · Paper · Code · Scholar Refines alternating binarizations for low-bit LLM representation.

• PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models Jiaqi Zhao, Miao Zhang, Ming Wang, Yuzhang Shang, Kaihao Zhang, Weili Guan, Yaowei Wang, Min Zhang ACL 2025 · LLM PTQ Extreme Low-Bit · Paper · Code · Scholar Explores extremely low-bit post-training quantization for LLMs.

• PT²-LLM: Post-Training Ternarization for Large Language Models Xianglong Yan, Chengzhu Bao, Zhiteng Li, Tianao Zhang, Kaicheng Yang, Haotong Qin, Ruobing Xie, Xingwu Sun, Yulun Zhang ICLR 2026 · LLM PTQ Ternary · Paper · Code · Scholar Converts pretrained large language models to ternary representations.

From CNN product quantization to LLM additive, lattice and trellis codes. QuIP provides the incoherence-processing precursor; RaBitQ contributes vector-search methodology.

• Compressing Deep Convolutional Networks using Vector Quantization Yunchao Gong, Liu Liu, Ming Yang, Lubomir Bourdev arXiv 2014 · CNN Vector Quantization Product Quantization · Paper · Scholar Studies clustering and product quantization of CNN parameters as early approaches to reducing model storage.

• QuIP: 2-Bit Quantization of Large Language Models With Guarantees Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, Christopher De Sa NeurIPS 2023 · LLM PTQ 2-Bit Incoherence · Paper · Code · Scholar Uses incoherence processing for low-bit quantization with guarantees, forming a precursor to the QuIP# lattice-codebook lineage.

• QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, Christopher De Sa ICML 2024 · LLM Lattice Codebook Hadamard · Paper · Code · Scholar Combines Hadamard incoherence processing with lattice codebooks for LLM quantization.

• QTIP: Quantization with Trellises and Incoherence Processing Albert Tseng, Qingyao Sun, David Hou, Christopher De Sa NeurIPS 2024 · LLM Trellis Coding Incoherence · Paper · Code · Scholar Combines trellis-based quantization with incoherence processing for compact LLM representation.

• GPTVQ: The Blessing of Dimensionality for LLM Quantization Mart van Baalen, Andrey Kuzmin, Ivan Koryakovskiy, Markus Nagel, Peter Couperus, Cedric Bastoul, Eric Mahurin, Tijmen Blankevoort, Paul Whatmough arXiv 2024 · LLM Vector Quantization Weights · Paper · Code · Scholar Exploits joint quantization of multiple weight coordinates rather than coding each weight independently.

• VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models Yifei Liu, Jicheng Wen, Yang Wang, Shengyu Ye, Li Lyna Zhang, Ting Cao, Cheng Li, Mao Yang EMNLP 2024 · LLM PTQ Vector Quantization Extreme Low-Bit · Paper · Code · Scholar Uses vector post-training quantization for extremely low-bit LLM compression.

• RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search Jianyang Gao, Cheng Long SIGMOD 2024 · Vector Quantization Binary Codes Vector Search · Paper · Code · Scholar Quantizes high-dimensional vectors with a theoretical error bound for approximate nearest-neighbor search.

• Extreme Compression of Large Language Models via Additive Quantization Vage Egiazarian, Andrei Panferov, Denis Kuznedelev, Elias Frantar, Artem Babenko, Dan Alistarh ICML 2024 · LLM PTQ Additive Codebooks 2–3 Bit · Paper · Code · Scholar Represents weight vectors as sums of learned codewords and jointly optimizes codebooks within transformer blocks.

• NestQuant: nested lattice quantization for matrix products and LLMs Semyon Savkin, Eitan Porat, Or Ordentlich, Yury Polyanskiy ICML 2025 · LLM Lattice Matrix Products · Paper · Code · Scholar Uses nested lattice quantization for matrix products and LLMs.

• Learning Grouped Lattice Vector Quantizers for Low-Bit Large Language Models Xi Zhang, Xiaolin Wu, Jiamang Wang, Weisi Lin NeurIPS 2025 · LLM Grouped Vector Quantization Lattice · Paper · Code · Scholar Learns grouped lattice vector quantizers for low-bit LLM representation.

• AnyBCQ: Hardware Efficient Flexible Binary-Coded Quantization for Multi-Precision LLMs Gunho Park, Jeongin Bae, Beomseok Kwon, Byeongwook Kim, Se Jung Kwon, Dongsoo Lee ICLR 2026 · LLM Binary-Coded Mixed Precision Hardware · Paper · Code · Scholar Develops flexible binary-coded quantization for hardware-efficient multi-precision LLMs.

• TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate Amir Zandieh, Majid Daliri, Majid Hadian, Vahab Mirrokni ICLR 2026 · Vector Quantization Online Distortion · Paper · Scholar Studies online vector quantization with near-optimal distortion rate.

These methods compress inference-time key and value tensors; their bit widths are separate from model weight precision.

• KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, Xia Hu ICML 2024 · LLM KV Cache 2-Bit · Paper · Code · Scholar Uses asymmetric, tuning-free 2-bit quantization to compress key and value caches.

• KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael Mahoney, Sophia Shao, Kurt Keutzer, Amir Gholami NeurIPS 2024 · LLM KV Cache Long Context · Paper · Code · Scholar Targets long-context inference by reducing the memory occupied by the KV cache.

• ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification Yefei He, Luoming Zhang, Weijia Wu, Jing Liu, Hong Zhou, Bohan Zhuang NeurIPS 2024 · LLM KV Cache Salient Tokens · Paper · Code · Scholar Uses salient-token identification to guide accurate and efficient cache quantization.

• PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs Tengxuan Liu, Shiyao Li, Jiayi Yang, Tianchen Zhao, Feng Zhou, Xiaohui Song, Guohao Dai, Shengen Yan, Huazhong Yang, Yu Wang ICLR 2026 · LLM KV Cache Mixed Precision · Paper · Code · Scholar Progressively quantizes KV caches with mixed precision for long chain-of-thought inference.

Early diffusion PTQ addresses denoising-step sensitivity; later work extends to diffusion transformers, low-rank outlier handling and video generation.

• Post-training Quantization on Diffusion Models Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, Yan Yan CVPR 2023 · Diffusion PTQ · Paper · Code · Scholar Adapts post-training quantization to diffusion model inference.

• Q-diffusion: Quantizing Diffusion Models Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, Kurt Keutzer ICCV 2023 · Diffusion PTQ · Paper · Code · Scholar Quantizes diffusion models to reduce the cost of iterative generation.

• PTQD: Accurate Post-Training Quantization for Diffusion Models Yefei He, Luping Liu, Jing Liu, Weijia Wu, Hong Zhou, Bohan Zhuang NeurIPS 2023 · Diffusion PTQ Error Handling · Paper · Code · Scholar Targets accurate diffusion generation through post-training quantization error handling.

• ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation Tianchen Zhao, Tongcheng Fang, Haofeng Huang, Rui Wan, Widyadewi Soedarmadji, Enshu Liu, Shiyao Li, Zinan Lin, Guohao Dai, Shengen Yan, Huazhong Yang, Xuefei Ning, Yu Wang ICLR 2025 · Diffusion Transformer Image + Video Low-Bit · Paper · Code · Scholar Quantizes diffusion transformers for both image and video generation.

• SVDQuant: Absorbing Outliers by Low-Rank Component for 4-Bit Diffusion Models Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, Song Han ICLR 2025 · Diffusion 4-Bit Low-Rank · Paper · Code · Scholar Absorbs outliers into a low-rank component to support 4-bit diffusion models.

• BinaryDM: Accurate Weight Binarization for Efficient Diffusion Models Xingyu Zheng, Xianglong Liu, Haotong Qin, Xudong Ma, Mingyuan Zhang, Haojie Hao, Jiakai Wang, Zixiang Zhao, Jinyang Guo, Michele Magno ICLR 2025 · Diffusion Binary Weights · Paper · Code · Scholar Binarizes diffusion model weights for efficient generation.

• Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformers Weilun Feng, Chuanguang Yang, Haotong Qin, Xiangqi Li, Yu Wang, Zhulin An, Libo Huang, Boyu Diao, Zixiang Zhao, Yongjun Xu, Michele Magno ICML 2025 · Video Diffusion Quantization Distillation · Paper · Code · Scholar Combines quantization and distillation for video-generation diffusion transformers.

• S²Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation Weilun Feng, Haotong Qin, Chuanguang Yang, Xiangqi Li, Han Yang, Yuqi Li, Zhulin An, Libo Huang, Michele Magno, Yongjun Xu NeurIPS 2025 · Video Diffusion Quantization Distillation · Paper · Code · Scholar Uses salient data and sparse-token distillation to improve quantized video diffusion transformers.

• QuantSparse: Comprehensively Compressing Video Diffusion Transformer with Model Quantization and Attention Sparsification Weilun Feng, Chuanguang Yang, Haotong Qin, Mingqiang Wu, Yuqi Li, Xiangqi Li, Zhulin An, Libo Huang, Yulun Zhang, Michele Magno, Yongjun Xu ICLR 2026 · Video Diffusion Quantization Attention Sparsity · Paper · Code · Scholar Combines model quantization and attention sparsification to compress video diffusion transformers.

Vision-language models and selective state space models introduce quantization sensitivities beyond those of language-only transformers.

• Q-VLM: Post-training Quantization for Large Vision-Language Models Changyuan Wang, Ziwei Wang, Xiuwei Xu, Yansong Tang, Jie Zhou, Jiwen Lu NeurIPS 2024 · VLM PTQ Cross-Layer Dependency · Paper · Code · Scholar Uses cross-layer dependencies to guide block partitioning and quantization of vision-language models.

• Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models Hung-Yueh Chiang, Chi-Chih Chang, Natalia Frumkin, Kai-Chiang Wu, Mohamed S. Abdelfattah, Diana Marculescu ICML 2025 · Mamba State Space Models PTQ W4A8 / W8A8 · Paper · Code · Scholar Uses channel clustering and state-group quantization to accommodate the sensitivity of Mamba’s selective state-space computations.