Event

QSelect: A Combined Low-Bit Compression Method Proposed for Spaceborne Language Models

1 reports · 1 sources

Jia Deng et al. proposed the QSelect method, which combines three quantizers—AWQ, GPTQ, and AutoRound—assigned per linear module into a single static low-bit model, for compressing DeepSeek-R1-Distill-Qwen-7B, Llama-3.1-8B, and Llama-3.2-2B under 3-bit and 4-bit settings. The default fidelity mode requires no labels or tuned thresholds and accepts a replacement only when the forward KL divergence decreases relative to the BF16 reference; the optional target-adaptation mode additionally uses answer likelihood on a small amount of labeled data. When using training or development set labels from the evaluated task families, the target-adaptation mode improved 3-bit aggregate accuracy over the best single quantizer by 4.3 and 4.5 percentage points on the two Llama models. At 3 bits, the unquantized embedding and output layers account for 43%–46% of the packaged artifact. All evaluations used general-domain benchmarks on ground-based GPUs; space-domain tasks and flight hardware are left for future work.