arXivAI Research
Rounding in Preconditioner Space: Redesigning 4-bit AdamW Quantization
預調節器空間捨入：重構 4-bit AdamW 優化器狀態量化
Researchers introduce preconditioner-space rounding methods (ZIP-SR and ZE-EDEN) for 4-bit AdamW quantization, mitigating error propagation and reducing the validation loss gap to 32-bit AdamW by up to 70%.
2 min read