AdamW

TACO Optimizer: Unleashing Full-Parameter 32B LLM Fine-Tuning on a Single GPU

TACO 最佳化器：將 32B 大模型全參數微調帶入單張 GPU 的極簡幾何學

TACO is an ultra-low-memory optimizer that reduces persistent optimizer states by 174x, allowing full-parameter fine-tuning of 32B models on a single 80GB GPU.