arXivAI Research
Decoupling Direction and Step-Size: ZFO Framework for LLM Fine-Tuning
結合零階與一階優化：ZFO 框架為大語言模型微調尋找最佳步長
The ZFO framework decouples direction selection from step-size by using a first-order optimizer for direction and only two zeroth-order evaluations to dynamically find the optimal curvature-aware step size.
2 min read