arXivAI Research
Untying Embeddings in LLMs Improves Privacy-Preserving DP-SGD Fine-Tuning Utility and Memory
差分隱私訓練新發現：解除權重共享能為 LLM 提升 4.74% 準確度並降低 60% 記憶體
Research reveals that untying input and output embeddings in decoder-only LLMs trained under DP-SGD improves accuracy by up to 4.74% and slashes memory usage by over 60%.
2 min read