#transformer
4 articles
伸縮自如的語言模型：單次訓練即可適應多種運算資源預算的 Telescopic LM
This research introduces Telescopic Language Models (TLM), which use stochastic prefix supervision to enable a single Transformer to act as a valid language model at any layer depth, serving diverse compute budgets from a single training run.
新增 LoRA 技能唯讀不寫：READ 解決多配接器融合干擾
This paper introduces READ, a method that resolves interference when merging multiple LoRA adapters by enforcing "read-only" one-way coupling and canonical factorization, preserving old skills while adding new ones with zero extra inference cost.
長序列決策救星：Trust Guided Decision Transformer 透過預測誤差與共形預測克服上下文漂移
This study introduces TGDT, which monitors Decision Transformer's next-state prediction errors and applies split conformal prediction to filter out unreliable contexts, improving long-horizon performance.
權重配對編碼（WeightPE）：將類神經網路權重壓縮為更小、更具彈性的語法結構
This study introduces WeightPE, the first method to use grammar size as an explicit training objective, optimizing neural network weights into highly compressible, hierarchical, and variable-length structures.