Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall
尚未评估计划受阻研究发现claim_fig2_reasoning_recall_tradeoff_sweep
Across distillation strengths alpha in [0.0, 1.0], KL directions (forward and reverse), and teacher sizes (1B, 7B, 13B), mid-training distillation traces a reasoning-recall frontier that falls below NTP, which Switch Distillation mitigates.
来源:source_paper:Figure 2, Page 4
报告指标与观测值
这条 Claim 暂无结构化指标。
Assessment(0)
这条 Claim 暂无已发布的不可变 Assessment。
关联运行(0)
这条 Claim 暂无关联执行。