Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall
Not assessedPlan blockedMeasurementclaim_fig10_factual_acquisition_1b_13b

Factual acquisition stratification by teacher entropy quintile persists when using OLMo-2 1B Instruct and 13B Instruct teachers.

Source: source_paper:Figure 10, Page 29

Reported and observed measurements

factual_examples_learned

fig10_factual_acq_1b_pretraining_100b_q1

Reported 76 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_1b_pretraining_100b_q2

Reported 48 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_1b_pretraining_100b_q3

Reported 30 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_1b_pretraining_100b_q4

Reported 12 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_1b_pretraining_100b_q5

Reported 6 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_1b_midtraining_init_4t_q1

Reported 81 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_1b_midtraining_init_4t_q2

Reported 61 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_1b_midtraining_init_4t_q3

Reported 40 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_1b_midtraining_init_4t_q4

Reported 18 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_1b_midtraining_init_4t_q5

Reported 8 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_13b_pretraining_100b_q1

Reported 54 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_13b_pretraining_100b_q2

Reported 48 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_13b_pretraining_100b_q3

Reported 37 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_13b_pretraining_100b_q4

Reported 28 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_13b_pretraining_100b_q5

Reported 6 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_13b_midtraining_init_4t_q1

Reported 66 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_13b_midtraining_init_4t_q2

Reported 58 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_13b_midtraining_init_4t_q3

Reported 43 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_13b_midtraining_init_4t_q4

Reported 32 percentage_points

Observed — percentage_points

factual_examples_learned

fig10_factual_acq_13b_midtraining_init_4t_q5

Reported 8 percentage_points

Observed — percentage_points

Assessments (0)

No immutable Assessment has been published for this Claim yet.

Runs (0)

No execution has been linked to this Claim yet.