Factual acquisition stratification by teacher entropy quintile persists when using OLMo-2 1B Instruct and 13B Instruct teachers.
Source: source_paper:Figure 10, Page 29
Reported and observed measurements
factual_examples_learned
fig10_factual_acq_1b_pretraining_100b_q1
Reported 76 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_1b_pretraining_100b_q2
Reported 48 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_1b_pretraining_100b_q3
Reported 30 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_1b_pretraining_100b_q4
Reported 12 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_1b_pretraining_100b_q5
Reported 6 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_1b_midtraining_init_4t_q1
Reported 81 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_1b_midtraining_init_4t_q2
Reported 61 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_1b_midtraining_init_4t_q3
Reported 40 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_1b_midtraining_init_4t_q4
Reported 18 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_1b_midtraining_init_4t_q5
Reported 8 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_13b_pretraining_100b_q1
Reported 54 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_13b_pretraining_100b_q2
Reported 48 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_13b_pretraining_100b_q3
Reported 37 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_13b_pretraining_100b_q4
Reported 28 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_13b_pretraining_100b_q5
Reported 6 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_13b_midtraining_init_4t_q1
Reported 66 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_13b_midtraining_init_4t_q2
Reported 58 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_13b_midtraining_init_4t_q3
Reported 43 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_13b_midtraining_init_4t_q4
Reported 32 percentage_points
Observed — percentage_points
factual_examples_learned
fig10_factual_acq_13b_midtraining_init_4t_q5
Reported 8 percentage_points
Observed — percentage_points
Assessments (0)
No immutable Assessment has been published for this Claim yet.
Runs (0)
No execution has been linked to this Claim yet.