Per-task downstream accuracy for mid-training ablation random_routing using OLMo-2 7B Instruct teacher.
Source: source_paper:Table 9, Page 33
Reported and observed measurements
downstream_accuracy
t9_random_routing_gsm8k
Reported 61.5 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_gsm_s
Reported 45.5 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_gsmplus
Reported 38.6 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_bbh
Reported 32 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_drop
Reported 40.6 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_math
Reported 10.8 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_tqa
Reported 53.2 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_nq
Reported 23.9 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_sqa
Reported 8.5 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_mmlu
Reported 49.6 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_mmlu_p
Reported 17.8 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_arc_c
Reported 61.6 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_obqa
Reported 62.6 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_wino
Reported 52.6 percentage_points
Observed — percentage_points
downstream_accuracy
t9_random_routing_agi
Reported 39.5 percentage_points
Observed — percentage_points
Assessments (0)
No immutable Assessment has been published for this Claim yet.
Runs (0)
No execution has been linked to this Claim yet.