In rollout count ablation on Qwen3-0.6B-Base, increasing K from 1 to 2 yields a significant performance jump (from 15.15% to 20.20% on GPQA-Diamond, and from 6.67% to 10.00% on AIME 25 Pass@16), while further increasing K to 4 or 8 yields negligible marginal benefits.
Source: source-paper:Table 2, Section 4.3, page 6
Reported and observed measurements
pass_at_16
m-osw-k1-aime24-p16
Reported 6.67 percentage_points
Observed — percentage_points
pass_at_16
m-osw-k1-aime25-p16
Reported 6.67 percentage_points
Observed — percentage_points
pass_at_1
m-osw-k1-math500-p1
Reported 22.8 percentage_points
Observed — percentage_points
pass_at_1
m-osw-k1-gpqad-p1
Reported 15.15 percentage_points
Observed — percentage_points
pass_at_16
m-osw-k2-aime24-p16
Reported 6.67 percentage_points
Observed — percentage_points
pass_at_16
m-osw-k2-aime25-p16
Reported 10 percentage_points
Observed — percentage_points
pass_at_1
m-osw-k2-math500-p1
Reported 24.2 percentage_points
Observed — percentage_points
pass_at_1
m-osw-k2-gpqad-p1
Reported 20.2 percentage_points
Observed — percentage_points
pass_at_1
m-osw-k4-gpqad-p1
Reported 20.71 percentage_points
Observed — percentage_points
pass_at_1
m-osw-k8-gpqad-p1
Reported 20.71 percentage_points
Observed — percentage_points
Assessments (0)
No immutable Assessment has been published for this Claim yet.
Runs (0)
No execution has been linked to this Claim yet.