Mamba-2-130M trained on 300B tokens of the Pile achieves zero-shot downstream task performance of 43.9% on LAMBADA, 35.3% on HellaSwag, 64.9% on PIQA, 47.4% on Arc-Easy, 24.2% on Arc-Challenge, 52.1% on WinoGrande, 30.6% on OpenbookQA, and 42.6% average accuracy across tasks.

来源：source-paper:Table 10, page 52 (and Table 1, page 29)

报告指标与观测值

lambada_acc

rm-130m-lambada-acc

论文报告 43.9 percent

实际观测 — percent

hellaswag_acc

rm-130m-hellaswag-acc

论文报告 35.3 percent

实际观测 — percent

piqa_acc

rm-130m-piqa-acc

论文报告 64.9 percent

实际观测 — percent

arc_easy_acc

rm-130m-arc-e-acc

论文报告 47.4 percent

实际观测 — percent

arc_challenge_acc

rm-130m-arc-c-acc

论文报告 24.2 percent

实际观测 — percent

winogrande_acc

rm-130m-winogrande-acc

论文报告 52.1 percent

实际观测 — percent

openbookqa_acc

rm-130m-openbookqa-acc

论文报告 30.6 percent

实际观测 — percent

average_acc

rm-130m-avg-acc

论文报告 42.6 percent

实际观测 — percent

这条 Claim 暂无已发布的不可变 Assessment。

这条 Claim 暂无关联执行。