Mamba-2-1.3B trained on 300B tokens of the Pile achieves zero-shot downstream task performance of 65.7% on LAMBADA, 59.9% on HellaSwag, 73.2% on PIQA, 64.3% on Arc-Easy, 33.3% on Arc-Challenge, 60.9% on WinoGrande, 37.8% on OpenbookQA, and 56.4% average accuracy across tasks.

来源：source-paper:Table 1, page 29 (and Table 10, page 52)

报告指标与观测值

lambada_acc

rm-1.3b-lambada-acc

论文报告 65.7 percent

实际观测 — percent

hellaswag_acc

rm-1.3b-hellaswag-acc

论文报告 59.9 percent

实际观测 — percent

piqa_acc

rm-1.3b-piqa-acc

论文报告 73.2 percent

实际观测 — percent

arc_easy_acc

rm-1.3b-arc-e-acc

论文报告 64.3 percent

实际观测 — percent

arc_challenge_acc

rm-1.3b-arc-c-acc

论文报告 33.3 percent

实际观测 — percent

winogrande_acc

rm-1.3b-winogrande-acc

论文报告 60.9 percent

实际观测 — percent

openbookqa_acc

rm-1.3b-openbookqa-acc

论文报告 37.8 percent

实际观测 — percent

average_acc

rm-1.3b-avg-acc

论文报告 56.4 percent

实际观测 — percent

这条 Claim 暂无已发布的不可变 Assessment。

这条 Claim 暂无关联执行。