Mamba-2-780M trained on 300B tokens of the Pile achieves zero-shot downstream task performance of 61.7% on LAMBADA, 54.9% on HellaSwag, 72.0% on PIQA, 61.0% on Arc-Easy, 28.5% on Arc-Challenge, 60.2% on WinoGrande, 36.2% on OpenbookQA, and 53.5% average accuracy across tasks.

来源：source-paper:Table 1, page 29 (and Table 10, page 52)

报告指标与观测值

lambada_acc

rm-780m-lambada-acc

论文报告 61.7 percent

实际观测 — percent

hellaswag_acc

rm-780m-hellaswag-acc

论文报告 54.9 percent

实际观测 — percent

piqa_acc

rm-780m-piqa-acc

论文报告 72 percent

实际观测 — percent

arc_easy_acc

rm-780m-arc-e-acc

论文报告 61 percent

实际观测 — percent

arc_challenge_acc

rm-780m-arc-c-acc

论文报告 28.5 percent

实际观测 — percent

winogrande_acc

rm-780m-winogrande-acc

论文报告 60.2 percent

实际观测 — percent

openbookqa_acc

rm-780m-openbookqa-acc

论文报告 36.2 percent

实际观测 — percent

average_acc

rm-780m-avg-acc

论文报告 53.5 percent

实际观测 — percent

这条 Claim 暂无已发布的不可变 Assessment。

这条 Claim 暂无关联执行。