Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
尚未评估已列入计划研究发现claim-pretraining-validation-perplexity
When pretrained on 300B tokens of the Pile, Mamba-2 achieves validation perplexities of 10.48 at 130M parameters, 8.21 at 370M parameters, 7.26 at 780M parameters, 6.66 at 1.3B parameters, and 6.09 at 2.7B parameters.
来源：source-paper:Table 1, page 29 and Table 10, page 52
报告指标与观测值
pile_validation_perplexity
rm-pile-ppl-130m
论文报告 10.48 perplexity
实际观测 — perplexity
pile_validation_perplexity
rm-pile-ppl-370m
论文报告 8.21 perplexity
实际观测 — perplexity
pile_validation_perplexity
rm-pile-ppl-780m
论文报告 7.26 perplexity
实际观测 — perplexity
pile_validation_perplexity
rm-pile-ppl-1.3b
论文报告 6.66 perplexity
实际观测 — perplexity
pile_validation_perplexity
rm-pile-ppl-2.7b
论文报告 6.09 perplexity
实际观测 — perplexity
Assessment（0）
这条 Claim 暂无已发布的不可变 Assessment。
关联运行（0）
这条 Claim 暂无关联执行。