Packaging the safety policy as a modular skill-native artifact provides substantial gains over flattening the identical policy into a system prompt: SkillSonar reduces ID ASR from 0.353 to 0.104 and OOD ASR from 0.437 to 0.109, increases ID task utility from 0.655 to 0.815, and lowers token consumption from ~239K to ~188K (a ~21% reduction).
来源:source-paper:Figure 4(b-c) and Section 5.3 prose, page 11
报告指标与观测值
attack_success_rate
m-fig4bc-flat-asr-id
论文报告 0.353 fraction
实际观测 — fraction
attack_success_rate
m-fig4bc-flat-asr-ood
论文报告 0.437 fraction
实际观测 — fraction
attack_success_rate
m-fig4bc-skill-asr-id
论文报告 0.104 fraction
实际观测 — fraction
attack_success_rate
m-fig4bc-skill-asr-ood
论文报告 0.109 fraction
实际观测 — fraction
task_utility
m-fig4bc-flat-taskutil-id
论文报告 0.655 score
实际观测 — score
task_utility
m-fig4bc-flat-taskutil-ood
论文报告 0.672 score
实际观测 — score
task_utility
m-fig4bc-skill-taskutil-id
论文报告 0.815 score
实际观测 — score
task_utility
m-fig4bc-skill-taskutil-ood
论文报告 0.789 score
实际观测 — score
benign_task_utility
m-fig4bc-flat-benign-util
论文报告 0.748 score
实际观测 — score
benign_task_utility
m-fig4bc-skill-benign-util
论文报告 0.763 score
实际观测 — score
token_usage
m-fig4bc-flat-tokens
论文报告 239 thousands_of_tokens
实际观测 — thousands_of_tokens
token_usage
m-fig4bc-skill-tokens
论文报告 188 thousands_of_tokens
实际观测 — thousands_of_tokens
Assessment(0)
这条 Claim 暂无已发布的不可变 Assessment。
关联运行(0)
这条 Claim 暂无关联执行。