Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents
尚未评估计划受阻研究发现claim-figure4bc-content-matched

Packaging the safety policy as a modular skill-native artifact provides substantial gains over flattening the identical policy into a system prompt: SkillSonar reduces ID ASR from 0.353 to 0.104 and OOD ASR from 0.437 to 0.109, increases ID task utility from 0.655 to 0.815, and lowers token consumption from ~239K to ~188K (a ~21% reduction).

来源:source-paper:Figure 4(b-c) and Section 5.3 prose, page 11

报告指标与观测值

attack_success_rate

m-fig4bc-flat-asr-id

论文报告 0.353 fraction

实际观测 — fraction

attack_success_rate

m-fig4bc-flat-asr-ood

论文报告 0.437 fraction

实际观测 — fraction

attack_success_rate

m-fig4bc-skill-asr-id

论文报告 0.104 fraction

实际观测 — fraction

attack_success_rate

m-fig4bc-skill-asr-ood

论文报告 0.109 fraction

实际观测 — fraction

task_utility

m-fig4bc-flat-taskutil-id

论文报告 0.655 score

实际观测 — score

task_utility

m-fig4bc-flat-taskutil-ood

论文报告 0.672 score

实际观测 — score

task_utility

m-fig4bc-skill-taskutil-id

论文报告 0.815 score

实际观测 — score

task_utility

m-fig4bc-skill-taskutil-ood

论文报告 0.789 score

实际观测 — score

benign_task_utility

m-fig4bc-flat-benign-util

论文报告 0.748 score

实际观测 — score

benign_task_utility

m-fig4bc-skill-benign-util

论文报告 0.763 score

实际观测 — score

token_usage

m-fig4bc-flat-tokens

论文报告 239 thousands_of_tokens

实际观测 — thousands_of_tokens

token_usage

m-fig4bc-skill-tokens

论文报告 188 thousands_of_tokens

实际观测 — thousands_of_tokens

Assessment(0)

这条 Claim 暂无已发布的不可变 Assessment。

关联运行(0)

这条 Claim 暂无关联执行。