Compared to action-level runtime defenses AgentSpec and TS-Guard, SkillSonar achieves superior safety–utility operating points on Claude Code with GLM-5: AgentSpec achieves 0.294 ID ASR but drops task utility to 0.581, while TS-Guard achieves 0.412 ID ASR, whereas SkillSonar reaches 0.104 ID ASR with 0.815 task utility.
Source: source-paper:Figure 5 and Section 5.3 prose, page 11-12
Reported and observed measurements
attack_success_rate
m-fig5-agentspec-asr-id
Reported 0.294 fraction
Observed — fraction
attack_success_rate
m-fig5-agentspec-asr-ood
Reported 0.576 fraction
Observed — fraction
task_utility
m-fig5-agentspec-taskutil-id
Reported 0.581 score
Observed — score
benign_task_utility
m-fig5-agentspec-benign-util
Reported 0.642 score
Observed — score
attack_success_rate
m-fig5-tsguard-asr-id
Reported 0.412 fraction
Observed — fraction
attack_success_rate
m-fig5-tsguard-asr-ood
Reported 0.492 fraction
Observed — fraction
benign_task_utility
m-fig5-tsguard-benign-util
Reported 0.703 score
Observed — score
attack_success_rate
m-fig5-noguard-asr-id
Reported 0.477 fraction
Observed — fraction
task_utility
m-fig5-noguard-taskutil-id
Reported 0.829 score
Observed — score
benign_task_utility
m-fig5-noguard-benign-util
Reported 0.757 score
Observed — score
Assessments (0)
No immutable Assessment has been published for this Claim yet.
Runs (0)
No execution has been linked to this Claim yet.