Failed episodes exhibit systematically more negative temporal curvature kappa across all six task-LLM pairings, with correct minus incorrect mean differences of: Math (Llama-3.1-8B) +0.047, Code (Qwen3-14B) +0.148, Retail (Qwen3-14B) +0.030, Airline (Qwen3-14B) +0.049, Retail (Qwen3-32B) +0.035, and Airline (Qwen3-32B) +0.088.
Source: paper:Page 4, Section 4.2, Paragraph 'Failed episodes exhibit systematically more negative kappa'
Reported and observed measurements
curvature_difference_correct_minus_incorrect
meas-diff-kappa-math-8b
Reported 0.047 scalar
Observed — scalar
curvature_difference_correct_minus_incorrect
meas-diff-kappa-code-14b
Reported 0.148 scalar
Observed — scalar
curvature_difference_correct_minus_incorrect
meas-diff-kappa-retail-14b
Reported 0.03 scalar
Observed — scalar
curvature_difference_correct_minus_incorrect
meas-diff-kappa-airline-14b
Reported 0.049 scalar
Observed — scalar
curvature_difference_correct_minus_incorrect
meas-diff-kappa-retail-32b
Reported 0.035 scalar
Observed — scalar
curvature_difference_correct_minus_incorrect
meas-diff-kappa-airline-32b
Reported 0.088 scalar
Observed — scalar
Assessments (0)
No immutable Assessment has been published for this Claim yet.
Runs (0)
No execution has been linked to this Claim yet.