Geometry of Divergence: Tracking Hidden-State Trajectories for Adaptive Multi-Turn Reasoning
PublicMore
AI research summary
LLM agents need to sustain goal-consistent reasoning across long multi-turn interactions under strict resource constraints. However, as the multi-turn context accumulates, it can destabilize the underlying LLM's internal representation of task-relevant information from earlier turns, blurring the boundary between constructive reasoning and representation drift. We formulate multi-turn reasoning as a hidden-state trajectory of the underlying LLM that is characterized via two complementary signals: temporal curvature that captures the directional consistency of turn-to-turn updates, and variance slope which measures the expansion or contraction of the exploration space. Across four tasks and three underlying LLMs, we observed that these geometric signals distinguish between correct and incorrect episodes prior to completion. We further decompose each episode into three-action chains formed from four actions (Read, Write, Respond, Transfer) and show that separability is action-dependent, with different signals distinguishing various chain patterns. Our experiments demonstrate that trajectory geometry can identify critical turns in the reasoning process, increasing task success rates on tau-Bench from 24.1% to 39.6% while reducing token cost by 11.2%.
Generated by CiteArk.
What this means
Autonomous LLM agents frequently fail during multi-turn interactions because accumulating interaction history destabilizes their internal representations, causing context saturation, representation drift, and unneeded reasoning overhead. This paper proposes tracking the trajectory of intermediate hidden states across conversational turns using two geometric metrics: temporal curvature (measuring directional consistency of hidden-state displacements) and variance slope (measuring expansion or contraction of the exploration space). Instead of relying on static context engineering or always-on chain-of-thought deliberation, the authors demonstrate that monitoring these latent geometric signals can trigger deliberative reasoning dynamically only when failure or drift is imminent. For practitioners and researchers developing conversational agents in tool-grounded domains like retail and airline customer support, this method offers a potential route to improve task success rates while lowering overall inference token costs. A primary operational limitation is that extracting hidden states requires direct white-box access to the model's internal activations at specific intermediate layers, making the approach inaccessible for closed-source proprietary API-only models.
Run · Match · Repeat
Run · Match · Repeat shows the deepest evidence stage reached by any claim in this repository, not paper-wide reproduction coverage.
An outer circle appears only after a paper author or trusted institution signs the Artifact.
Not reproduced yet
0/14
No verifiable claim has successful reproduction evidence yet.
Research claims
The main empirical claims extracted in this research plan, with their plan coverage and live evidence state. Click a claim to expand it inline.
Failed episodes exhibit systematically more negative temporal curvature kappa across all six task-LLM pairings, with correct minus incorrect mean differences of: Math (Llama-3.1-8B) +0.047, Code (Qwen3-14B) +0.148, Retail (Qwen3-14B) +0.030, Airline (Qwen3-14B) +0.049, Retail (Qwen3-32B) +0.035, and Airline (Qwen3-32B) +0.088.Experiment blocked; see the specific reasonReported 0.047 scalar
Reported
0.047 scalar
Observed
—
The offline episode traces, prompt sharding scripts for Math and Code from Lost in Conversation, and exact hidden state trajectory extraction codebase are not provided in the paper snapshot or any public repository.
curvature_difference_correct_minus_incorrect · Not assessed
Reported: 0.047 scalar
curvature_difference_correct_minus_incorrect · Not assessed
Reported: 0.148 scalar
curvature_difference_correct_minus_incorrect · Not assessed
Reported: 0.03 scalar
curvature_difference_correct_minus_incorrect · Not assessed
Reported: 0.049 scalar
curvature_difference_correct_minus_incorrect · Not assessed
Reported: 0.035 scalar
curvature_difference_correct_minus_incorrect · Not assessed
Reported: 0.088 scalar
The learned classifier predicting the agent's next action achieves F1 scores of 0.826 for Read, 0.711 for Respond, 0.700 for Write, and 0.531 for Transfer.Experiment blocked; see the specific reasonReported 0.826 fraction
Reported
0.826 fraction
Observed
—
The paper does not provide the classifier model, training code, loss function, evaluation split, or feature representation.
f1_score · Not assessed
Reported: 0.826 fraction
f1_score · Not assessed
Reported: 0.711 fraction
f1_score · Not assessed
Reported: 0.7 fraction
f1_score · Not assessed
Reported: 0.531 fraction
Action-conditioned trigger policies exhibit domain-dependent performance: on Retail with Qwen3-14B, triggering on Read (0.393) and Write (0.381) outperforms Respond (0.290), whereas on Airline with Qwen3-14B, the ordering inverts, with Respond achieving 0.372 while Read achieves 0.352 and Write achieves 0.340.Experiment blocked; see the specific reasonReported 0.393 fraction
Reported
0.393 fraction
Observed
—
The learned action classifier (predicting next action a_hat_t) is not released, nor are the training data, features, architecture, or tau-Bench integration harness.
task_reward · Not assessed
Reported: 0.393 fraction
task_reward · Not assessed
Reported: 0.381 fraction
task_reward · Not assessed
Reported: 0.29 fraction
task_reward · Not assessed
Reported: 0.372 fraction
task_reward · Not assessed
Reported: 0.352 fraction
task_reward · Not assessed
Reported: 0.34 fraction
In the analysis of top-20 three-action chains, the information-acquisition chain Read->Read->Read (ID 1) accounts for 19.6%/22.2% (Correct/Incorrect, Rank #1) in Retail and 17.1%/14.5% (Rank #2) in Airline, while the repetitive output chain Resp->Resp->Resp (ID 2) accounts for 3.6%/8.2% (Rank #5) in Retail and 9.1%/23.6% (Rank #1) in Airline, showing strong error concentration in ungrounded conversational loops.Experiment blocked; see the specific reasonReported 19.6 pp
Reported
19.6 pp
Observed
—
Complete sliding window logs across episodes and domain trajectories are not available.
chain_window_share · Not assessed
Reported: 19.6 pp
chain_window_share · Not assessed
Reported: 22.2 pp
chain_window_share · Not assessed
Reported: 17.1 pp
chain_window_share · Not assessed
Reported: 14.5 pp
chain_window_share · Not assessed
Reported: 3.6 pp
chain_window_share · Not assessed
Reported: 8.2 pp
chain_window_share · Not assessed
Reported: 9.1 pp
chain_window_share · Not assessed
Reported: 23.6 pp
Correct episodes exhibit higher variance slopes beta_t than incorrect episodes across all six task-LLM pairings. Separability is statistically significant (p < .001) for Math (Llama-3.1-8B), Code (Qwen3-14B), Retail (Qwen3-14B), and Retail (Qwen3-32B), but not statistically significant for Airline (Qwen3-14B, p=0.30; Qwen3-32B, p=0.057).Experiment blocked; see the specific reasonReported 0.3 scalar
Reported
0.3 scalar
Observed
—
Raw episode traces, prompt sharding scripts, and hidden state trajectory extraction codebase are unreleased.
mann_whitney_u_p_value · Not assessed
Reported: 0.3 scalar
mann_whitney_u_p_value · Not assessed
Reported: 0.057 scalar
Across four domain-model settings on tau-Bench, geometry-conditioned adaptive reasoning triggers raise average task reward from 0.241 (24.1%) under Never-thinking to 0.396 (39.6%) while reducing mean token cost from 104.8k to 93.0k, an 11.2% reduction.Experiment blocked; see the specific reasonReported 0.241 fraction
Reported
0.241 fraction
Observed
—
The paper does not provide an official repository or executable code release (repository-identity.json indicates available: false). Furthermore, key protocol components including prompt templates, user simulator conversation driver scripts, probe extraction and logging pipelines, the exact classifier training split for Learned-HST, and detailed per-task configurations are deferred to an unreleased Appendix.
average_task_reward · Not assessed
Reported: 0.241 fraction
average_task_reward · Not assessed
Reported: 0.396 fraction
mean_token_cost_thousands · Not assessed
Reported: 104.8 thousands_tokens
mean_token_cost_thousands · Not assessed
Reported: 93 thousands_tokens
token_cost_reduction · Not assessed
Reported: 11.2 pp
Reproduction and technical details0
The experiment runs a repository and commit explicitly declared by the paper and verified by CiteArk.
No author code is used. CiteArk independently rebuilds the paper-grounded protocol and records every generated file and assumption.
A falsifiable execution still needs a missing method detail, input, split, metric, or evaluation rule.
Signed research planVerifiedThe Research Plan CAP fixes paper sources, complete Claim coverage, planned Experiments, and source-declared values.Download research plan
3b707a824b