Emerging User Intent Detection From Agent Session Traces

Production agent behavior improves through a compounding loop of trace analysis, harness changes, and replay validation. Emerging user intent detection from agent session traces makes that loop concrete by reconstructing what users needed across the execution trajectory, locating where the harness lost that intent, and testing a targeted fix against historical sessions.

Why the opening prompt is an unreliable picture of what a user actually needs

Users rarely state a complete goal at the start of a session. They disclose missing details, revise earlier constraints, and switch tasks while retaining some prior context. An underspecified request can therefore lead to an initial action, a correction, and then a pivot toward a different goal. The agent must distinguish what remains valid from what the user has replaced. Taking the opening prompt as the final word on the task hides that distinction.

Microsoft Research's evolving-intent study shows that even frontier models degrade substantially as intent shifts across turns, dropping to 80.5% on math tasks after six intent transitions. A static, single-turn evaluation leaves out the transitions the agent must handle, so it cannot show that failure. Engineers therefore need to check both the agent's understanding of the first request and the alignment of its actions as the request changed. Its workflow, memory, and prompt structure carry each update into execution or leave it in the dialogue history.

What the full execution trajectory actually contains as intent evidence

A production trace is a structured record of provenance, connecting each action to the information available when it occurred. LLM calls, tool invocations, retrieval steps, memory reads and writes, and planning decisions each carry evidence about the task being pursued. Retrieval shows whether the agent fetched material relevant to the user's need at that turn. Tool arguments show how it translated that need into an operation. Whether the tool output was used or discarded shows whether the resulting evidence influenced execution.

Memory lineage shows which earlier statements shaped later decisions and whether those statements had become stale. Planning decisions show where the agent recognized a mismatch and changed course. Hesitation, including a step with no tool call or a repeated query, can signal that the agent no longer knows how to advance the task. None of these signals establishes intent on its own. Their value comes from chaining them back to user turns and forward to subsequent actions.

Final-answer accuracy cannot explain those links or show where they broke. Engineers must reconstruct the provenance chain to distinguish a justified action from one that merely produced a plausible answer. That reconstruction is labor-intensive when each retrieval, memory read, and planning step must be assembled from raw logs. Platforms such as Moda can automatically parse production traces to show which signal nodes dropped intent evidence. This supports the same diagnostic work rather than replacing the need to interpret the evidence.

How intent drifts inside a session and what the trace record looks like when it does

With incremental disclosure, early tool calls may target the wrong scope because a necessary detail has not yet been supplied. The relevant evidence is the gap between those arguments and the user's later clarification. Revision creates a different test because an earlier statement directly conflicts with a later one. If memory keeps supplying the earlier statement, the actions that follow show that the intent state is stale.

Task-switching changes the active goal while preserving only the context that still applies. The IDSS paper's travel-agent scenario moves from flight to hotel to attraction, with a weather constraint forcing replanning. This sequence requires more than remembering the conversation because the active task and its dependencies change. A tool return that contradicts an assumed fact must also update what the agent can execute.

A replanning step after such a correction is evidence that the workflow detected a conflict. By contrast, continued execution under the old constraint indicates that the new fact did not reach the task state. Stale memory contradicting later turns, corrected tool parameters, and goal resets are diagnosable drift markers. When session-history tools such as Moda highlight these markers automatically, engineers can focus on the signal node where continuity broke. The distinction matters because a changed goal is normal, while failure to carry that change into execution is not.

Why agents lose track of evolving intent: the harness layers where signal breaks down

The prompt layer loses intent when it contains no explicit representation of the current task state. The agent must then infer active goals and constraints from raw dialogue at each step, leaving room for inconsistent interpretations. In the trace, the relevant pattern is repeated reconstruction of intent without a stable account of what remains active. IDSS research explains how compression and summarization reduce context pressure while leaving grounded facts and task-state judgments to be inferred from compressed history.

The memory layer fails differently because it can preserve an instruction accurately after that instruction has ceased to apply. Without versioning and conflict detection, an early memory item remains available even after a later turn revises it. The trace then connects a superseded statement to a downstream action. At the tool layer, arguments that lag the latest user constraint show that intent has not propagated into dispatch. Either failure can lead to an operation that looks right but uses the wrong task state.

Workflow structure determines what happens when that state cannot be resolved. Without an exit condition, the agent can loop, and without fallback handling, a tool error can make it stop before asking for clarification again. Without format specifications, the agent also cannot handle task changes in a consistent way. In multi-agent pipelines, natural-language delegation adds another point of loss because the receiving agent must reconstruct intent from ambiguous history without a deterministic state contract. These are correctable harness failures, so diagnosis should locate the broken transfer before treating the outcome as a model-capability problem.

Reading intent signals from traces: what to look for and how to interpret it

Begin with the observed failure and work backward to the first divergence from the user's actual need. A wrong final answer is a symptom. Retrieval, tool misuse, stale memory, unsafe instructions, inter-agent miscommunication, and invalid intermediate reasoning can all produce that symptom. AgentRx's 2026 analysis found a 75% average improvement in step localization over prior work, underscoring the importance of identifying the critical failure step. For intent diagnosis, that step is where the agent's task representation first departs from the evidence available to it.

In a trace viewer, search for the latest stated constraint and compare it with the next tool call's arguments. If they disagree, follow the inputs to that call backward through the prompt, memory reads, and planning decisions. Then inspect retrieval targets for a scope that still matches a superseded task. Search memory reads for statements that later user turns revised, and check whether those items were marked as superseded.

Next, check tool returns that changed a key fact and look for a replanning step that followed that change. Its presence can show correct conflict detection, while its absence warrants checking whether the new fact reached the task constraints. Repeated calls with identical or near-identical arguments are another useful search target, especially when intervening turns supplied new information. Such repetition is a symptom that requires further investigation to establish intent loss. IntPro's distinction between immediate context and accumulated interaction history provides a useful interpretive check because either source alone can mislead. Its retrieval-conditioned approach stores intent explanations so that patterns from past context can help the agent infer intent now while still using the current turn.

How do teams turn real agent failures into systematic prompt and logic improvements

A trace diagnosis becomes actionable when it specifies the failed layer, the proposed change, and the evidence that would demonstrate repair. If traces show repeated reconstruction of intent from raw history, maintain a structured intent state in the prompt. That state should show the active goals, completed goals, open variables, and violated constraints. Verify the change by checking whether those fields remain accurate through the transitions that previously caused divergence. IDSS demonstrates the supporting mechanism by parsing tool returns into provenance-aware entities and propagating new facts to task constraints.

When a stale memory item leads to a wrong action, add versioned writes with conflict detection. A user turn that revises a prior statement should trigger an update that marks the old item as superseded. Replay should then show that later reads and actions follow the revised statement. When the memory is current but dispatched arguments still lag, the trace instead points to tool selection or parameter propagation. Apply schema enforcement at dispatch and update parameters after each turn that revises intent, then check that later calls follow the active constraints.

When unresolved intent causes loops, the agent needs clear exit and fallback conditions so that it can stop trying the same operation. Structured termination rules should apply when the tools at hand cannot resolve the current intent state. Verification should establish that the replay reaches the intended exit or fallback. If a handoff corrupts constraints, replace natural-language delegation with a deterministic state contract carrying current intent explicitly. The corresponding check is whether the receiving agent preserves that state instead of reconstructing a conflicting task from history.

TensorOpera's super-agent design uses intent detection to send requests to task agents or workflows that specialize in them. For long-horizon sessions, the improvement implied by drift traces is to apply intent-aware routing at each replanning step. Verification should focus on whether a goal reset leads to routing consistent with the revised task. These changes fix prompt and execution logic without the need for model retraining.

Validating intent-tracking improvements before shipping them to production

A fix can solve the incident that led to it and still break sessions that worked before. Validation must compare intent-drift traces whose failures have been diagnosed with a set of traces where intent tracking was correct. The former become positive test cases, while the latter establish the regression baseline. Replay both sets against the proposed harness and score whether the intent state stays accurate from each turn to the next. A better final answer alone does not establish that the change repaired the intended failure.

Block a change that degrades accuracy on the regression set even if it improves the failure set. This protects correct handling of disclosure, revision, and task-switching from fixes tailored too narrowly to one incident. Historical replay grounds the decision in actual execution evidence rather than the apparent reasonableness of a revised prompt.

Moda operationalizes this workflow by ingesting production traces, attributing intent-tracking failures to harness layers, generating reviewable improvements, and validating them against historical traces before recommending what to ship. Whether teams assemble the process themselves or use a trace-driven platform, the governing discipline is the same. Each diagnosed production incident should become a structured evaluation case in the regression suite. Future changes then face an accumulating library of real intent failures alongside synthetic scenarios.