The Agent Stability Index (ASI), Explained

Long-running agent systems rarely fail loudly. They degrade across dozens of small dimensions at once, and per-task metrics stay plausible while the trajectory bends away from intended behavior, the pattern this site calls behavioral state decay. The measurement problem is that no single signal captures this: response quality, tool usage, and coordination each decay separately. The Agent Stability Index (ASI) is a proposed answer: one composite score designed to make drift visible before it becomes an incident.

The method

ASI was introduced in a January 2026 preprint on agent drift (Rath, arXiv:2601.04170). It is a weighted composite over four categories; the paper lists three sub-metrics per category, twelve dimensions in total, normalized to a 0 to 1 scale. One caveat before treating it as fully specified: the paper's printed equation appears inconsistent with its own list, omitting the human-intervention term from the behavioral numerator while still dividing by three. Any repaired formula is an adaptation pending author clarification, not the paper's own. The categories:

• Response consistency, weight 0.30: stability of outputs on comparable inputs over time.
• Tool usage patterns, weight 0.25: including tool selection stability and tool sequencing consistency.
• Inter-agent coordination, weight 0.25: agreement rates and consensus health in multi-agent settings.
• Behavioral boundaries, weight 0.20: in the source, output-length variation, emerging error patterns, and human-intervention rate. Direct action-space or policy-violation tracking is not among the paper's metrics; adding it is a proposed adaptation, not part of the source framework.

The composite is ASI = 0.30 times response consistency plus 0.25 times tool usage plus 0.25 times coordination plus 0.20 times behavioral boundaries. The paper evaluates it over 50-interaction windows and flags drift when ASI stays below 0.75 for three consecutive windows, a design that trades detection speed for resistance to single-window noise.

The paper's evidence is simulation-based: modeled multi-agent workflows across three enterprise domains, not production deployments. Within those simulations, under its assumptions, it projects a 42.0% reduction in task success rate for unchecked drift, human interventions rising from 0.31 to 0.98 per task, and detectable drift after a median of 73 interactions. Note that the 73-interaction result uses a looser criterion, ASI below 0.85, than the formal alarm rule above; the paper's early-onset observation and its alarm threshold are two different rules. It also projects that combined mitigations (episodic memory consolidation, drift-aware routing, adaptive behavioral anchoring) reduce drift-related errors by 81.5% at about 23% added computational overhead. All of these figures are projections from the paper's models and should be quoted as such.

Implementing an ASI-like metric

You do not need the paper's exact twelve dimensions; you need the shape. A minimal adaptation:

• Step 1: choose two or three sub-metrics per category that your telemetry already supports. Example: embedding distance of responses on recurring task types (consistency), edit distance between tool-call traces on standardized tasks (tool usage), schema-mismatch count per handoff (coordination), out-of-policy action attempts caught by the harness (boundaries).
• Step 2: normalize each sub-metric to 0 to 1 against a baseline window captured when the system is known-good, for example baseline mean mapping to 1.0 and a defined failure level to 0. The paper does not supply reproducible transforms, and its raw metrics run in opposite directions (similarity and agreement rise when healthy; divergence, distance, variation, and intervention counts rise when unhealthy), so document direction, transform, clipping, missing-data handling, and uncertainty per metric before combining anything.
• Step 3: compute the weighted sum per evaluation window. Start with the paper's weights, then reweight when your incident history shows which category actually predicts your failures.
• Step 4: alert on sustained depression (several consecutive windows below threshold), not single dips.
• Step 5: on alert, trigger a corrective routine such as re-anchoring or a fresh session seeded from externalized state, and log the ASI trajectory around the event for later tuning.

Pair the score with a fixed replayed task suite, per drift regression tests, so that at least part of the input distribution is held constant while state ages.

Pitfalls

• Treating projections as benchmarks. The 42% and 81.5% figures describe the paper's simulations. Your production numbers will differ; the framework transfers, the constants do not.
• Unstable baselines. If the baseline window itself contains early drift, every later score is graded against a degraded reference. Capture baselines immediately after a known-good validation, and version them.
• Weight cargo-culting. The 0.30/0.25/0.25/0.20 split encodes the paper's assumptions. A single-agent deployment, for instance, has no coordination category at all and must redistribute that weight.
• Gaming by averaging. A composite can hide one collapsing dimension behind three healthy ones. Alert on individual sub-metrics as well as the composite.
• Measuring fresh state. Computing ASI on newly started sessions tells you about the model, not about decay. The score is only meaningful when computed against the agent's aged, accumulated state.

Checklist

• Baseline window captured on a validated system, stored and versioned.
• Two or more sub-metrics per applicable category, computed from real telemetry.
• Normalization bounds documented for every sub-metric.
• Windowed evaluation with a sustained-depression alert rule, not single-sample alerts.
• Sub-metric level alerting alongside the composite.
• A defined corrective action wired to the alert, with its trigger logged.
• All quoted effect sizes attributed to arXiv:2601.04170 as simulation-based projections.