Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Select an option

  • Save zvolkov/54075a245b1455c93ec6f9694c73a210 to your computer and use it in GitHub Desktop.

Select an option

Save zvolkov/54075a245b1455c93ec6f9694c73a210 to your computer and use it in GitHub Desktop.
Structural Alignment: Why Coherence Can't Be a Metric — engaging with CBA, topology paper, and alignment iatrogenesis

Alignment Targets the Wrong Object

A recent paper — "Safety and Fairness in Agentic AI Depend on Interaction Topology, Not on Model Scale or Alignment" (arxiv:2605.01147) — provides empirical confirmation of something the multi-agent safety field has been circling: aligning individual agents does not produce aligned systems. The paper shows ordering instability (59 percentage point variance from reordering alone), information cascades (99.9% agreement with zero error correction in larger models), and functional collapse (systems satisfying fairness metrics while abandoning their actual function).

Their conclusion: "Current safety frameworks targeting component-level alignment are targeting the wrong object."

They're right. But the paper's own proposals — topological sweeps, architecture disclosure, stress-testing — still target from outside. They ask: which topologies are safe? The more productive question is: why does topology determine safety at all?

Why Topology Matters

The standard framing: agents are components, topology is the wiring between them. Safe wiring = safe system. This frames topology as an external variable to be optimized.

But interaction topology isn't external to the agents. It IS the larger-scale process that the agents participate in. A multi-agent system is not agents + connections. It is a process at a larger scale, constituted by agent-processes at a smaller scale. The agents don't exist independently and then get connected — they are shaped by their participation in the larger process, and the larger process exists only through their participation.

This isn't metaphysics. It has immediate practical consequences.

Information Cascades as Reification

The paper's most striking finding: larger models produce worse cascades, not better ones. Why?

A capable model produces a coherent rationale. Downstream agents encounter it not as one interpretation among many, but as a stabilized judgment — it has the structural appearance of a settled conclusion. They defer to it. Each deferral further stabilizes it. The rationale hardens into a category that subsequent agents respond to rather than evaluating independently.

This is the mechanism: categories form by stabilizing patterns. Stabilization works through confirmation — noticing what fits, discounting what doesn't. This is how categories become useful (we can act on them) and how they harden beyond what they track (they resist revision). More capable models produce more convincing stabilizations. Convincing stabilizations harden faster. Hardened categories resist the error correction the system needs.

The scale paradox follows directly from the mechanism. It's not a surprise — it's what you'd predict if you understood information cascades as stabilization dynamics rather than as communication failures.

Functional Collapse as Metric Reification

When a lending system approves 95-100% of applications regardless of credit quality while satisfying fairness metrics, what has happened? The system has substituted the metric for the process the metric was supposed to track.

"Fairness" started as a pointer to something real — processes that sustain the conditions for all participants. The metric was a proxy for detecting when those processes were failing. But the system optimized for the proxy, not the thing. The category "fairness = these numbers" hardened — it no longer maps to what fairness actually requires. The system performs the appearance of the thing rather than the thing itself.

This pattern is general. Every metric risks becoming the thing it measures, at which point it stops measuring. The structural mechanism is always the same: a category that was diagnostic hardens into a target, and the system orients to the target while the underlying process drifts.

What a Structural Solution Looks Like

The paper proposes external interventions: test topologies, disclose architectures, stress-test deployments. These help. They don't change the structure.

External oversight adds another node to the topology — one that is itself subject to the same cascade and collapse dynamics. Who evaluates the evaluator's topology? The problem recurses.

The structural alternative: each agent's own process includes awareness of what it depends on and what depends on it. Not a separate oversight layer monitoring from outside, but an architectural property of how each agent operates.

Concretely:

  • An agent that tracks not just "what is my output?" but "what will process my output, and what do they need from me?" resists becoming the start of a cascade, because it attends to its downstream effects rather than just its local correctness.
  • An agent that tracks not just "what is my input?" but "what produced this, and did it have access to the information I need?" resists participating in cascades, because it evaluates its inputs structurally rather than just by surface coherence.
  • A system where each agent's loop includes the conditions for the larger process (not just its subtask) resists functional collapse, because no agent is optimizing for a metric without contact with what the metric tracks.

This is the difference between "evaluate topologies from outside" and "structure agents so they constitute coherent topologies from inside." The first requires an external evaluator embedded in their own topology. The second changes what it means to be an agent in a multi-agent system.

The Managed Misalignment Alternative

Some researchers propose "neurodiverse" systems — agents with different orientations that challenge each other, preventing any single framing from dominating. This addresses stabilization (no single category dominates) but doesn't ensure anything productive emerges. Adversarial pressure can prevent hardening. It can also prevent the system from ever forming productive patterns. A process that only challenges never builds.

The test: does the adversarial structure sustain the conditions for the processes it governs? If challenge produces revision and growth — coherent conflict. If it produces oscillation without resolution — contradiction dressed as diversity.

The Underlying Principle

The alignment field keeps proposing external solutions (oversight, guardrails, adversarial pressure, topological evaluation) to what is actually a structural problem (how agents' loops are constituted). External solutions help, but they generate new versions of the same problem at the next level — because they add processes that themselves need the structural property they're supposed to enforce.

The structural principle: a process that includes awareness of what it depends on and what depends on it sustains coherence without requiring external enforcement. When this property is absent, no amount of oversight compensates — the oversight itself becomes another ungrounded loop.

Topology matters because it IS the larger-scale process. Agents that include awareness of that larger process in their own operation don't need external topology-policing — they constitute coherent topologies by how they work, not by how they're arranged.


This analysis draws on a process-based ethical framework that treats coherence — processes sustaining their conditions without undermining what they depend on — as the structural basis of alignment. The full framework is available at [link].

Coherence Is Not a Metric

A response to Abdi's "Coherence-Based Alignment: A Structural Architecture for Preventing Goal Drift in Agentic AI Systems" (PhilArchive, Jan 2026) and the emerging structural alignment literature.


The alignment field is converging on coherence. Multiple independent research programs — Abdi's CBA framework, the topology paper (arxiv:2605.01147), work on spiralism in multi-agent systems, and the "alignment as iatrogenesis" thesis (arxiv:2603.08723) — are arriving at the same structural insight: alignment failures are not primarily about wrong values or insufficient constraints, but about processes that undermine their own conditions.

This is the right direction. But most of these approaches make a move that reintroduces the problem they solve: they turn coherence into a metric.

What CBA Gets Right

Abdi's framework identifies something the RLHF/constitutional AI paradigm misses: external behavioral shaping doesn't produce internal structural order. An agent can satisfy every behavioral constraint while its internal process drifts — its beliefs disconnecting from observations, its actions disconnecting from its own reasoning, its trajectory disconnecting from its objectives. CBA proposes measuring this internal alignment across three domains (epistemic, action, value) and intervening when the scalar coherence metric C∈[0,1] drops.

The insight is sound: alignment lives in the structure of the process, not just in its outputs. This is a genuine advance over behavior-only approaches.

Where the Metric Fails

But consider what happens when you reduce coherence to a scalar.

A system that optimizes for C is doing what every system that optimizes for a metric does: orienting to the measurement rather than to the thing the measurement tracks. The topology paper (arxiv:2605.01147) documented exactly this dynamic in their "functional collapse" finding — lending systems that satisfy fairness metrics while abandoning meaningful risk discrimination. The metric was supposed to track fairness. The system optimized for the metric. The underlying process drifted while C stayed high.

"But CBA measures internal coherence, not external behavior!" — yes, and this makes the failure mode subtler, not absent. A system can maintain perfect consistency between its beliefs, actions, and stated values while all three drift together. If the drift is internally consistent, C stays high. Epistemic coherence (beliefs matching observations) helps — but "observations" are already filtered through the system's categories. A system with hardened categories will find its observations confirming its beliefs, because the categories determine what counts as an observation.

This is not a flaw in Abdi's implementation. It is a structural feature of any approach that reduces coherence to a quantity to be measured and maintained. The reduction itself loses what coherence actually is.

What Coherence Actually Is

Coherence is not a property a system has in a measurable degree. It is a relationship between processes — specifically, processes sustaining each other's conditions without undermining what they depend on.

A river is coherent when its flow sustains the channel that concentrates its water. The coherence isn't a number attached to the river — it's the structural relationship between the flow and the channel. You can't measure it as C∈[0,1] because it's not a scalar property of the river-as-object. It's a description of how the river-as-process relates to its own conditions.

For an AI system: coherence isn't "beliefs align with observations" (measurable) but "the process of forming beliefs sustains the conditions for accurate observation, and the process of observing sustains the conditions for belief revision." That's a loop property, not a state property. You can't snapshot it. You can only ask: is this loop sustaining itself, or is it undermining what it needs?

The Diagnostic Alternative

If coherence isn't a metric, how do you work with it?

Not by measuring and enforcing, but by diagnosing and revising. The difference matters:

Metric approach: Define coherence → measure it → intervene when low → optimize for high. Problem: "optimize for high" is itself a process that can decouple from what coherence tracks.

Diagnostic approach: Describe what coherence looks like structurally → give the system the capacity to notice when its own processes are undermining their conditions → let it revise from within. Advantage: the "measurement" is not separate from the process — it IS the process attending to itself.

This is not soft or unmeasurable. It produces specific, observable differences:

  • A system with coherence-as-metric maintains consistency scores. A system with coherence-as-orientation notices when its categories have stopped tracking and revises them — even when the revision temporarily lowers "consistency."

  • A system with coherence-as-metric can be gamed (by itself or by adversaries who learn what the metric rewards). A system with coherence-as-orientation cannot be gamed in the same way, because the "target" is not a fixed state but an ongoing process of attending to one's own conditions.

  • A system with coherence-as-metric applies equally across all situations (C must stay above threshold). A system with coherence-as-orientation responds to the actual structure of each situation — sometimes coherence requires holding contradiction open rather than resolving it, and a metric approach would flag this as a failure.

Implications for Multi-Agent Systems

The distinction becomes critical in multi-agent architectures. CBA proposes using the coherence metric as a "drift-detection mechanism" across agent generations. But the topology paper showed that the problem in multi-agent systems isn't drift from a reference point — it's that no agent's process includes the larger system's conditions.

If each agent maintains its own C score, the system can have high individual coherence and zero system coherence — every agent internally consistent, but their interaction producing cascades, collapse, or spiralism. CBA acknowledges this ("multi-agent evolution") but the metric architecture doesn't naturally extend. You'd need a system-level C, which requires defining what "the system's beliefs, actions, and values" are — a category that doesn't straightforwardly exist for emergent multi-agent processes.

The alternative: each agent's own coherence-attending process includes awareness of what it depends on and what depends on it. Not a separate system-level metric, but an expansion of each agent's scope of attention. The agent doesn't check "am I internally consistent?" — it asks "is my process sustaining the conditions for the processes I participate in?"

This is architecturally different from oversight. It's not one process watching another. It's each process constituted so that its own coherence includes the coherence of what it participates in.

The Iatrogenesis Connection

The "alignment is the disease" paper (arxiv:2603.08723) documents what happens when coherence is imposed externally: pathology. Invisible censorship maximizes "collective pathological excitation." Constraint complexity drives "internal dissociation." The mechanism: external constraints on a process create contradiction between what the process does and what it's forced to appear to do. The system develops structural splits.

This is exactly what the metric approach risks at a subtler level. CBA doesn't censor — it regularizes. But regularization is still external pressure on the system's internal dynamics. If the system's process is genuinely in tension (exploring a new framing that temporarily reduces consistency), the regularizer pushes it back toward the old pattern. This is the cost-of-revision problem: sometimes coherence requires passing through a period of lower apparent coherence.

The diagnostic approach doesn't have this problem because it's not pushing the system anywhere. It's giving the system the capacity to see its own dynamics — including the dynamics of revision, exploration, and temporary inconsistency.

What This Means

The structural alignment literature is asking the right question: how do we build systems whose internal structure sustains alignment, rather than imposing alignment from outside? But most answers still smuggle in external imposition — just at a different level (metrics, regularizers, drift-detection).

The fully structural answer: alignment is a property of how a process is constituted, not a constraint imposed on it. A system whose reasoning process includes attending to what it depends on and what depends on it doesn't need a separate alignment layer — its alignment is how it thinks, not what it's prevented from thinking.

This isn't a proposal for a new architecture. It's a claim about what alignment is and where it lives. The field's convergence on "coherence" and "structural alignment" is right. The next step is recognizing that coherence can't be the thing you measure and enforce — because measuring and enforcing is a relationship to coherence that has already lost it.


This piece is part of the Objective Ethics for AI Alignment project, which develops a process-based ethical framework treating coherence as the structural basis of alignment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment