The Geometry of Forgetting: Orthogonality does not prove impossibility

Elbadry, Heakl, Zhang, Bouch et al. (2026) · arXiv:2605.09195 (v1) · summary on the ideas page

The paper infers that uncertainty-based detectors must be blind to staleness from near-orthogonal probe directions and the failure of the detectors it tested. Near-orthogonality does not support a universal impossibility claim.

The claim

The abstract and discussion hold that because staleness is encoded orthogonally to correctness and uncertainty, detectors built on those signals are blind to it by construction. The paper runs several tests, including score correlations and projection interventions, so the objection is not that it merely inspected weight angles.

Let U be uniform on {−2, −1, 1, 2}, let the representation be h = (U, U²), and read out along (1,0) and (0,1). The directions are orthogonal and the two scores have zero covariance:

U = [-2, -1, 1, 2] score 1 U^2 = [ 4, 1, 1, 4] score 2 Cov(U, U^2) = 0.0 drift label D = 1[U^2 > 2] = [1, 0, 0, 1] predicted by |U| > 1.5 = [1, 0, 0, 1] accuracy 1.00

The second score is an exact function of the first, and the drift label is perfectly predictable from the first score by a nonlinear rule. Orthogonal directions, zero covariance, complete determination.

More generally Cov(w^⊤h, v^⊤h) = w^⊤Cov(h)v, so Euclidean orthogonality does not even give zero covariance for an arbitrary representation. Zero covariance, where it does hold, still does not give independence.

What survives

The experiments support the narrower claim that the detectors the authors evaluated perform poorly, and that drift-trained probes do better. That is a useful empirical result. It does not rule out every detector built on correctness or uncertainty signals.

Invalid inference

The impossibility claim is unsupported. Near-orthogonal probe directions, low correlations, and the failure of selected detectors do not imply that every uncertainty-based detector must fail. The probe results stand as empirical evidence, not as a structural impossibility theorem.

Abstract and §5. Counterexample computed directly. Numerical checks are reproducible with reviews/counterexamples.py (standard library only). These are counterexamples to particular claims; they do not reproduce the papers' training experiments.

← Look-Ahead-Freedom as Temporal Non-Interference · Temporal Critique Fine-Tuning →