TEMPO: The gradient alignment claim has a counterexample

Zhang & Stadie (2026) · arXiv:2605.18843 (v1) · summary on the ideas page

Proposition 1 asserts that the leakage-mode gradient descends the leakage cost. Its proof changes the weighting distribution without changing the centering means. Two independent witnesses make the inner product positive.

The claim

Proposition 1 (Appendix A.1) states that whenever Var_π(c) > 0, ⟨ḡ_leak, ∇V_c⟩ ≤ −γ(θ) ‖∇V_c‖² with γ(θ) > 0, so the exponential cleanliness reward is a provable descent direction for expected leakage cost. Proposition 2, Theorem 1 and Corollary 1 all invoke it.

The proof derives equation (8) correctly. Writing f = exp(−0.5c) and taking both means under π:

S = sum_y pi(y)^2 ( f(y) - fbar ) ( c(y) - cbar ) (8)

It then reads S as Cov_Q(f, c) under the tilted law Q(y) ∝ π(y)², and signs it with the Chebyshev covariance inequality. A covariance under Q must centre under Q. Equation (8) centres under π. The two differ by a term the proof drops:

S = Z * [ Cov_Q(f,c) + ( E_Q f - E_pi f )( E_Q c - E_pi c ) ], Z = sum_y pi(y)^2 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ dropped

Chebyshev signs the first term. It says nothing about the second, which can be positive and larger. Two witnesses, found independently:

c = (0, 1, 100) pi = (0.15, 0.83, 0.02) inner product = +0.011648 c = (0, 5, 39) pi = (0.08, 0.82, 0.10) inner product = +0.042248 second witness in detail: Var_pi(c) = 108.60 > 0, the stated hypothesis Cov_Q(f, c) = -0.086443 negative, as Chebyshev gives dropped term = +0.101790 S = +0.042248 ||grad V_c||^2 = 16.0712 implied gamma = -0.002629 must be > 0

Inequality (5) requires the inner product to be strictly negative. At these policies it is positive, so the leakage-mode gradient ascends the leakage cost. Taking a step of size 10⁻⁴ along the reward gradient at the first witness raises expected cost from 2.830000 to 2.830001 while also raising the reward, which is the behaviour the proposition rules out.

The finite-sample gate

The mode gate switches to performance mode when a sampled group is clean. That is a statement about a sample, not about the policy. With true leakage probability 1%, twelve independent samples are all clean with probability 0.99¹² ≈ 88.6%, so a clean batch is weak evidence of a leak-free policy.

What survives

The empirical results are unaffected, and the reframing from unlearning to instance-conditioned selection stands on its own. What fails is the claim that the method provably converges.

Proof error

Appendix A.1's gradient alignment claim is false. An explicit three-outcome counterexample raises the reward and the expected leakage cost together, and the implied alignment constant is negative. The sampled gate does not enforce zero policy leakage either. Empirical performance has to be assessed separately from the claimed guarantee.

Appendix A.1. Inner product recomputed directly from the score functions s(y) = e_y − π, and the two witnesses cross-checked against each other. Numerical checks are reproducible with reviews/counterexamples.py (standard library only). These are counterexamples to particular claims; they do not reproduce the papers' training experiments.

← Temporal Critique Fine-Tuning · The Memorization Problem →