Appearance

Hebbian Surgery ​

All basin surgery operations use Hebbian and anti-Hebbian learning rules — rank-1 perturbations of the weight matrix W.

Basin surgery operations

Hebbian Learning ​

The classic Hebbian rule: "neurons that fire together wire together." In matrix form:

ΔW = α · outer(pattern, f(pattern))

This strengthens the association at pattern, making it a deeper basin in the energy landscape.

Anti-Hebbian Learning ​

The inverse: weaken an association.

ΔW = -α · outer(pattern, f(pattern))

This raises the energy at pattern, reducing or eliminating the basin.

Strength Normalization ​

The strength parameter α is normalized relative to the existing weight matrix:

α = strength · ‖W‖_F / ‖outer(pattern, f(pattern))‖_F

This makes surgery scale-preserving — a strength of 0.1 means approximately 0.1% change to the Frobenius norm of W, regardless of the model size or layer.

Operations ​

Inject ​

python
W_new = W + α · outer(target, f(target))

Creates or deepens a basin at target. The target vector is first converged to find the natural basin direction, then the outer product update is applied.

Remove ​

python
W_new = W - α · outer(target, f(target))

Weakens or eliminates the basin nearest to target.

Move ​

python
W_tmp = W - α · outer(source, f(source))    # remove at source
W_new = W_tmp + α · outer(dest, f(dest))    # inject at destination

Combines remove and inject in sequence.

Strengthen / Weaken ​

Same as inject/remove but applied to an existing basin with a scaling factor.

Verification ​

After every surgery, verification checks that the edit worked:

  1. Take the target state
  2. Add 10% Gaussian noise
  3. Run convergence dynamics
  4. Check if cosine similarity to original target > 0.9

If yes: the basin exists (surgery succeeded). If no: the basin was not created or was destroyed (adjust strength).

Why Rank-1? ​

A rank-1 update is the minimal perturbation that can create or destroy a single basin. Higher-rank updates would affect multiple basins simultaneously, making surgery imprecise. The Hebbian outer product targets exactly one pattern while minimally disturbing the rest of the landscape.