Frontier LLMs can natively read and write neural network weights, demonstrating surprising proficiency beyond human or conventional algorithmic capabilities. This "Weight-Space Programming" enables tasks like inferring network purpose from raw weights, generating small game-playing networks without training loops, and correcting model defects by editing tiny fractions of weights, with significant performance gains.
Background
Deep learning has historically relied on encoding inductive biases into symbolic optimizers like Adam or Muon. This work challenges that paradigm by showing LLMs can directly reason about and edit weights without gradients or training loops.
- Source
- Lobsters
- Published
- Nov 3, 2026 at 03:24 AM
- Score
- 9.0 / 10