A new technique rotates Transformer LLM internal coordinate systems into a canonical basis using orthogonal matrices from weight singular vectors, preserving outputs while revealing hidden geometric structures. This makes every hidden axis independently measurable and controllable, uncovering bipolar oscillators, rhythmic layer patterns, and homeostatic defenses. The approach provides a standardized lens for mapping how models perform reasoning and can expose low effective rank in correlation structures.
Background
LLM interpretability research often struggles with opaque internal mechanisms; this work introduces a mathematically grounded transformation to align model axes with observable geometric structures.
- Source
- Lobsters
- Published
- Aug 30, 2026 at 04:16 AM
- Score
- 8.0 / 10