arXiv cs.CLPaper
When Decodability Is Not Enough: Logical Validity Representations, Behavioral Dissociation, and Causal Tests in Language Models
This is important: a model can internally represent something correctly without using it. Decoding validity from layers doesn't mean the model reasons about it. For interpretability researchers this is a humbling result. For builders it's a reminder that probes and saliency maps can mislead you about what a model actually does. If you're relying on mechanistic understanding to trust model behavior, this should make you skeptical.