https://github.com/xiangchensong/sae-feature-consistency

\n","updatedAt":"2025-05-27T02:50:40.061Z","author":{"_id":"64755a83e0b188d3cb2579d8","avatarUrl":"/avatars/2c50590905f4bd398a4c9991e1b4b5bb.svg","fullname":"Aashiq Muhamed","name":"aashiqmuhamed","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":2}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8806052803993225},"editors":["aashiqmuhamed"],"editorAvatarUrls":["/avatars/2c50590905f4bd398a4c9991e1b4b5bb.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2505.20254","authors":[{"_id":"683528109f968fc5c604495f","user":{"_id":"5f12485c0c833276f61f1afb","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1595033594228-noauth.jpeg","isPro":false,"fullname":"Xiangchen Song","user":"xiangchensong","type":"user"},"name":"Xiangchen Song","status":"extracted_pending","statusLastChangedAt":"2025-05-27T02:48:53.929Z","hidden":false},{"_id":"683528109f968fc5c6044960","user":{"_id":"64755a83e0b188d3cb2579d8","avatarUrl":"/avatars/2c50590905f4bd398a4c9991e1b4b5bb.svg","isPro":false,"fullname":"Aashiq Muhamed","user":"aashiqmuhamed","type":"user"},"name":"Aashiq Muhamed","status":"claimed_verified","statusLastChangedAt":"2025-05-27T07:51:25.328Z","hidden":false},{"_id":"683528109f968fc5c6044961","user":{"_id":"6804a894ea0734d6b26f19c3","avatarUrl":"/avatars/b3f5321366bbbb2fd91624289fbea958.svg","isPro":false,"fullname":"Yujia Zheng","user":"yujiazheng","type":"user"},"name":"Yujia Zheng","status":"admin_assigned","statusLastChangedAt":"2025-05-27T13:00:31.633Z","hidden":false},{"_id":"683528109f968fc5c6044962","user":{"_id":"6685f9e40284cbb0bcaef4f3","avatarUrl":"/avatars/475a708742e5a5198409f904b81153b1.svg","isPro":false,"fullname":"Lingjing Kong","user":"lkong","type":"user"},"name":"Lingjing Kong","status":"admin_assigned","statusLastChangedAt":"2025-05-27T13:00:37.345Z","hidden":false},{"_id":"683528109f968fc5c6044963","user":{"_id":"6553cd2296c5902fa386f57f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6553cd2296c5902fa386f57f/AEDbKbifdmDW4eJf6pUqw.png","isPro":false,"fullname":"Zeyu Tang","user":"zeyutang","type":"user"},"name":"Zeyu Tang","status":"admin_assigned","statusLastChangedAt":"2025-05-27T13:00:43.584Z","hidden":false},{"_id":"683528109f968fc5c6044964","name":"Mona T. Diab","hidden":false},{"_id":"683528109f968fc5c6044965","user":{"_id":"67c66c3febb87abbdf7af5ff","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/49hg7ZTN0Oupydi-BP4v-.png","isPro":false,"fullname":"Virginia Smith","user":"gingsmith","type":"user"},"name":"Virginia Smith","status":"admin_assigned","statusLastChangedAt":"2025-05-27T13:00:55.633Z","hidden":false},{"_id":"683528109f968fc5c6044966","name":"Kun Zhang","hidden":false}],"publishedAt":"2025-05-26T17:31:36.000Z","submittedOnDailyAt":"2025-05-27T01:20:40.055Z","title":"Position: Mechanistic Interpretability Should Prioritize Feature\n Consistency in SAEs","submittedOnDailyBy":{"_id":"64755a83e0b188d3cb2579d8","avatarUrl":"/avatars/2c50590905f4bd398a4c9991e1b4b5bb.svg","isPro":false,"fullname":"Aashiq Muhamed","user":"aashiqmuhamed","type":"user"},"summary":"Sparse Autoencoders (SAEs) are a prominent tool in mechanistic\ninterpretability (MI) for decomposing neural network activations into\ninterpretable features. However, the aspiration to identify a canonical set of\nfeatures is challenged by the observed inconsistency of learned SAE features\nacross different training runs, undermining the reliability and efficiency of\nMI research. This position paper argues that mechanistic interpretability\nshould prioritize feature consistency in SAEs -- the reliable convergence to\nequivalent feature sets across independent runs. We propose using the Pairwise\nDictionary Mean Correlation Coefficient (PW-MCC) as a practical metric to\noperationalize consistency and demonstrate that high levels are achievable\n(0.80 for TopK SAEs on LLM activations) with appropriate architectural choices.\nOur contributions include detailing the benefits of prioritizing consistency;\nproviding theoretical grounding and synthetic validation using a model\norganism, which verifies PW-MCC as a reliable proxy for ground-truth recovery;\nand extending these findings to real-world LLM data, where high feature\nconsistency strongly correlates with the semantic similarity of learned feature\nexplanations. We call for a community-wide shift towards systematically\nmeasuring feature consistency to foster robust cumulative progress in MI.","upvotes":5,"discussionId":"683528159f968fc5c6044aff","githubRepo":"https://github.com/xiangchensong/sae-feature-consistency","ai_summary":"Prioritizing feature consistency in sparse autoencoders improves mechanistic interpretability of neural networks by ensuring reliable and interpretable features.","ai_keywords":["Sparse Autoencoders (SAEs)","mechanistic interpretability (MI)","feature consistency","Pairwise Dictionary Mean Correlation Coefficient (PW-MCC)","TopK SAEs","LLM activations","synthetic validation","semantic similarity","learned feature explanations"],"githubStars":7},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64755a83e0b188d3cb2579d8","avatarUrl":"/avatars/2c50590905f4bd398a4c9991e1b4b5bb.svg","isPro":false,"fullname":"Aashiq Muhamed","user":"aashiqmuhamed","type":"user"},{"_id":"5f12485c0c833276f61f1afb","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1595033594228-noauth.jpeg","isPro":false,"fullname":"Xiangchen Song","user":"xiangchensong","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"62f7714247d782a6e2833ab8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/TtsMKiIqAwamRxwTZl9Xw.png","isPro":false,"fullname":"Christopher","user":"0x2A","type":"user"},{"_id":"6358edff3b3638bdac83f7ac","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1666772404424-noauth.jpeg","isPro":false,"fullname":"Pratyay Banerjee","user":"Neilblaze","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0}">

arxiv:2505.20254

Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs

Published on May 26

· Submitted by

Aashiq Muhamed on May 27

Upvote

Authors:

Xiangchen Song ,

Aashiq Muhamed ,

Yujia Zheng ,

Lingjing Kong ,

Zeyu Tang ,

Virginia Smith ,

Abstract

Prioritizing feature consistency in sparse autoencoders improves mechanistic interpretability of neural networks by ensuring reliable and interpretable features.

AI-generated summary

Sparse Autoencoders (SAEs) are a prominent tool in mechanistic interpretability (MI) for decomposing neural network activations into interpretable features. However, the aspiration to identify a canonical set of features is challenged by the observed inconsistency of learned SAE features across different training runs, undermining the reliability and efficiency of MI research. This position paper argues that mechanistic interpretability should prioritize feature consistency in SAEs -- the reliable convergence to equivalent feature sets across independent runs. We propose using the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC) as a practical metric to operationalize consistency and demonstrate that high levels are achievable (0.80 for TopK SAEs on LLM activations) with appropriate architectural choices. Our contributions include detailing the benefits of prioritizing consistency; providing theoretical grounding and synthetic validation using a model organism, which verifies PW-MCC as a reliable proxy for ground-truth recovery; and extending these findings to real-world LLM data, where high feature consistency strongly correlates with the semantic similarity of learned feature explanations. We call for a community-wide shift towards systematically measuring feature consistency to foster robust cumulative progress in MI.