Amiri Hayes @amirihayes_ · 29 Jun 2026
What if attention were code? We show that many attention heads in transformer LMs can be replaced by human-readable Python programs. Swap them in and the model barely notices. See our experiments here: Explaining Attention with Program Synthesis [https://t.co/tkFopEYtaV]
161 621Views
993Likes
113Reposts
28Replies
18Quotes
849Bookmarks
Is that a lot?
29.5×vs this author's median5 474 views is typical
75Percentile for this authorof 4 recent posts
38.8×vs under 10K median4 164 views is typical
565.1× audienceReachviews ÷ followers
0.71%Engagement rateof viewers reacted