j⧉nus @repligate · 04 Sep 2025
@repligate KV caching doesn’t provide anything you wouldn’t get by recalculating attention after every token. It only speeds up the calc by not repeating work that you’ve already done. So it isn’t allowing introspection of past computations beyond what is inherent in autoregression.
961Views
12Likes
1Reposts
2Replies
0Quotes
1Bookmarks
Is that a lot?
0.19×vs this author's median4 944 views is typical
0Percentile for this authorof 6 recent posts
0.52×vs 10K–100K median1 856 views is typical
1.41%Reachviews ÷ followers
1.56%Engagement rateof viewers reacted