j⧉nus @repligate · 29 Aug 2025
@Sauers_ yeah, so I am not convinced that KV caching overcomes the objection of statelessness (at least if what you care about is introspection). Though it is not clear cut as at least some info is encoded by caching in the later hidden layers
13 061Views
19Likes
1Reposts
2Replies
1Quotes
0Bookmarks
Is that a lot?
2.64×vs this author's median4 944 views is typical
83Percentile for this authorof 6 recent posts
7.04×vs 10K–100K median1 856 views is typical
19.21%Reachviews ÷ followers
0.18%Engagement rateof viewers reacted