TH

Ashwin Gopinath ✓

@ashwingop

CEO of Sentra; Used to be a Prof. at MIT; Random musings at https://t.co/C2NpmjRTl9

7 432Followers

839Following

713Posts total

72.2KViews on collected posts

โพสต์ล่าสุด

Ashwin Gopinath ✓@ashwingop18 May

@ashwingop @sentra_app @GoogleResearch OKAY NOW WE ARE FUCKING TALKING 327 views · 3 likes · 0 reposts · 0 replies Open on X →

Ashwin Gopinath ✓@ashwingop18 May

We've open-sourced the whole thing, code, data, and results. Take a look: https://t.co/VgkJrI2A7e 1.1K views · 10 likes · 1 reposts · 1 replies Open on X →

Ashwin Gopinath ✓@ashwingop18 May

7. This isn't an LLM trick — The eigenvalue cliff shows up wherever attention does. We tested on: · Mistral 7B → 5.95× KV compression, +7.5% perplexity · DINOv2 / Depth Anything → compression improves accuracy · ESM-2 → matched fold quality on CASP15 · VideoMAE / AlphaFold https

0:17

1.2K views · 8 likes · 1 reposts · 2 replies Open on X →

Ashwin Gopinath ✓@ashwingop18 May

5. Then the second trick — selective correction TurboQuant applies its JL bias correction to all 128 dims. Costs 128 sign bits per token. Correction noise scales with √128. We apply it to only the 4 signal dims. 4 sign bits. Noise scales with √4. 30× fewer dot products at http

0:08

881 views · 5 likes · 0 reposts · 1 replies Open on X →

Ashwin Gopinath ✓@ashwingop18 May

6. And water-filling for the dominant dims — Even within the top 4 signal dims, eigenvalues vary. λ₀ = 0.50, λ₃ = 0.04. Uniform allocation: [3, 3, 3, 3] bits per dim. Water-filling (Shannon-Berger, 1948): [5, 3, 2, 2]. The dominant dim gets 64 levels of precision. Same https:

0:09

801 views · 5 likes · 0 reposts · 1 replies Open on X →

Ashwin Gopinath ✓@ashwingop18 May

3. Why random rotation wastes bits — TurboQuant rotates KV vectors by a random orthogonal matrix Π before quantizing. That's mathematically optimal *if you don't know anything about your data*. But we do know — and after Π, the variance is roughly evenly spread across all 128 h

0:10

1.1K views · 8 likes · 0 reposts · 2 replies Open on X →

Ashwin Gopinath ✓@ashwingop18 May

4. What we do instead — SpectralQuant rotates by the eigenvectors of the K covariance, measured during a 15-second calibration pass. After rotation: dim 0 has variance λ₀ (largest). Dim 127 has variance λ₁₂₇ (~0). We know exactly where the information lives, so we put the https:

0:09

975 views · 7 likes · 0 reposts · 2 replies Open on X →

Ashwin Gopinath ✓@ashwingop18 May

1. The problem — Modern transformers spend most of their inference memory on the KV cache. Mistral 7B at 8k context = 4 GB just for keys and values. Llama 70B at 32k = 80 GB. Quantizing this is the highest-leverage compression target in inference today. TurboQuant (ICLR 2026) h

0:07

2.3K views · 12 likes · 0 reposts · 1 replies Open on X →

Ashwin Gopinath ✓@ashwingop18 May

2. The insight Google missed — KV vectors are not random. We measured the eigenspectrum of every attention head in Mistral, Qwen, ESM-2 and ViT. The pattern is universal: ~80% of the variance lives in 4 of 128 dimensions. This is the empirical fact every compressor should htt

0:09

1.7K views · 12 likes · 0 reposts · 2 replies Open on X →

Ashwin Gopinath ✓@ashwingop18 May

@sentra_app just killed @GoogleResearch's TurboQuant. SpectralQuant — 5.95× KV cache compression on Mistral 7B at +7.5% perplexity overhead. TurboQuant at the same compression: +22%. 3× less degradation. 15-second calibration. One per-model, then drop-in for any HuggingFace ht

7:32

61.8K views · 223 likes · 27 reposts · 14 replies Open on X →

Growth & engagement

How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.

Views per post

61.8K18 May

1.7K

2.3K

975

1.1K

801

881

1.2K

1.1K

327

Last 10 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.

Engagement rate per post

0.44%18 May

0.85%

0.56%

0.92%

0.90%

0.75%

0.68%

0.91%

1.20%

0.92%

Reactions — likes, reposts, replies and quotes — divided by views.

What the audience does

Likes53.2%293 in total

Reposts5.3%29 in total

Replies4.7%26 in total

Quotes2.0%11 in total

Bookmarks34.8%192 in total

Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.

The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.

บัญชีที่คล้ายกัน