tweetindex
DE

Niklas Muennighoff ✓

@Muennighoff · joined 15 May 2020

ai/llms @spacex @stanford

10 547Followers
570Following
300Posts total
886.7KViews on collected posts

Neueste Beiträge

interesting work! sophisticated ways to remove past LLM context ("kv-cache eviction") seem to mostly come down to how much of the prefix they preserve it's a cleaner motivation for prefix sliding than ours in https://t.co/fUw7yJBkVb 8.4K views · 61 likes · 6 reposts · 3 replies Open on X →
Every KV cache evictor for efficient long reasoning scores cached tokens to decide which ones to keep. We find the score buys almost nothing: keep the prompt, evict at random per head, and you match the strongest evictor at 32–43% higher throughput. Paper: https://t.co/ZvbJLCFUw
24.1K views · 104 likes · 39 reposts · 24 replies Open on X →
how clean/simple a method is makes for a good metric i think! a rough measure is # of hyperparams it adds: compaction: summary max len, summary prompt; summary trigger, summary model, summary placement prefix sliding: window size 4.4K views · 39 likes · 0 reposts · 1 replies Open on X →
Incredible that this works! Really clean idea, I would’ve thought a more complicated compaction scheme would be needed. 10.9K views · 27 likes · 0 reposts · 0 replies Open on X →
new paper: Prefix Sliding for efficient test-time scaling vanilla full attention OOMs on long tasks & compaction loses important details -- prefix sliding is a simple & fast alternative that can outperform both 📜https://t.co/fUw7yJAN5D https://t.co/PgcyrsfIZb
59.1K views · 308 likes · 43 reposts · 18 replies Open on X →
can LLMs replace embedding models? in a new work "embedder's dilemma" we find LLMs now beat embedding models -- but at much higher cost. when to choose which?... 📜 https://t.co/OVmGSkL7i0 https://t.co/nQ6xmf79m3
101.4K views · 566 likes · 77 reposts · 19 replies Open on X →
Did LLMs finally subsume embedding models? We compared 10 LLMs vs. 26 embedding models across 37 MTEB tasks. LLMs scored 77.6 while embedding models scored 77.2 But the costs are very different. 🧵 https://t.co/tNESOiukQ4
55.3K views · 63 likes · 5 reposts · 3 replies Open on X →
@Muennighoff 👍 740 views · 3 likes · 0 reposts · 0 replies Open on X →
📜 https://t.co/wmAvNwmZ9A 💻 https://t.co/pVGU3KaAHo 🌐 https://t.co/laYykGT8Ot 🤖 https://t.co/d6kyOgi5mS Thanks to a great team @ZitongYang0 @WeijiaShi2 @XiangLisaLi2 @drfeifei @HannaHajishirzi @LukeZettlemoyer @percyliang @EmmanuelCandes @tatsu_hashimoto + @ryan_marten! 🧵8/9 3.3K views · 35 likes · 2 reposts · 1 replies Open on X →
Thanks to @BLeavesYe & @WenhuChen & @mbalunovic for their AIME25 evals. You can reproduce our 60% on AIME25 I here: https://t.co/ReRodsflUU The result file & generations are: https://t.co/e6whCwUXLa & https://t.co/E2g1npagZV 🧵9/9 3K views · 23 likes · 1 reposts · 0 replies Open on X →
𝐒𝐞𝐪𝐮𝐞𝐧𝐭𝐢𝐚𝐥 𝐯𝐬 𝐏𝐚𝐫𝐚𝐥𝐥𝐞𝐥 𝐬𝐜𝐚𝐥𝐢𝐧𝐠 While budget forcing “Wait” improves s1 AIME24 performance 50%➡️57% it eventually flattens out (left). Overall, however, it scales better than generating many times in parallel & taking the majority answer (right). 🧵6/9 https://t.co/pEFcX3JWU3
3.5K views · 24 likes · 4 reposts · 1 replies Open on X →
𝐁𝐞𝐲𝐨𝐧𝐝 𝐛𝐮𝐝𝐠𝐞𝐭 𝐟𝐨𝐫𝐜𝐢𝐧𝐠 Scaling too much harms performance due to running out of context length. How to solve this? Adding parallel scaling approaches like majority voting or a tree-based approach (REBASE) on top of our model allows scaling further 🎢 🧵7/9 https://t.co/mNqQYa9aL2
3.2K views · 26 likes · 1 reposts · 2 replies Open on X →
𝐁𝐮𝐝𝐠𝐞𝐭 𝐟𝐨𝐫𝐜𝐢𝐧𝐠 𝐭𝐨 𝐜𝐨𝐧𝐭𝐫𝐨𝐥 𝐜𝐨𝐦𝐩𝐮𝐭𝐞 To control test-time compute, we develop “budget forcing”. We either force the model to end its thinking or extend it by appending Wait when the model tries to stop. This simple method improves our model. 🧵5/9 https://t.co/pWNu7D6Vlr
3.8K views · 27 likes · 0 reposts · 1 replies Open on X →
𝐌𝐨𝐝𝐞𝐥 We train Qwen2.5-32B-Instruct on s1K to create s1/s1.1. As it is just 1K examples, training is extremely cheap and took just 26 minutes. After training, s1.1 is an extremely sample-efficient reasoning model with strong performance. 🧵4/9 https://t.co/YmSsucoAZ3
3.4K views · 18 likes · 1 reposts · 2 replies Open on X →
𝐆𝐞𝐦𝐢𝐧𝐢 -> 𝐃𝐞𝐞𝐩𝐒𝐞𝐞𝐤 For our original s1K data, we used Gemini to generate reasoning traces for our dataset, but @ryan_marten regenerated traces with DeepSeek r1, which are, on average, longer with more diversity in lengths. Huge credits to @ryan_marten! 🧵3/9 https://t.co/xSaaJ4v
7.7K views · 44 likes · 5 reposts · 1 replies Open on X →
𝐃𝐚𝐭𝐚𝐬𝐞𝐭 For s1, we curated a small dataset of 1,000 examples: “s1K”. It is extremely diverse, covering various subfields of Mathematics, Biology, Astronomy, etc. Our ablations show that three criteria are key in our data filtering: Quality, Diversity & Difficulty. 🧵2/9 https://t
3.7K views · 27 likes · 1 reposts · 1 replies Open on X →
Last week we released s1 - our simple recipe for sample-efficient reasoning & test-time scaling. We’re releasing 𝐬𝟏.𝟏 trained on the 𝐬𝐚𝐦𝐞 𝟏𝐊 𝐪𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐬 but performing much better by using r1 instead of Gemini traces. 60% on AIME25 I. Details in 🧵1/9 https://t.co/lbAbDX7X3q
165.5K views · 758 likes · 116 reposts · 23 replies Open on X →
@Muennighoff Do you have a solana wallet? 1.3K views · 17 likes · 3 reposts · 10 replies Open on X →
@Muennighoff Did you know you have a coin about s1? BNCAmrjigo67N96V862VtArmXA8wTasJVUzk6mL4pump 1.8K views · 10 likes · 1 reposts · 0 replies Open on X →
@Muennighoff @stanfordnlp But, fwiw, they did provide a test-time scaling plot for their DeepSeek-R1-Lite-Preview! (Even though, as with the OpenAI models, it’s relationship to the later non-preview model isn’t totally clear.) https://t.co/3whCsfd3jH https://t.co/mYc6nB0lC0
5.9K views · 28 likes · 4 reposts · 1 replies Open on X →
DeepSeek r1 is exciting but misses OpenAI’s test-time scaling plot and needs lots of data. We introduce s1 reproducing o1-preview scaling & performance with just 1K samples & a simple test-time intervention. 📜https://t.co/wmAvNwmrk2 https://t.co/YPKUHNsmQU
416.2K views · 976 likes · 185 reposts · 40 replies Open on X →

Im Vergleich zu Konten gleicher Größe

5 Beiträge aus den letzten 90 Tagen, verglichen mit der Größenklasse 10K–100K Follower. wird weit gezeigt, aber nur wenige dieser Zuschauer reagieren.

Medianaufrufe10 921dieses Konto924Median für 10K–100K
Reichweite, %103.55%dieses Konto3.62%Median für 10K–100K
Interaktion, %0.71%dieses Konto1.52%Median für 10K–100K
KennzahlDieses KontoMedian für 10K–100KVerhältnis
Medianaufrufe pro Beitrag10 92192411.8×
Reichweite (Aufrufe ÷ Follower)103.55%3.62%28.6×
Interaktionsrate0.71%1.52%0.47×

Weitere Konten dieser Größe →   Mit einem anderen Konto vergleichen →   Wie diese Vergleichswerte entstehen →

Growth & engagement

How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.

Views per post

3.4K11 Feb
3.8K
3.2K
3.5K
3K
3.3K
740
55.3K19 Aug
101.4K20 Aug
59.1K27 Aug
10.9K28 Aug
4.4K
24.1K4 Sep
8.4K7 Sep

Last 14 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.

Engagement rate per post

0.62%11 Feb
0.77%
0.90%
0.85%
0.80%
1.14%
0.41%
0.14%19 Aug
0.66%20 Aug
0.64%27 Aug
0.26%28 Aug
0.91%
0.71%4 Sep
0.83%7 Sep

Reactions — likes, reposts, replies and quotes — divided by views. Median for 10K–100K accounts is 1.52%.

What the audience does

Likes54.1%3 184 in total
Reposts8.4%494 in total
Replies2.6%151 in total
Quotes1.6%94 in total
Bookmarks33.3%1 960 in total

Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.

The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.

Ähnliche Konten

Frequently asked questions about @Muennighoff

How many followers does @Muennighoff have?

10 547 followers.

How many posts has @Muennighoff published?

300 posts.

How do posts by @Muennighoff perform?

Collected posts: 21. Views: 886 668. Likes: 3 184.