Adam Paszke
@apaszke · Berlin, Germany · joined 02 Sep 2014
Author of PyTorch, Research Scientist at Google DeepMind. Currently working on Pallas, Mosaic and dex-lang. MIMUW CS & Math graduate. apaszke@types.pl
11 461Followers
531Following
785Posts total
241.4KViews on collected posts
Latest posts
Looking for new posts…
Want to improve GPU compute/comms overlap? We just published a new short tutorial for you!
A few small changes to the Pallas:MGPU matmul kernel is all it takes to turn it into an all-gather collective matmul that overlaps NVLINK comms with local compute: https://t.co/HY4C3MwMb7
33.7K views · 303 likes
· 46 reposts · 7 replies
Open on X →
Curious how to write SOTA performance Blackwell matmul kernels using MGPU? We just published a short step-by-step tutorial: https://t.co/XRVX34juEz
At each step, we show exactly what (small) changes are necessary to refine the kernel and the final kernel is just under 150 lines.
55.4K views · 413 likes
· 67 reposts · 4 replies
Open on X →
A small Mosaic GPU update: TMA multicast is now supported! Async copies take a cluster dim and the DSL handles the details for you: https://t.co/tJ8XWGEq61
It's also quite cool that GPU kernels with multicast look very similar to Pallas TPU collective kernels (more convergence!)
4.3K views · 36 likes
· 4 reposts · 1 replies
Open on X →
@apaszke congrats :)
but NGL it looks super hard to understand (compared to say writing something in Triton)
5.4K views · 26 likes
· 0 reposts · 2 replies
Open on X →
GPUs have gotten surprisingly similar to TPUs and this project is about trying to transfer what we’ve learned while building Pallas TPU to latest GPUs. And it's working out surprisingly well! Soon Pallas should be able to produce terse SOTA kernels on both platforms.
5.4K views · 49 likes
· 1 reposts · 2 replies
Open on X →
Also a caveat: Mosaic GPU is still very early and experimental so the DSL is unstable. Our goal is to make Pallas the stable interface for it, but it will require more work.
4.3K views · 35 likes
· 0 reposts · 2 replies
Open on X →
Peak perf is what counts, but it should also be fun to use! It's been my playground for exploring GPU optimization that includes cool features like a profiler that lets you explicitly see the matmul and softmax being overlapped during attention. https://t.co/eF2T2UxF3N
5.6K views · 47 likes
· 1 reposts · 1 replies
Open on X →
Many of you are excited about H100 attention, so it’s a good time to show you Mosaic GPU: a Python DSL for H100s.
The attention example matches FA3 performance, while being only ~200 lines of Python: https://t.co/12ecz3LftV
It's easy to install too! Latest JAX packages have it.
116.8K views · 658 likes
· 109 reposts · 14 replies
Open on X →
Growth & engagement
How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.
Views per post
10.5K19 Apr
116.8K15 Jul
5.6K
4.3K
5.4K
5.4K
4.3K25 Jul
55.4K2 Oct
33.7K13 Oct
Last 9 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.
Engagement rate per post
0.72%19 Apr
0.68%15 Jul
0.88%
0.86%
0.96%
0.52%
0.96%25 Jul
0.89%2 Oct
1.06%13 Oct
Reactions — likes, reposts, replies and quotes — divided by views.
What the audience does
Likes54.7%1 634 in total
Reposts7.9%236 in total
Replies1.1%34 in total
Quotes0.7%22 in total
Bookmarks35.5%1 060 in total
Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.
The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.
Similar accounts
Steve
@SteveSPORTTalk1
22.9K followers
· 437.9K posts
KSJ ✓
@KSJ_Kreative
22.9K followers
· 125.6K posts
Ubuntu Times
@ubuntutimesUg
22.9K followers
· 6.4K posts
Richard Callander
@richieplz
22.9K followers
· 43.4K posts
Tomás González-Martín
@tgm46
22.9K followers
· 17.7K posts
Mike Kayihura
@MikeKayihura
22.9K followers
· 5.5K posts
Resham Saleem🇵🇰🐅 ✓
@ReshamSaleem3
22.9K followers
· 82K posts
Michael Quesada ✓
@michaelpcmaster
22.9K followers
· 5.5K posts