Greg Diamos

@GregoryDiamos

I build AI supercomputers

3 504Followers

142Following

863Posts total

168.7KViews on collected posts

Son gönderiler

Greg Diamos@GregoryDiamos7 Sep

@intel I guess it is doing a good job? 1.1K views · 5 likes · 0 reposts · 0 replies Open on X →

Greg Diamos@GregoryDiamos7 Sep

@intel Agree that this is with basic bf16 and it is possible to make the active param count higher through better weight compression. 1.5K views · 9 likes · 0 reposts · 2 replies Open on X →

Greg Diamos@GregoryDiamos7 Sep

It seems like this got a lot of likes. Remember this is an experiment - don't expect it to be a good production model. I'd do a multi trillion token foundation model run and then fine tune it before using it. ~6k token/sec/core would take a couple months on a 32-core CPU. 6.5K views · 79 likes · 0 reposts · 2 replies Open on X →

Greg Diamos@GregoryDiamos7 Sep

Fitting active params into the L2 cache gets up to 14,572 tok/sec on an @intel emerald rapids CPU with AMX 7.1K views · 97 likes · 2 reposts · 3 replies Open on X →

Greg Diamos@GregoryDiamos7 Sep

Data is doing more of the work than it used to. Every source in our mixture is a curated artifact built with large models Training a model this small on them is distillation When models of this size were last studied seriously such corpora did not exist 9.8K views · 145 likes · 6 reposts · 1 replies Open on X →

Greg Diamos@GregoryDiamos7 Sep

The loss does not saturate. Across a 4.91B-token run, smoothed training loss falls monotonically within each curriculum phase and is still descending at the end 10.1K views · 82 likes · 0 reposts · 1 replies Open on X →

Greg Diamos@GregoryDiamos7 Sep

The behaviours appear early. We assumed in-context copying, positional manipulation and arithmetic would need budgets well beyond one core’s reach. They do not. At 259M tokens — about nine hours — a block-routed MoE with E = 128 experts -- begins learning. 11.7K views · 141 likes · 0 reposts · 0 replies Open on X →

Greg Diamos@GregoryDiamos7 Sep

@GregoryDiamos Wow, this is really cool Gregory! Thanks for sharing. So partly what you're saying is L2 cache is really cool and we all should want more of it! 3.9K views · 16 likes · 0 reposts · 1 replies Open on X →

Greg Diamos@GregoryDiamos7 Sep

I think we should revisit outrageously small neural nets. I needed a 10k tok/s CPU model for data processing. So I gave Anthropic claude code a pile of tokens to build one. It made three interesting discoveries: https://t.co/eR28fVZdDI https://t.co/Pv1zDOPLS0

117K views · 2.1K likes · 112 reposts · 85 replies Open on X →

Aynı büyüklükteki hesaplara karşı

Son 90 güne ait 9 gönderi, under 10K takipçi aralığıyla yan yana. takipçi aralığının tam medyanında.

Medyan görüntülenme7 086bu hesap2 488under 10K için medyan

Erişim, %202.23%bu hesap163.70%under 10K için medyan

Etkileşim, %1.20%bu hesap1.42%under 10K için medyan

Ölçüt	Bu hesap	under 10K için medyan	Oran
Gönderi başına medyan görüntülenme	7 086	2 488	2.85×
Erişim (görüntülenme ÷ takipçi)	2.0× audience	163.70%	1.24×
Etkileşim oranı	1.20%	1.42%	0.85×

Bu aralıktaki diğer hesaplar → Başka bir hesapla karşılaştır → Bu kıyas değerleri nasıl kuruluyor →

Growth & engagement

How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.

Views per post

117K7 Sep

3.9K

11.7K

10.1K

9.8K

7.1K

6.5K

1.5K

1.1K

Last 9 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.

Engagement rate per post

1.93%7 Sep

0.44%

1.20%

0.82%

1.55%

1.45%

1.25%

0.75%

0.45%

Reactions — likes, reposts, replies and quotes — divided by views. Median for under 10K accounts is 1.42%.

What the audience does

Likes52.2%2 626 in total

Reposts2.4%120 in total

Replies1.9%95 in total

Quotes0.3%16 in total

Bookmarks43.2%2 173 in total

Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.

The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.

Son gönderiler

Aynı büyüklükteki hesaplara karşı

Growth & engagement

Views per post

Engagement rate per post

What the audience does

Benzer hesaplar