Keller Jordan ✓
@kellerjordan0 · San Francisco · joined 23 Mar 2016
CIFAR-10 fanatic Pretraining @OpenAI OpCo LLC. Personal opinion poster
19 317Followers
459Following
1 681Posts total
534.5KViews on collected posts
โพสต์ล่าสุด
กำลังค้นหาโพสต์ใหม่…
Not entirely. Human beings dedicating their lives to speedrunning provides a service that capitalism desperately needs: Affirmation that the greatness of the Computer Maniac way of life transcends capitalistic incentives.
9.3K views · 74 likes
· 1 reposts · 4 replies
Open on X →
Human civilizations are well-regularized by our mortality & inability to copy ourselves.
Machine civs will have these regularization controls turned off.
=> No one knows if a global machine civ would even be stable: It could be like training MoE without load balancing.
8.9K views · 74 likes
· 3 reposts · 6 replies
Open on X →
Expanding on this:
A human brain throughout its lifespan uses ~50GJ of energy. How far can you get using AI technology with 50GJ? Basically nowhere: Just training Llama 3 8B (a tiny model) used 3,000GJ.
In fact, the power of machine intelligence has never been about being a
75.6K views · 467 likes
· 32 reposts · 28 replies
Open on X →
Not going to give examples, but it’s interesting how you can see a population of divisive posts on here which aren’t quite false enough to get directly community noted, but are certainly false enough to get debunked by any chatbot that you paste them into.
I wonder what will
8.6K views · 63 likes
· 0 reposts · 8 replies
Open on X →
Broke: We should be careful about AI because it might mistreat humanity during its glorious conquest of the galaxy
Woke: We should be careful about AI because a global swarm of immortal perfect-clone agents might be vulnerable to catastrophic fungal infections
80.2K views · 177 likes
· 5 reposts · 9 replies
Open on X →
@kellerjordan0 Neat 🥳 In DDU, we argue that bc the mutual information of an ensemble is also great for uncertainty estimation, this requires the members to disagree in their predictions, which leads to variance, so some will have to perform better and some worse 🤗
https://t.co/X
2.7K views · 27 likes
· 1 reposts · 1 replies
Open on X →
14/ It also wouldn’t have been possible to efficiently train the ~350K networks needed for it, without the FFCV library from @aleks_madry‘s lab.
Thanks to @bneyshabur and @esiamid for their advisements during the project.
4.4K views · 38 likes
· 1 reposts · 4 replies
Open on X →
11/ These findings were obtained from studying standard, well-tuned trainings on CIFAR-10 and ImageNet. As a caveat, for unstable trainings (e.g. too high learning rate) variance exceeds the hypothesis and is certainly not harmless. https://t.co/UVEuUcviaF
4.4K views · 30 likes
· 1 reposts · 1 replies
Open on X →
13/ Shoutouts: This project was largely inspired by @david_picard‘s empirical study “Torch.manual_seed(3407) is all you need”.
https://t.co/RFQHcr4mmB
6.5K views · 44 likes
· 0 reposts · 2 replies
Open on X →
12/ When trained networks are evaluated on distribution-shifted test-sets, there is also significant excess variance. https://t.co/t1qBl0vlTY
4.3K views · 28 likes
· 0 reposts · 2 replies
Open on X →
8/ Turning to the origin of variance, prior works (especially https://t.co/OsVeRFogKu) have observed that ensembles of independently trained networks make roughly calibrated predictions. I prove that this calibration property alone implies variation in test-set accuracy. https://
7.3K views · 38 likes
· 1 reposts · 1 replies
Open on X →
10/ My conclusions: variance is both _harmless_ (does not imply almost any differences in model quality) and _inevitable_ (cannot be gotten rid of without sacrificing other beneficial properties of training).
6.8K views · 47 likes
· 2 reposts · 2 replies
Open on X →
7/ Also usefully, the excess observed variance over this statistical model forms an unbiased estimator for the variance in accuracy on the test _distribution_, which turns out to be very small. (Full definitions & proof in the paper) https://t.co/6kAUGN3Y24
5.6K views · 29 likes
· 0 reposts · 2 replies
Open on X →
6/ This would imply that the test-set accuracy distribution is generated as the sum of a series of independent coin flips, one for each test-set example.
And…this simple statistical model actually turns out to be a very good approximation. https://t.co/RZmvJ0la0M
6K views · 52 likes
· 1 reposts · 2 replies
Open on X →
5/ Given this lack of correlation between performance on splits of test-set data, I next test the hypothesis that there also aren’t even any correlations between individual examples. https://t.co/9ojxqNp1iq
6.2K views · 36 likes
· 0 reposts · 1 replies
Open on X →
4/ But instead, it turns out that by the end of training, there’s almost no correlation between performance on the two splits.
For example, out of ~10^5 repeated trainings, the best network on the first split isn’t even above average on the second. https://t.co/ew74cE8tIw
7.7K views · 65 likes
· 2 reposts · 2 replies
Open on X →
3/ However, it’s not yet clear if these lucky runs of training are actually better than unlucky ones.
To find out, I split the CIFAR-10 test-set into two halves, and evaluated thousands of trained networks against both. Intuitively, lucky networks should do well on both splits.
7.4K views · 44 likes
· 0 reposts · 1 replies
Open on X →
2/ Background: for standard CIFAR-10 trainings there exist rare “lucky seeds/runs” attaining over +0.5% higher test-set accuracy than the average (10% fewer errors). ImageNet trainings are similar with +0.4%.
These differences are considered significant in computer vision.
8.6K views · 55 likes
· 0 reposts · 2 replies
Open on X →
Neural network trainings are nondeterministic. Repeated runs each produce a unique network, often with significantly _varying_ test-set performance.
🆕📜 I demonstrate that this variation has a simple statistical structure, and is harmless & inevitable
https://t.co/1zzpNHi0Vy htt
273.8K views · 1.3K likes
· 160 reposts · 28 replies
Open on X →
"torch.manual_seed(3407) is all you need"!
draft 📜: https://t.co/MyLlhHPcsM
Sorry for the title. I promise it's not (entirely) just for trolling. It's my little spare time project of this summer to investigate unaccounted randomness in #ComputerVision and #DeepLearning.
🧵👇 1/n
0 views · 402 likes
· 79 reposts · 6 replies
Open on X →
เทียบกับบัญชีขนาดเดียวกัน
4 โพสต์จาก 90 วันที่ผ่านมา เทียบกับช่วง 10K–100K ผู้ติดตาม แสดงในวงกว้าง แต่มีผู้ชมตอบสนองน้อย.
ยอดดูมัธยฐาน9 106บัญชีนี้924ค่ามัธยฐานของ 10K–100K
การเข้าถึง, %47.14%บัญชีนี้3.62%ค่ามัธยฐานของ 10K–100K
การมีส่วนร่วม, %0.83%บัญชีนี้1.52%ค่ามัธยฐานของ 10K–100K
| ตัวชี้วัด | บัญชีนี้ | ค่ามัธยฐานของ 10K–100K | อัตราส่วน |
|---|---|---|---|
| ยอดดูมัธยฐานต่อโพสต์ | 9 106 | 924 | 9.85× |
| การเข้าถึง (ยอดดู ÷ ผู้ติดตาม) | 47.14% | 3.62% | 13.0× |
| อัตราการมีส่วนร่วม | 0.83% | 1.52% | 0.55× |
บัญชีอื่นในช่วงนี้ → เปรียบเทียบกับบัญชีอื่น → ค่าอ้างอิงเหล่านี้คำนวณอย่างไร →
Growth & engagement
How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.
Views per post
6K13 Apr
5.6K
6.8K
7.3K
4.3K
6.5K
4.4K
4.4K
2.7K
80.2K27 Jun
8.6K13 Aug
75.6K14 Aug
8.9K18 Aug
9.3K4 Sep
Last 14 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.
Engagement rate per post
0.91%13 Apr
0.55%
0.75%
0.54%
0.72%
0.70%
0.73%
0.97%
1.09%
0.24%27 Jun
0.83%13 Aug
0.71%14 Aug
0.94%18 Aug
0.85%4 Sep
Reactions — likes, reposts, replies and quotes — divided by views. Median for 10K–100K accounts is 1.52%.
What the audience does
Likes65.1%2 716 in total
Reposts5.0%210 in total
Replies2.5%106 in total
Quotes0.7%31 in total
Bookmarks26.6%1 112 in total
Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.
The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.
บัญชีที่คล้ายกัน
Biohacker ✓
@biohacker
38.6K followers
· 21K posts
OnSevilla - Ocio en Sevilla ✓
@SevillaOcio
38.6K followers
· 34.1K posts
LandonTengwall ✓
@LandonTengwall
38.6K followers
· 19.6K posts
FINAL FANTASY XIV
@FF_XIV_DE
38.6K followers
· 9.3K posts
Sortiraparis
@sortirapariscom
38.6K followers
· 110.8K posts
Kamen America (仮面アメリカ) ✓
@kamenamerica
38.6K followers
· 61.5K posts
RPC
@rpcparana
38.6K followers
· 33.8K posts
Arquidiócesis de Caracas ✓
@ArquiCaracas
38.6K followers
· 10.8K posts