Ziming Liu
@ZimingLiu11 · joined 07 May 2021
Assistant Professor @Tsinghua_Uni CollegeAI. Chief Scientist @MetaCircle_AI Science of AI, AI for AI, AI for Science, KAN.
14 473Followers
890Following
680Posts total
1.7MViews on collected posts
Son gönderiler
Yeni gönderiler aranıyor…
Foundation Model is dead, Meta Model is future.
Stage 1: Expert systems -- Specialized models
Stage 2: Foundation models -- Unified models
Stage 3: Meta Model -- A unified model that designs specialized models
天下大势，分久必合，合久必分。
https://t.co/Je1xpGxwwf https://t.co/MgmDDWElS7
5.1K views · 75 likes
· 12 reposts · 5 replies
Open on X →
@ZimingLiu11 @MetaCircle_AI Prediction curves are easy to draw. The hard part is knowing when the meta model is wrong. What's your threshold for trusting it on the next run?
18 views · 0 likes
· 0 reposts · 0 replies
Open on X →
@ZimingLiu11 @MetaCircle_AI https://t.co/B4uR58QaNo
115 views · 0 likes
· 0 reposts · 0 replies
Open on X →
@ZimingLiu11 @MetaCircle_AI have you trained/tested on the prior marin runs?
154 views · 0 likes
· 0 reposts · 1 replies
Open on X →
At @MetaCircle_AI , we are building a "Meta Model" (a model for model) that aims to predict model preformance and design better models automatically.
Here is our prediction for the Marin Hero Run:
https://t.co/SEsSCZVVtk
Can't wait Marin's final results to be revealed ! https:/
9.7K views · 79 likes
· 11 reposts · 4 replies
Open on X →
Staring from 2022, I have the habit of MANUALLY browsing and collecting fun arXiv papers, gathering ~2000 papers in total.
Today we release this paper list, and test if an LLM agent could learn my taste simply from this paper list.
https://t.co/RNv4M7kRwc https://t.co/nvp8IIQf5
6.3K views · 88 likes
· 11 reposts · 6 replies
Open on X →
Astronomy has three stages: Tycho — Kepler — Newton.
Previously, the “Physics of AI” is still in the pre-Tycho era — we only observed a few stars (e.g, scaling laws, grokking) in a vast phenomenological sky.
Today, we are finally entering the Tycho era.
https://t.co/mcPMtJtmOW
5K views · 60 likes
· 3 reposts · 6 replies
Open on X →
20/N Adapt to transformers: I have no idea how to do that, although a naive (but might be working!) extension is just replacing MLPs by KANs.
23.4K views · 155 likes
· 1 reposts · 11 replies
Open on X →
19/N Why is training slow?
Reason 1: technical. learnable activation functions (splines) are more expensive to evaluate than fixed activation functions.
Reason 2: personal. The physicist in my body would suppress my coder personality so I didn't try (know) optimizing efficiency.
26.2K views · 163 likes
· 4 reposts · 7 replies
Open on X →
Also fun to collaborate (again) with my friend @RoyWang67103904 whom I knew from elementary school :)
14.8K views · 37 likes
· 0 reposts · 0 replies
Open on X →
18/N Computation requirements:
All examples in our paper can be reproduced in less than 10 minutes on a single CPU (except for sweeping hyperparams). Admittedly, the scale of our problems are smaller than many machine learning tasks, but are typical for science-related tasks.
26.6K views · 124 likes
· 5 reposts · 3 replies
Open on X →
@ZimingLiu11 The dichotomy of "nodes" vs "weights" for placing the activation function is really a distinction between
•placing functions on individual outputs ("weights") or
•restricting functions to sums of outputs ("nodes")—right?
10.9K views · 43 likes
· 2 reposts · 1 replies
Open on X →
17/N Given our empirical results, we believe that KANs will be a useful model/tool for AI + Science due to their accuracy, parameter efficiency and interpretability. The usefulness of KANs for machine learning-related tasks is more speculative and left for future work. https://t.
49.9K views · 173 likes
· 8 reposts · 2 replies
Open on X →
16/N, or symbolically https://t.co/JLanyudRlm
21.7K views · 75 likes
· 0 reposts · 1 replies
Open on X →
15/N KANs are also helpful assistants or collaborators for scientists. We showed how KANs can help study Anderson localization, a type of phase transition in condensed matter physics. KANs make extraction of mobility edges super easy, either numerically, https://t.co/efz7agOlUA
21.6K views · 90 likes
· 1 reposts · 1 replies
Open on X →
14/N In particular, Deepmind’s MLPs have ~300000 parameters, while our KANs only have ~200 parameters. KANs are immediately interpretable, while MLPs require feature attribution as post analysis.
21.1K views · 148 likes
· 6 reposts · 1 replies
Open on X →
13/N We used KANs to rediscover mathematical laws in knot theory. KANs not only reproduced Deepmind's results with much smaller networks and much more automation, KANs also discovered new formulas for signature and discovered new relations of knot invariants in unsupervised ways.
24.7K views · 192 likes
· 13 reposts · 2 replies
Open on X →
12/N Human users can interact with KANs to make them more interpretable. It’s easy to inject human inductive biases or domain knowledge into KANs. https://t.co/x8V2njBMzD
23.4K views · 132 likes
· 8 reposts · 2 replies
Open on X →
11/N KANs are also interpretable. KANs can reveal compositional structures and variable dependence of synthetic datasets from symbolic formulas. https://t.co/LrHfkCU0Z2
26.1K views · 138 likes
· 8 reposts · 1 replies
Open on X →
10/N As a bonus, we also find KANs' natural ability to avoid catastrophic forgetting, at least in a toy case we tried. https://t.co/qkZxplB5k7
25K views · 164 likes
· 6 reposts · 3 replies
Open on X →
9/N KANs are more accurate than MLPs in PDE solving, e.g, solving the Poisson equation. https://t.co/LeogCCdUkw
24.1K views · 102 likes
· 5 reposts · 1 replies
Open on X →
8/N KANs are more accurate than MLPs in function fitting, e.g, fitting special functions. https://t.co/iRmwMcVMUH
24.8K views · 95 likes
· 4 reposts · 1 replies
Open on X →
7/N Neural scaling laws: KANs have much faster scaling than MLPs, which is mathematically grounded in the Kolmogorov-Arnold representation theorem. KAN's scaling exponent can also be achieved empirically. https://t.co/oko0O1g4jK
34.7K views · 143 likes
· 9 reposts · 3 replies
Open on X →
6/N From practical aspects: We find that KANs are more accurate and interpretable than MLPs, although we have to be honest that KANs are slower to train due to their learnable activation functions. Below we present our results.
25K views · 115 likes
· 3 reposts · 3 replies
Open on X →
5/N From the algorithmic aspect: KANs and MLPs are dual in the sense that -- MLPs have (usually fixed) activation functions on neurons, while KANs have (learnable) activation functions on weights. These 1D activation functions are parameterized as splines. https://t.co/HYPsTjys0
28.8K views · 130 likes
· 7 reposts · 5 replies
Open on X →
@tegmark 4/N From the math aspect: MLPs are inspired by the universal approximation theorem (UAT), while KANs are inspired by the Kolmogorov-Arnold representation theorem (KART). Can a network achieve infinite accuracy with a fixed width? UAT says no, while KART says yes (w/ cave
27.6K views · 122 likes
· 4 reposts · 1 replies
Open on X →
@tegmark 3/N Inspired by the representation theorem, we explicitly parameterize the Kolmogorov-Arnold representation with neural networks. In honor of two great late mathematicians, Andrey Kolmogorov and Vladimir Arnold, we call them Kolmogorov-Arnold Networks (KANs). https://t.
32.6K views · 163 likes
· 7 reposts · 3 replies
Open on X →
@tegmark 2/N This change sounds from nowhere at first, but it has rather deep connections to approximation theories in math. It turned out, Kolmogorov-Arnold representation corresponds to 2-Layer networks, with (learnable) activation functions on edges instead of on nodes. https:
34K views · 131 likes
· 7 reposts · 1 replies
Open on X →
@tegmark 1/N MLPs are foundational for today's deep learning architectures. Is there an alternative route/model? We consider a simple change to MLPs: moving activation functions from nodes (neurons) to edges (weights)! https://t.co/MTjrpUm9yP
53.1K views · 308 likes
· 41 reposts · 5 replies
Open on X →
This is a joint work w/ @tegmark and awesome collaborators from MIT, Northeastern, IAIFI and Caltech.
40.2K views · 106 likes
· 2 reposts · 1 replies
Open on X →
Aynı büyüklükteki hesaplara karşı
Son 90 güne ait 7 gönderi, 10K–100K takipçi aralığıyla yan yana. geniş kitleye gösteriliyor, ama izleyenlerin azı tepki veriyor.
Medyan görüntülenme4 993bu hesap92410K–100K için medyan
Erişim, %34.50%bu hesap3.62%10K–100K için medyan
Etkileşim, %0.98%bu hesap1.52%10K–100K için medyan
| Ölçüt | Bu hesap | 10K–100K için medyan | Oran |
|---|---|---|---|
| Gönderi başına medyan görüntülenme | 4 993 | 924 | 5.40× |
| Erişim (görüntülenme ÷ takipçi) | 34.50% | 3.62% | 9.53× |
| Etkileşim oranı | 0.98% | 1.52% | 0.64× |
Bu aralıktaki diğer hesaplar → Başka bir hesapla karşılaştır → Bu kıyas değerleri nasıl kuruluyor →
Growth & engagement
How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.
Views per post
21.7K1 May
49.9K
10.9K
26.6K
14.8K
26.2K
23.4K
5K14 Aug
6.3K23 Aug
9.7K28 Aug
154
115
18
5.1K30 Aug
Last 14 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.
Engagement rate per post
0.35%1 May
0.37%
0.42%
0.50%
0.25%
0.68%
0.73%
1.38%14 Aug
1.69%23 Aug
0.98%28 Aug
0.65%
0.00%
0.00%
1.82%30 Aug
Reactions — likes, reposts, replies and quotes — divided by views. Median for 10K–100K accounts is 1.52%.
What the audience does
Likes79.6%3 351 in total
Reposts4.5%188 in total
Replies1.9%81 in total
Quotes1.0%41 in total
Bookmarks13.0%549 in total
Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.
The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.
Benzer hesaplar
Carlos
@Sanchez52Carlos
28.9K followers
· 25.7K posts
andrew ✓
@AndrewChamings
28.9K followers
· 6.4K posts
Nuria Bookstore Kenya ✓
@NuriaStore
28.9K followers
· 67.2K posts
CIJA ✓
@CIJAinfo
28.9K followers
· 47.7K posts
Larry Dignan
@ldignan
28.9K followers
· 26.4K posts
BabyBonk BNB ✓
@BabyBonkCoin
28.9K followers
· 3.4K posts
Michigan Volleyball
@umichvball
28.9K followers
· 18.1K posts
Misunderstood Multibaggers ✓
@meeijer
28.9K followers
· 22K posts