Andrew Gordon Wilson ✓
@andrewgwils · New York, NY · joined 09 Sep 2014
Research Lead @perplexity_ai. Professor of machine learning at NYU Courant.
40 793Followers
1 185Following
3 483Posts total
854.5KViews on collected posts
Latest posts
Looking for new posts…
@andrewgwils queued up to watch this, epiplexity looks super interesting
525 views · 1 likes
· 0 reposts · 0 replies
Open on X →
Doing one thing really well carries so much more value than doing five things at an average level.
7.5K views · 137 likes
· 7 reposts · 5 replies
Open on X →
@andrewgwils I'll have a look. I've long said, that is since I began in NN 1987-1994, that there's no such thing as generalization.
583 views · 1 likes
· 0 reposts · 0 replies
Open on X →
I had a great time presenting the "Foundations of Modern AI" at the Berkeley Deep Learning for Science Summer School. The talk covered a prescriptive theory of generalization and epiplexity. Video now online! https://t.co/muxf74g8mo
81.7K views · 588 likes
· 80 reposts · 10 replies
Open on X →
With NeurIPS registration sold out weeks before even author notification, I'm worried the conference will simply be unrecognizable. Basically no community regulars will have registered -- they won't have been using agents to register within minutes of opening.
73.1K views · 313 likes
· 15 reposts · 5 replies
Open on X →
As much as AI is impressively progressing, current systems are appallingly poor at conducting end-to-end scientific research. 1/4
49.1K views · 224 likes
· 16 reposts · 13 replies
Open on X →
@andrewgwils One year ago it was the usual AI cannot solve anything and won’t. Now it is but not good enough. Next step, all the important math questions and what will you say? It will help physics tremendously for once. But what do I know, I’m not a mathematician… !
3.7K views · 40 likes
· 1 reposts · 0 replies
Open on X →
@andrewgwils AI will disprove this conjecture too.
6.5K views · 201 likes
· 4 reposts · 0 replies
Open on X →
@andrewgwils Prediction: we'll see cope from academics on social media increase exponentially over the next 6 months.
12.8K views · 521 likes
· 18 reposts · 4 replies
Open on X →
Prediction: we’ll see loads of conjectures and theorems proven and disproven by LLMs over the next 6 months and it will make basically zero difference to the world, and won’t even contribute meaningfully to advancing mathematics.
189.6K views · 1.5K likes
· 89 reposts · 155 replies
Open on X →
@andrewgwils I am a fan of soft inductive bias (like those employed in PAC-Bayes, union bounds, their ilk). I think the challenge is accounting for the last O(1)% of generalization using it, at least in vision models. This was the goal of attack that Karolina and I started back
2.9K views · 25 likes
· 2 reposts · 0 replies
Open on X →
The textbooks don't need to be rewritten -- they just needed to pay attention to what was already known about generalization, decades ago! I've had thoughts about this for 12 years, and people always ask for the paper -- so I finally wrote it. Thankful to many for feedback! 12/12
5.6K views · 70 likes
· 2 reposts · 4 replies
Open on X →
We also further consider overparametrization and double descent. Deep learning of course _is_ different, and much is not well understood. To this end, we particularly highlight representation learning, mode connectivity, and universal learning. Much more in the paper! 11/12 https
6.2K views · 34 likes
· 1 reposts · 1 replies
Open on X →
Benign overfitting describes perfectly fitting a mix of signal and noise, but still generalizing respectably. Like with the polynomial, we can exactly reproduce this behaviour with a Gaussian process and bound it with PAC-Bayes through the marginal likelihood. 8/12 https://t.co/I
4.8K views · 35 likes
· 0 reposts · 1 replies
Open on X →
Computing the bounds is also very simple: (i) train a model to find hypothesis h*, using any optimizer; (ii) measure the empirical risk R(h*) (e.g., training loss); (iii) measure the filesize of the stored model. 10/12
4.3K views · 34 likes
· 0 reposts · 1 replies
Open on X →
These frameworks provide a prescription for generalization, in line with soft inductive biases: embrace a maximally flexible hypothesis space, combined with a compression bias. 9/12
4.9K views · 41 likes
· 1 reposts · 1 replies
Open on X →
We can use a Solomonoff prior, which represents a maximally overparametrized model, but assigns exponentially higher weights to simpler (shorter) programs with lower Kolmogorov complexity, leading to non-vacuous generalization bounds on even billion parameter neural nets. 7/12 h
5.6K views · 49 likes
· 2 reposts · 2 replies
Open on X →
While this behaviour cannot be explained by Rademacher complexity or VC dimension (which measures a model's ability to fit noise), it _can_ be described by decades-old countable hypothesis bounds with a prior, which do not penalize the size of the hypothesis space. 6/12 https://t
6K views · 46 likes
· 3 reposts · 1 replies
Open on X →
This model also performs well over a range of data sizes and complexities. Notably, a central observation in "understanding DL requires rethinking generalization" was a neural net can fit noise perfectly, but still generalize on structured data; but this polynomial can too. 5/12
6.2K views · 42 likes
· 1 reposts · 3 replies
Open on X →
Rather than restricting the solutions a model can represent, specify a preference for certain solutions over others, through _soft_ inductive biases. This approach guides us towards structure where it exists, without significant penalty where it doesn't. 3/12 https://t.co/y7flCxm
8.7K views · 65 likes
· 4 reposts · 1 replies
Open on X →
Let's consider the simplest possible example: a polynomial. We’ll use an arbitrarily high-order polynomial with order-dependent regularization that penalizes the norms of higher order coefficients more. This model scales its complexity as needed, and can also fit pure noise! 4/12
6.5K views · 45 likes
· 1 reposts · 2 replies
Open on X →
What makes deep learning different? Not overparametrization, benign overfitting, or double descent, which can be reproduced with other models and explained with old generalization frameworks. Understanding DL doesn't require rethinking generalization -- and it never did! 2/12
12.2K views · 73 likes
· 4 reposts · 2 replies
Open on X →
My new paper "Deep Learning is Not So Mysterious or Different": https://t.co/AgHdSQkals. Generalization behaviours in deep learning can be intuitively understood through a notion of soft inductive biases, and formally characterized with countable hypothesis bounds! 1/12 https://t
355K views · 2.2K likes
· 328 reposts · 20 replies
Open on X →
Against accounts of the same size
7 posts from the last 90 days, next to the 10K–100K follower range. shown widely, but few of those viewers react.
Median views7 474this account924median for 10K–100K
Reach, %18.32%this account3.62%median for 10K–100K
Engagement, %0.46%this account1.52%median for 10K–100K
| Metric | This account | Median for 10K–100K | Ratio |
|---|---|---|---|
| Median views per post | 7 474 | 924 | 8.09× |
| Reach (views ÷ followers) | 18.32% | 3.62% | 5.06× |
| Engagement rate | 0.46% | 1.52% | 0.30× |
Others in this range → Compare with another account → How these benchmarks are built →
Growth & engagement
How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.
Views per post
6.2K5 Mar
5.6K
2.9K
189.6K1 Aug
12.8K
6.5K
3.7K
49.1K5 Sep
73.1K7 Sep
81.7K
716
583
7.5K
5258 Sep
Last 14 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.
Engagement rate per post
0.58%5 Mar
1.35%
0.94%
0.94%1 Aug
4.26%
3.14%
1.11%
0.52%5 Sep
0.46%7 Sep
0.83%
0.28%
0.17%
1.99%
0.19%8 Sep
Reactions — likes, reposts, replies and quotes — divided by views. Median for 10K–100K accounts is 1.52%.
What the audience does
Likes58.1%6 315 in total
Reposts5.3%579 in total
Replies2.1%231 in total
Quotes0.4%46 in total
Bookmarks34.1%3 703 in total
Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.
The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.
Similar accounts
✝️ 🇺🇸 Mike Compton 🇺🇸 ✝️ ✓
@MikeCompton
81.6K followers
· 146.5K posts
amit
@AmitOffline
81.6K followers
· 40.8K posts
K.G.F
@KGFTheFilm
81.6K followers
· 2.1K posts
Akasa Air ✓
@AkasaAir
81.6K followers
· 55.7K posts
Voice Of Levant ✓
@VoiceOfLevant
81.6K followers
· 56.9K posts
けいきちゃんねる ✓
@keikiww
81.6K followers
· 24.5K posts
Porttada ✓
@porttada
81.6K followers
· 156.4K posts
MarkZ ✓
@originalmarkz
81.6K followers
· 8K posts
Frequently asked questions about @andrewgwils
How many followers does @andrewgwils have?
40 793 followers.
How many posts has @andrewgwils published?
3 483 posts.
How do posts by @andrewgwils perform?
Collected posts: 24. Views: 854 488. Likes: 6 315.