Andrew Gordon Wilson ✓
@andrewgwils · New York, NY · joined 09 Sep 2014
Research Lead @perplexity_ai. Professor of machine learning at NYU Courant.
40 725Followers
1 183Following
3 471Posts total
642KViews on collected posts
Ultimi post
Cerchiamo nuovi post…
@andrewgwils queued up to watch this, epiplexity looks super interesting
525 views · 1 likes
· 0 reposts · 0 replies
Open on X →
Doing one thing really well carries so much more value than doing five things at an average level.
7.5K views · 137 likes
· 7 reposts · 5 replies
Open on X →
@andrewgwils I'll have a look. I've long said, that is since I began in NN 1987-1994, that there's no such thing as generalization.
583 views · 1 likes
· 0 reposts · 0 replies
Open on X →
I had a great time presenting the "Foundations of Modern AI" at the Berkeley Deep Learning for Science Summer School. The talk covered a prescriptive theory of generalization and epiplexity. Video now online! https://t.co/muxf74g8mo
81.7K views · 588 likes
· 80 reposts · 10 replies
Open on X →
With NeurIPS registration sold out weeks before even author notification, I'm worried the conference will simply be unrecognizable. Basically no community regulars will have registered -- they won't have been using agents to register within minutes of opening.
73.1K views · 313 likes
· 15 reposts · 5 replies
Open on X →
As much as AI is impressively progressing, current systems are appallingly poor at conducting end-to-end scientific research. 1/4
49.1K views · 224 likes
· 16 reposts · 13 replies
Open on X →
@andrewgwils I am a fan of soft inductive bias (like those employed in PAC-Bayes, union bounds, their ilk). I think the challenge is accounting for the last O(1)% of generalization using it, at least in vision models. This was the goal of attack that Karolina and I started back
2.9K views · 25 likes
· 2 reposts · 0 replies
Open on X →
The textbooks don't need to be rewritten -- they just needed to pay attention to what was already known about generalization, decades ago! I've had thoughts about this for 12 years, and people always ask for the paper -- so I finally wrote it. Thankful to many for feedback! 12/12
5.6K views · 70 likes
· 2 reposts · 4 replies
Open on X →
We also further consider overparametrization and double descent. Deep learning of course _is_ different, and much is not well understood. To this end, we particularly highlight representation learning, mode connectivity, and universal learning. Much more in the paper! 11/12 https
6.2K views · 34 likes
· 1 reposts · 1 replies
Open on X →
Benign overfitting describes perfectly fitting a mix of signal and noise, but still generalizing respectably. Like with the polynomial, we can exactly reproduce this behaviour with a Gaussian process and bound it with PAC-Bayes through the marginal likelihood. 8/12 https://t.co/I
4.8K views · 35 likes
· 0 reposts · 1 replies
Open on X →
These frameworks provide a prescription for generalization, in line with soft inductive biases: embrace a maximally flexible hypothesis space, combined with a compression bias. 9/12
4.9K views · 41 likes
· 1 reposts · 1 replies
Open on X →
Computing the bounds is also very simple: (i) train a model to find hypothesis h*, using any optimizer; (ii) measure the empirical risk R(h*) (e.g., training loss); (iii) measure the filesize of the stored model. 10/12
4.3K views · 34 likes
· 0 reposts · 1 replies
Open on X →
We can use a Solomonoff prior, which represents a maximally overparametrized model, but assigns exponentially higher weights to simpler (shorter) programs with lower Kolmogorov complexity, leading to non-vacuous generalization bounds on even billion parameter neural nets. 7/12 h
5.6K views · 49 likes
· 2 reposts · 2 replies
Open on X →
This model also performs well over a range of data sizes and complexities. Notably, a central observation in "understanding DL requires rethinking generalization" was a neural net can fit noise perfectly, but still generalize on structured data; but this polynomial can too. 5/12
6.2K views · 42 likes
· 1 reposts · 3 replies
Open on X →
While this behaviour cannot be explained by Rademacher complexity or VC dimension (which measures a model's ability to fit noise), it _can_ be described by decades-old countable hypothesis bounds with a prior, which do not penalize the size of the hypothesis space. 6/12 https://t
6K views · 46 likes
· 3 reposts · 1 replies
Open on X →
Let's consider the simplest possible example: a polynomial. We’ll use an arbitrarily high-order polynomial with order-dependent regularization that penalizes the norms of higher order coefficients more. This model scales its complexity as needed, and can also fit pure noise! 4/12
6.5K views · 45 likes
· 1 reposts · 2 replies
Open on X →
Rather than restricting the solutions a model can represent, specify a preference for certain solutions over others, through _soft_ inductive biases. This approach guides us towards structure where it exists, without significant penalty where it doesn't. 3/12 https://t.co/y7flCxm
8.7K views · 65 likes
· 4 reposts · 1 replies
Open on X →
What makes deep learning different? Not overparametrization, benign overfitting, or double descent, which can be reproduced with other models and explained with old generalization frameworks. Understanding DL doesn't require rethinking generalization -- and it never did! 2/12
12.2K views · 73 likes
· 4 reposts · 2 replies
Open on X →
My new paper "Deep Learning is Not So Mysterious or Different": https://t.co/AgHdSQkals. Generalization behaviours in deep learning can be intuitively understood through a notion of soft inductive biases, and formally characterized with countable hypothesis bounds! 1/12 https://t
355K views · 2.2K likes
· 328 reposts · 20 replies
Open on X →
Rispetto ad account della stessa dimensione
7 post degli ultimi 90 giorni, accanto alla fascia di 10K–100K follower. arriva a molti, ma pochi di loro reagiscono.
Visualizzazioni mediane7 474questo account998mediana per 10K–100K
Copertura, %18.35%questo account3.68%mediana per 10K–100K
Interazione, %0.46%questo account1.63%mediana per 10K–100K
| Metrica | Questo account | Mediana per 10K–100K | Rapporto |
|---|---|---|---|
| Visualizzazioni mediane per post | 7 474 | 998 | 7.49× |
| Copertura (visualizzazioni ÷ follower) | 18.35% | 3.68% | 4.99× |
| Tasso di interazione | 0.46% | 1.63% | 0.28× |
Altri account di questa fascia → Confronta con un altro account → Come sono costruiti questi parametri →
Growth & engagement
How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.
Views per post
5.6K5 Mar
4.3K
4.9K
4.8K
6.2K
5.6K
2.9K
49.1K5 Sep
73.1K7 Sep
81.7K
716
583
7.5K
5258 Sep
Last 14 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.
Engagement rate per post
0.95%5 Mar
0.81%
0.89%
0.75%
0.58%
1.35%
0.94%
0.52%5 Sep
0.46%7 Sep
0.83%
0.28%
0.17%
1.99%
0.19%8 Sep
Reactions — likes, reposts, replies and quotes — divided by views. Median for 10K–100K accounts is 1.63%.
What the audience does
Likes50.3%4 061 in total
Reposts5.8%467 in total
Replies0.9%72 in total
Bookmarks43.0%3 473 in total
Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.
The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.
Account simili
Myriam Montemayor
@MyriamMonteCruz
81.5K followers
· 77.4K posts
The JRE Companion
@TheJRECompanion
81.5K followers
· 8.8K posts
Mamas A. Mamas ✓
@mmamas1973
81.4K followers
· 88.1K posts
Nicolas is a cowboy
@niggaolas
81.4K followers
· 209.6K posts
Chitrakoot Police
@chitrakootpol
81.4K followers
· 117.1K posts
SHAPE - NATO Allied Command Operations
@SHAPE_NATO
81.4K followers
· 14.5K posts
ally kirser ✨⚔️ VA | VTUBER ✓
@alekirser
81.4K followers
· 4.6K posts
ELLE Canada
@ElleCanada
81.4K followers
· 61.1K posts