Martian ✓

@withmartian

Understanding Intelligence. Measurement. Explanation. Application. That's how we're tackling AI interpretability: the greatest scientific problem of our age.

3 786Followers

14Following

433Posts total

347.6KViews on collected posts

Derniers posts

Martian ✓@withmartian3 Sep

@withmartian Thats fantastic!! Gj 3.2K views · 14 likes · 0 reposts · 0 replies Open on X →

Martian ✓@withmartian3 Sep

We all knows the behaviour of LLMs can be a little random sometimes, so we introduce the concept of LLM Reliability. A measuring indicating how consistent an LLM is in successfully solving a problem, enabling a more reliable user experience. 710 views · 19 likes · 0 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

LLM providers charge you for input, thinking, and output tokens yet you only control one of them. Some LLMs may have higher quoted token costs but are more efficient, answering the prompt with less thinking and fewer output tokens, resulting in a lower real cost. See real LLM htt 735 views · 21 likes · 0 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

This research has was selected to for oral presentation at The International Conference of Learning Representations - https://t.co/txd97GHt0r 611 views · 20 likes · 1 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

Analyse the behaviour of any LLM with our `Single Model Insights` view, seeing examples where the LLM is uniquely good and bad compared to it's peers. https://t.co/neGzj8kmnK 864 views · 20 likes · 0 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

We got 46% fewer errors than the single best LLM across the 16 most used benchmarks (TerminalBench, LiveCodeBench, etc). Here's how that's possible and what each model can achieve when used optimally (every benchmarks misses the majority of model capabilities) 👇 Interactive htt 338.1K views · 220 likes · 22 reposts · 58 replies Open on X →

Martian ✓@withmartian3 Sep

Our insights: ✦ The best LLM system is often not the best LLM. ✦ Advertised token costs are bad predictors of real LLM cost. ✦ LLMs have variable reliability in consistently solving a problem. ✦ Single-model benchmarks are systematically understating current LLM capabilities. 2.1K views · 31 likes · 0 reposts · 5 replies Open on X →

Martian ✓@withmartian3 Sep

We call this analysis the Capability Frontier: A Pareto frontier of Quality vs Cost you can achieve by routing between models instead of locking into one. https://t.co/9mrRUU2OQE 1.3K views · 23 likes · 0 reposts · 1 replies Open on X →

Face aux comptes de taille comparable

8 posts des 90 derniers jours, à côté de la tranche de under 10K abonnés. touche moins de monde que ses pairs, mais les engage bien davantage.

Vues médianes1 068ce compte4 175médiane pour under 10K

Portée, %28.21%ce compte250.82%médiane pour under 10K

Engagement, %2.16%ce compte1.41%médiane pour under 10K

Indicateur	Ce compte	Médiane pour under 10K	Rapport
Vues médianes par post	1 068	4 175	0.26×
Portée (vues ÷ abonnés)	28.21%	2.5× audience	0.11×
Taux d'engagement	2.16%	1.41%	1.53×

Autres comptes de cette tranche → Comparer avec un autre compte → Comment ces repères sont établis →

Growth & engagement

How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.

Views per post

1.3K3 Sep

2.1K

338.1K

864

611

735

710

3.2K

Last 8 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.

Engagement rate per post

1.89%3 Sep

1.72%

0.11%

2.43%

3.60%

2.99%

2.82%

0.44%

Reactions — likes, reposts, replies and quotes — divided by views. Median for under 10K accounts is 1.41%.

What the audience does

Likes56.1%368 in total

Reposts3.5%23 in total

Replies10.4%68 in total

Quotes8.7%57 in total

Bookmarks21.3%140 in total

Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.

The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.

Derniers posts

Face aux comptes de taille comparable

Growth & engagement

Views per post

Engagement rate per post

What the audience does

Comptes similaires