Martian ✓

@withmartian

Understanding Intelligence. Measurement. Explanation. Application. That's how we're tackling AI interpretability: the greatest scientific problem of our age.

3 786Followers

14Following

433Posts total

347.6KViews on collected posts

Son gönderiler

Martian ✓@withmartian3 Sep

@withmartian Thats fantastic!! Gj 3.2K views · 14 likes · 0 reposts · 0 replies Open on X →

Martian ✓@withmartian3 Sep

We all knows the behaviour of LLMs can be a little random sometimes, so we introduce the concept of LLM Reliability. A measuring indicating how consistent an LLM is in successfully solving a problem, enabling a more reliable user experience. 710 views · 19 likes · 0 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

LLM providers charge you for input, thinking, and output tokens yet you only control one of them. Some LLMs may have higher quoted token costs but are more efficient, answering the prompt with less thinking and fewer output tokens, resulting in a lower real cost. See real LLM htt 735 views · 21 likes · 0 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

This research has was selected to for oral presentation at The International Conference of Learning Representations - https://t.co/txd97GHt0r 611 views · 20 likes · 1 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

Analyse the behaviour of any LLM with our `Single Model Insights` view, seeing examples where the LLM is uniquely good and bad compared to it's peers. https://t.co/neGzj8kmnK 864 views · 20 likes · 0 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

We got 46% fewer errors than the single best LLM across the 16 most used benchmarks (TerminalBench, LiveCodeBench, etc). Here's how that's possible and what each model can achieve when used optimally (every benchmarks misses the majority of model capabilities) 👇 Interactive htt 338.1K views · 220 likes · 22 reposts · 58 replies Open on X →

Martian ✓@withmartian3 Sep

Our insights: ✦ The best LLM system is often not the best LLM. ✦ Advertised token costs are bad predictors of real LLM cost. ✦ LLMs have variable reliability in consistently solving a problem. ✦ Single-model benchmarks are systematically understating current LLM capabilities. 2.1K views · 31 likes · 0 reposts · 5 replies Open on X →

Martian ✓@withmartian3 Sep

We call this analysis the Capability Frontier: A Pareto frontier of Quality vs Cost you can achieve by routing between models instead of locking into one. https://t.co/9mrRUU2OQE 1.3K views · 23 likes · 0 reposts · 1 replies Open on X →

Aynı büyüklükteki hesaplara karşı

Son 90 güne ait 8 gönderi, under 10K takipçi aralığıyla yan yana. benzerlerinden daha az kişiye ulaşıyor, ama onları çok daha güçlü bağlıyor.

Medyan görüntülenme1 068bu hesap4 175under 10K için medyan

Erişim, %28.21%bu hesap250.82%under 10K için medyan

Etkileşim, %2.16%bu hesap1.41%under 10K için medyan

Ölçüt	Bu hesap	under 10K için medyan	Oran
Gönderi başına medyan görüntülenme	1 068	4 175	0.26×
Erişim (görüntülenme ÷ takipçi)	28.21%	2.5× audience	0.11×
Etkileşim oranı	2.16%	1.41%	1.53×

Bu aralıktaki diğer hesaplar → Başka bir hesapla karşılaştır → Bu kıyas değerleri nasıl kuruluyor →

Growth & engagement

How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.

Views per post

1.3K3 Sep

2.1K

338.1K

864

611

735

710

3.2K

Last 8 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.

Engagement rate per post

1.89%3 Sep

1.72%

0.11%

2.43%

3.60%

2.99%

2.82%

0.44%

Reactions — likes, reposts, replies and quotes — divided by views. Median for under 10K accounts is 1.41%.

What the audience does

Likes56.1%368 in total

Reposts3.5%23 in total

Replies10.4%68 in total

Quotes8.7%57 in total

Bookmarks21.3%140 in total

Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.

The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.

Son gönderiler

Aynı büyüklükteki hesaplara karşı

Growth & engagement

Views per post

Engagement rate per post

What the audience does

Benzer hesaplar