Martian ✓

@withmartian

Understanding Intelligence. Measurement. Explanation. Application. That's how we're tackling AI interpretability: the greatest scientific problem of our age.

3 786Followers

14Following

433Posts total

347.6KViews on collected posts

Последние посты

Martian ✓@withmartian3 Sep

@withmartian Thats fantastic!! Gj 3.2K views · 14 likes · 0 reposts · 0 replies Open on X →

Martian ✓@withmartian3 Sep

We all knows the behaviour of LLMs can be a little random sometimes, so we introduce the concept of LLM Reliability. A measuring indicating how consistent an LLM is in successfully solving a problem, enabling a more reliable user experience. 710 views · 19 likes · 0 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

LLM providers charge you for input, thinking, and output tokens yet you only control one of them. Some LLMs may have higher quoted token costs but are more efficient, answering the prompt with less thinking and fewer output tokens, resulting in a lower real cost. See real LLM htt 735 views · 21 likes · 0 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

This research has was selected to for oral presentation at The International Conference of Learning Representations - https://t.co/txd97GHt0r 611 views · 20 likes · 1 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

Analyse the behaviour of any LLM with our `Single Model Insights` view, seeing examples where the LLM is uniquely good and bad compared to it's peers. https://t.co/neGzj8kmnK 864 views · 20 likes · 0 reposts · 1 replies Open on X →

Martian ✓@withmartian3 Sep

We got 46% fewer errors than the single best LLM across the 16 most used benchmarks (TerminalBench, LiveCodeBench, etc). Here's how that's possible and what each model can achieve when used optimally (every benchmarks misses the majority of model capabilities) 👇 Interactive htt 338.1K views · 220 likes · 22 reposts · 58 replies Open on X →

Martian ✓@withmartian3 Sep

Our insights: ✦ The best LLM system is often not the best LLM. ✦ Advertised token costs are bad predictors of real LLM cost. ✦ LLMs have variable reliability in consistently solving a problem. ✦ Single-model benchmarks are systematically understating current LLM capabilities. 2.1K views · 31 likes · 0 reposts · 5 replies Open on X →

Martian ✓@withmartian3 Sep

We call this analysis the Capability Frontier: A Pareto frontier of Quality vs Cost you can achieve by routing between models instead of locking into one. https://t.co/9mrRUU2OQE 1.3K views · 23 likes · 0 reposts · 1 replies Open on X →

На фоне аккаунтов своего размера

8 постов за последние 90 дней рядом с диапазоном under 10K подписчиков. доходит до меньшего числа людей, зато вовлекает их куда сильнее.

Медианные просмотры1 068этот аккаунт4 175медиана для under 10K

Охват, %28.21%этот аккаунт250.82%медиана для under 10K

Вовлечённость, %2.16%этот аккаунт1.41%медиана для under 10K

Показатель	Этот аккаунт	Медиана для under 10K	Отношение
Медианные просмотры на пост	1 068	4 175	0.26×
Охват (просмотры ÷ подписчики)	28.21%	2.5× audience	0.11×
Вовлечённость	2.16%	1.41%	1.53×

Другие в этом диапазоне → Сравнить с другим аккаунтом → Как считаются эти ориентиры →

Growth & engagement

How the posts we collected actually performed: views and reaction rate post by post, what the audience did with them, and where the follower count goes.

Views per post

1.3K3 Sep

2.1K

338.1K

864

611

735

710

3.2K

Last 8 collected posts, oldest on the left. The scale is logarithmic: one post can outrun the rest a hundred times over.

Engagement rate per post

1.89%3 Sep

1.72%

0.11%

2.43%

3.60%

2.99%

2.82%

0.44%

Reactions — likes, reposts, replies and quotes — divided by views. Median for under 10K accounts is 1.41%.

What the audience does

Likes56.1%368 in total

Reposts3.5%23 in total

Replies10.4%68 in total

Quotes8.7%57 in total

Bookmarks21.3%140 in total

Share of every reaction we collected for this account. Replies mean argument, reposts mean endorsement, bookmarks mean the post was worth keeping.

The follower curve appears once this account has two daily snapshots — we take one a day, and this one is on its first.

Последние посты

На фоне аккаунтов своего размера

Growth & engagement

Views per post

Engagement rate per post

What the audience does

Похожие аккаунты