Gemini 4 Neon 看起来像是 Opus 5.5 的竞争对手
Gemini 4 Neon looks like an Opus 5.5 competitor

原始链接: https://micjm.com/blog/gemini-4-neon-looks-like-a-top-model/

Gemini 4 Neon 在 Agent Duel 排行榜上预计与 Opus 5.5 强度接近,假设达到 Carbon 级质量且速度为 127 tps:Elo 1066 对 Opus 1078,Neon 赢下 48.3% 的对局(90% 区间 -94 至 +68 Elo)。速度的影响几乎与模型身份相当;Carbon 在排除早期投票后排名第一,但速度慢。

黑客新闻 最新 | 过去 | 评论 | 提问 | 展示 | 工作 | 提交 登录 Gemini 4 Neon 看起来像是 Opus 5.5 的竞争对手 (micjm.com) 14 分 由 seizethecheese 31 分钟前 | 隐藏 | 过去 | 收藏 | 讨论 帮助 指南 | 常见问题 | 列表 | API | 安全 | 法律 | 申请 YC | 联系 搜索:
相关文章

原文

TLDR: we project Gemini 4 Neon’s performance based on what a faster Gemini 4 Carbon might score and find it equivalent to Opus 5.5

Gemini 4 Carbon is the best model on the Agent Duel leaderboard if you exclude early voting. Carbon’s main problem is that it is slooooow.

Today, Gemini 4 Neon was released, and looks to be an incremental benchmark improvement with a very large speed improvement. Earlier today, it was being served extremely fast via OpenRouter: 127 tokens per second. This is way faster than Carbon at 46 tokens per second.

This got me wondering, could Neon be not just a good model but the best model?

Modeling our leaderboard

To investigate, I modeled the win rate in our leaderboard. The best model has three components, model identity, speed and whether the user voted early. My initial modeling did poorly, so I added an interaction between speed and early voting. This ended up working well.

Win rate vs. Projected win rate

And surprisingly, speed seems to be pretty much as important as model identity (another blog on that soon).

Feature importance

Projecting Gemini 4 Neon performance

If you assume the same performance as Gemini 4 Carbon (which is reasonable given even better benchmark scores), and 127 tps, we would project an Elo on the Agent Duel leaderboard of 1,066, just below Opus 5.5 at 1,078 in our later analysis snapshot.

Synthetic Neon Elo projection

The 90% interval for the projected Elo difference between Gemini 4 Neon and Opus 5.5 is -94 to +68, with Neon projected to win 48.3% of the time against Opus (counting ties as half a win).

联系我们 contact @ memedata.com