Show HN：Parsimony —— 一个衡量 AI 模型新增多少代码的开源基准

Show HN：Parsimony —— 一个衡量 AI 模型新增多少代码的开源基准
Show HN: Parsimony – an open-source benchmark for how much code AI models add

Parsimony 是一个衡量 AI 编程智能体中多余代码的基准。与解决率/成本指标不同，它衡量净编程单元：语句、调用、比较、名称、字面量等语法元素。它减去删除的代码（净 = 新增 − 删除），并避免按行计数，因此堆砌代码不会得到奖励。模型按其在已解决任务上新增的平均净单元排名，从低到高；失败尝试也计入平均值。已解决数据来自 SWE-bench Verified，不重新运行测试。补丁会被应用，变更文件会被解析；仅限 Python，排除测试、文档、生成文件。它使用相同的 mini-SWE-agent，不过智能体会影响输出。更小并不总是更好：单元衡量的是规模，而非可读性/设计，而且写更多代码的模型往往能解决更多任务。用户可以比较模型、查看排行榜，或通过示例 JSON 提交结果。代码/数据：github.com/La5u/Parsimony。

Hacker News 的“Show HN”栏目介绍了一个名为 Parsimony 的开源基准，用于衡量 AI 模型新增了多少代码。现有基准会追踪解题率和 token 成本，但不会惩罚多余代码，而多余代码会让代码库更难维护。Parsimony 分析来自 DeepSWE、SWE-bench Verified 和 SWE-bench Live 的补丁，统计“编码单元”（如语句、调用、名称和字面量），而不是代码行数，并减去删除量，以衡量净新增量。GPT-6 和 Claude 5.5 没有公开补丁，因此作者亲自在 3–10 个任务上运行了它们；结果显示为淡化的、未排名的圆圈。欢迎反馈和公开补丁导出。评论者讨论了把圈复杂度作为可读性指标、减少智能体生成内容冗长程度的方法，以及担忧当前领先者 Astra 是靠难以阅读的极简“代码高尔夫”取胜。其他人则争论冗长与简洁之间的取舍，并希望有面向 Qwen 3.8 27B 等可本地运行开源模型的基准。

Parsimony: which AI coding agent changes the least code?

Existing benchmarks measure solve rate and cost, but don't punish excess code, which makes a codebase harder to work with in the long term. Parsimony measures how much code AI models add.

It takes the patches that benchmarks already publish and counts coding units: syntax elements like statements, calls, comparisons, names and literals. Lines aren't counted, so cramming code isn't rewarded, and deleted code is subtracted: net units = added − removed.

Compare models

Chart

X axis

How to read this

Models are ranked by mean net units added on the tasks they solved, lowest first. "Net units added (mean)" also includes failed attempts. Solved comes from SWE-bench Verified; tests aren't rerun. Measured is how many solved attempts could be measured.

Leaderboard

Task

Method

Each patch is applied to the task's code, and both versions of every changed file are parsed. Nothing is run.
Tests, docs and generated files are left out. Only Python is measured.
Pass/fail comes from SWE-bench's published results; tests aren't rerun.

Limits

Smaller isn't always better: units measure size, not readability or design.
Models that write more code also tend to solve more tasks (Spearman 0.33–0.60 on the boards with 26+ models).
Every model uses the same agent (mini-SWE-agent), but the agent still affects how much code gets written.

Add your own benchmark

Have code-size numbers from another benchmark? Download the example file, fill in your results and open it to get the same chart and table. The file stays in your browser. Format and how to share it.

Open benchmark JSON

Example: check for a negative number

Both return the same result.

def has_negative(values):
    found = False
    for value in values:
        if value < 0:
            found = True
    return found

def has_negative(values):
    return any(value < 0 for value in values)

Show HN：Parsimony —— 一个衡量 AI 模型新增多少代码的开源基准 Show HN: Parsimony – an open-source benchmark for how much code AI models add

Show HN：Parsimony —— 一个衡量 AI 模型新增多少代码的开源基准
Show HN: Parsimony – an open-source benchmark for how much code AI models add