展示 HN: 通过的测试并非证明 – C/C++ 与固件修复的验证器
Show HN: Green tests are not proof – verifier for C/C++ and firmware fixes

原始链接: https://unfix.dev/

Unfix 是一个用于 C/C++ 和固件缺陷修复的独立验证器:它不编写代码,但检查修复是否真正消除了缺陷,并返回 VERIFIED、REJECTED 或 NOT VERIFIABLE。它在基础版本上复现缺陷,应用修复后在启用消毒器的情况下重新运行测试,检查其他测试是否仍然通过,移除修复以确认缺陷重现,并扫描差异以发现诸如禁用消毒器或削弱测试等掩盖问题。裁决来自确定性构建、测试、消毒器和硬件运行——而非 LLM。它支持在真实板卡(如通过 ST-Link 的 STM32 NUCLEO)上的固件,提供 /check 每测试裁决和 /fix 单一裁决。Unfix 以 MCP 工具、终端工具、GitHub PR 检查和硬件测试台在本地运行,无云服务或 LLM。在 10 个 AI 生成补丁的基准测试中,它针对隐藏参考测试验证了 4 个并拒绝了 6 个。它面向 C/C++ 和嵌入式团队,尤其是医疗科技等受监管产品。

Hacker News 最新 | 过往 | 评论 | 提问 | 展示 | 招聘 | 提交 登录 展示 HN:绿色测试不是证据 —— C/C++ 与固件修复的验证器 (unfix.dev) 10 分 由 u-pump 6小时前 | 隐藏 | 过往 | 收藏 | 1 条评论 帮助 Neywiny 0 分钟前 [–] 我仍然无法想象将 LLM 连接到真实硬件,正是因此。 回复 指南 | 常见问题 | 列表 | API | 安全 | 法律 | 申请 YC | 联系 搜索:
相关文章

原文
What is Unfix?

Unfix is an independent verifier for bug fixes in C, C++ and firmware. It does not write code. It checks whether a fix — written by a developer or an AI coding agent — really removes the bug, and returns VERIFIED, REJECTED or NOT VERIFIABLE (when the evidence can't decide).

How does Unfix verify a fix?

It reproduces the bug on the base version (it must fail every run), applies the fix and re-runs the tests with sanitizers, checks that other tests still pass, then removes the fix and confirms the bug comes back. It also scans the diff for masking: disabled sanitizers, weakened or deleted tests, added delays.

Why "unfix"?

Because the strongest proof that a fix works is to take it out again: if the bug does not come back when the fix is removed, the test never proved anything.

Does Unfix use an LLM to decide the verdict?

No. The verdict comes from builds, tests, sanitizers and hardware runs. Deterministic rules, no LLM in the verdict. Flaky and race bugs are judged by repeated runs, not a single pass.

Does it work with firmware and real boards?

Yes. Unfix can build firmware, flash a real board (today STM32 NUCLEO via ST-Link) and run the tests on the hardware. /check gives per-test verdicts on your rig: FIXED (failed before the change, passes now), BROKEN (passed before, fails with the change), FLAKY ON THE BENCH (unstable without the change too; fix the rig before trusting any result) and FLAKY, WORSE (flaky before, significantly worse with the change). For a single bug fix, /fix gives one verdict: VERIFIED, REJECTED or NOT VERIFIABLE.

Where does Unfix run?

As an MCP tool inside AI coding agents such as Claude Code and Codex, as a GitHub pull request check that posts the verdict as a comment (shown on our demo repository; not yet available for partner repositories), in the terminal, and on your hardware rig. Built for closed networks. No cloud, no LLM — Unfix runs entirely on your machines. After a one-time setup, builds and tests run with the network off.

What did the public benchmark show?

We tested 10 AI-generated C/C++ patches against real bugs from open-source projects (fmt, spdlog, zlib-ng, cpp-peglib). Judged against the maintainers' own hidden reference tests, Unfix verified 4 and rejected 6.

Who is Unfix for?

C/C++ and embedded teams of roughly 20–200 engineers, starting with medtech and other regulated products where "the agent said so" is not an acceptable answer.

联系我们 contact @ memedata.com