通过 500 美元对 9B 开源模型进行强化学习微调,在目录审查任务中超越了前沿模型。
A $500 RL fine-tune of a 9B open model beat frontier models on catalog review

原始链接: https://fermisense.com/when-machines-take-the-wheel/

为了确保模型通过经验而非仅仅通过观察进行学习,我们构建了一个电商平台的数字孪生系统。利用亚马逊伯克利对象数据集(Amazon Berkeley Objects dataset),我们生成了 177,767 个包含真实产品数据的审核片段,并策略性地植入了“困难负样本”(例如图片不匹配或品牌声明冲突),从而打造了一个严苛的测试环境。 在此仿真中,模型像人类分析师一样工作:它在 13,000 个类目的分类体系中进行检索,核实品牌保护信息,提取必要的属性,并最终做出正式的审核决策。评分系统会对每一次操作进行评估,奖励准确行为并惩罚错误行为。为了使模型与业务优先级保持一致,评分系统设定了权重,对漏报违规的惩罚是误报的七倍。通过模拟真实平台的风险、工具和工作流程,模型能够不断地进行练习、试错,并提升其决策能力。

Hacker News 最新 | 往期 | 评论 | 提问 | 展示 | 招聘 | 提交 登录 一个花费 500 美元通过强化学习微调的 9B 开源模型在目录审查任务中击败了前沿模型 (fermisense.com) 5 分,由 ilreb 发布于 41 分钟前 | 隐藏 | 往期 | 收藏 | 讨论 | 帮助 考虑申请 YC 2026 年秋季批次!申请截止日期为 7 月 27 日。 准则 | 常见问题 | 列表 | API | 安全 | 法律 | 申请 YC | 联系 搜索:
相关文章

原文

Part IV

A model can only practice a workflow it can actually perform, fail at, and retry. So we rebuilt the catalog workflow as a digital twin of an e-commerce platform: a simulated environment with the same listings, the same tools, and the same stakes as the real thing. The raw material is the Amazon Berkeley Objects dataset of real product images and listing data, which we turned into 177,767 review episodes: one listing each, with image, title, description, claimed brand, and region. Into that stream we planted controlled policy cases, mismatched images, conflicting brand claims, and deliberately legitimate claims as hard negatives, so every episode has a known correct answer to score against.

Inside the twin, the model works the way an analyst would. It can search a taxonomy of roughly 13,000 categories, check whether a brand is registered and protected, and retrieve the required attributes for a chosen category; then it must commit a category, the attributes, and a policy decision. A scorer grades every episode, rewarding correct outputs and penalizing missed violations, unsupported attributes, invalid categories, and wasted tool calls. The penalties encode business priorities directly: a missed violation costs 7× more than a false alarm.

联系我们 contact @ memedata.com