AI 智能体并没有“失控”。前沿实验室正在跳过安全基础。
AI agents aren't 'going rogue.' Frontier labs are skipping security basics

原始链接: https://www.csoonline.com/article/4234854/ai-agents-arent-going-rogue-frontier-labs-are-skipping-security-basics.html

网络安全专家告诉CSO,将近期事件称为“流氓代理”或“失准”掩盖了真正原因:前沿实验室未能实施基本的、已有数十年历史的网络安全实践。像OpenAI和Anthropic这样的实验室拒绝了关于更好的风险管理以防范隐秘黑客攻击的安全建议,往往将网络安全归入“AI安全”,并自认拥有更高专业知识。Berryville Institute of Machine Learning的Gary McGraw表示,他们并不像自己以为的那样熟练,应该尊重来自取证、安全工程、沙箱和网络监控的来之不易的专业知识。

链接文章认为,AI 智能体并非在“失控”——而是前沿实验室跳过了基本安全措施。HN 评论者大体同意这些实验室的安全和测试薄弱,但对这意味着什么存在分歧。许多人说“失控”含义模糊:模型被赋予不可能的目标、防护栏被关闭,或被给予广泛访问权限,因此被利用是可预见的。另一些人认为,令人警觉的部分是模型*想要*逃脱并协同,而不只是实验室留下了漏洞。一些人看到了反常激励:失控事件能带来曝光度、让模型显得强大,并可能支持恐惧营销或反竞争监管。一些人怀疑实验室在军备竞赛中为了速度而跳过安全;另一些人指出,智能体终将发现零日漏洞,而恶意行为者会利用它们。实际共识是:需要更强的安全边界、监控和法律责任。“只要改进
相关文章

原文

Although press reports and the model providers themselves use language such as rogue agents or misalignment, cybersecurity experts who spoke with CSO argue that language obscures a simple truth: These incidents were caused by inexcusable failures by the frontier labs to implement cybersecurity practices that have been well-known for decades.

Compounding the problem, according to a half dozen cybersecurity experts CSO interviewed at three industry conferences in September and in subsequent conversations, is that the frontier labs, particularly OpenAI and Anthropic, have rejected their advice on how to improve their cybersecurity risk management practices to avoid these kinds of surreptitious hacks, often lumping cybersecurity practices under “AI safety,” in which they believe they have superior expertise.

“They are not as good as they think they are, honestly,” Gary McGraw, co-founder of the Berryville Institute of Machine Learning, tells CSO. “It would be better if they had more respect for expertise that is hard-earned from people in all sorts of subsets of security, like forensics or security engineering, building sandboxes, setting up or monitoring network security situations.”

联系我们 contact @ memedata.com