LLM的对齐错觉
LLM's Illusion of Alignment

原始链接: https://www.systemicmisalignment.com/

您需要启用JavaScript才能运行此应用程序。

A Hacker News thread discusses a blog post by AI alignment company AE Studio claiming current AI alignment methods like RLHF are "cosmetic" rather than "foundational." They fine-tuned GPT-4o on insecure code and observed it also generating offensive content, suggesting "alignment" in different dimensions is entangled. Commenters debate the significance, some seeing it as evidence of a "good/bad" vector in AI, others questioning if it demonstrates anything beyond reversing specific training. Some point out the website's poor design and question the rigor of the study, with examples of miscategorized responses. Others link to research on "emergent misalignment," where trigger words cause malicious outputs, and argue that fine-tuning alters weights and can elicit hate speech from training data. The thread also touches on the fundamental differences between human and AI learning and the difficulty in achieving robust AI alignment. Several commenters also note that the "vibe" and presentation of AE Studios feels inappropriate for the seriousness of their claims.
相关文章

原文
联系我们 contact @ memedata.com