清晰度没用,改试神秘主义。
Clarity didn't work, trying mysterianism (2012)

原始链接: https://gwern.net/doc/fiction/science-fiction/2012-10-03-yvain-thewhisperingearring.html

这场讨论探讨了诸如“耳环”或“因果眼镜”这类通过指定最优选择来优化人类生活的设备所带来的哲学影响。参与者权衡了个人自主权与由更高级的预测性智能来规划生活所带来的效率之间的利弊。 一个主要的担忧是,这些工具究竟是促进了真正的成长,还是仅仅将使用者变成了“哲学僵尸”或某种异质目标系统的傀儡。虽然有人认为依赖这种指导等同于在生活中“作弊”,剥夺了人们发现的乐趣和认知锻炼的机会;但也有人认为,这仅仅是我们现有缺陷的生物过滤器的一种技术进化。通过调整我们的感知以更好地适应现实,这些工具或许可以作为一种“学习辅助”,或者绕过过时的进化偏差。 这场对话最终突显了对顺遂人生渴望与对丧失“自我”的生存恐惧之间的张力。如果一股外力能比你更有效地达成目标,那么个人成就与成为一个被精心管理的工具之间的界限就会变得极其模糊。

```Hacker News最新 | 过往 | 评论 | 提问 | 展示 | 招聘 | 提交登录透明度行不通,尝试神秘主义 (gwern.net)15 分,由 dboon 发布于 2 小时前 | 隐藏 | 过往 | 收藏 | 2 条评论 帮助 avaer 5 分钟前 | 下一条 [–] 这是关于人工智能如何接管世界最引人入胜(也最可怕)的故事之一。如果你喜欢这个,也请看看我们的 Pluribus。回复mmooss 2 分钟前 | 上一条 [–] (2012) 也在 archive.org 上:https://web.archive.org/web/20131230055512/http://squid314.l... 有人知道这个故事的来源吗?有一条评论在回复某人建议发表它时说:> 问题在于它已经被发表了,所以他只能出售二次版权。回复 考虑申请 YC 2026 秋季批次!申请截止日期为 7 月 27 日。 准则 | 常见问题 | 列表 | API | 安全 | 法律 | 申请 YC | 联系 搜索:```
相关文章

原文
This post gave me hope. Every time I try to really write out my idea of the future I'm really scared of, I end up getting it wrong in some way. I was afraid for a moment that you'd done it with little effort in a random blog post, but then your earring ended up making people brainless which is obviously villainous and terrible and a clear reason to simply never put it on in the first place, and so the earring immediately became trivially avoidable and therefore non-scary. If you can't quite do it right, at least not yet, then maybe my inability is less because of my lack of creativity and more because it actually is a difficult idea.

This will be getting it wrong, too, but I'm going to write only a couple of paragraphs so that the ways I'll get it wrong will at least be excusable by the lack of apparent effort.

What if you could trust your hunch about people? What if, at least according to your perception, you could trust a person with well-fitting, clean, situation-appropriate clothes to make a good person to do business with while a huckster almost always looked like a huckster to you? What if there were pretty people in the world, but people who were available and who would make a good romantic partner to you just looked more glowingly beautiful than anyone else? What if people pitching to you ideas that would make bad career moves had annoying voices? What if, while fast food was decent, salad tasted really amazing?

Say you could get Consequence Glasses that do just that. Say they even have a tuning knob. It goes from 0 to 1 because I am a geek. At 0 you get your natural perception with its natural biases, at 1 things probably will have to get a little bit abstract because so much becomes just a representation of what kind of consequences it leads to. Maybe it even works on some kind of a curve, so that if you just use them at, say, 0.1, it simply protects you from whatever tricks you're most vulnerable to - your most sinful foods taste just a bit stale but not in such a way that it makes you less happy, the well-designed rewards of videogames that keep you playing seem just a bit more arbitrary and distant, etc.. Only once you turn it higher does it really start directing you toward anything specific.

You can decide to just not put the glasses on at all. You can decide to only run them at a very low level. Every turn of the knob is precisely your own choice. It's simply that the higher you turn it, the easier it will be to just do the right decisions in every situation. It won't even feel like someone else is in charge - it's you calling the shots, calling them as you see them. Hell, you were already wearing consequence glasses anyway, it's just that they're biased for inclusive fitness in mankind's ancestral environment, not for happiness and fulfillment in the modern world. Your existing emotional filters were never designed for the stimuli and for the consequences of this environment - changing your perception only makes them work better. It makes you into a more formidable person. A person with an easier life, yes, but only because your decisions are more grounded in reality, not less!

Yes, you will become dependent on the glasses, just as you would on the earring. They're probably eventually even going to cause changes to your brain structure as you stop worrying so much about whether people are lying to you, about whether you're making the best possible use of your time, etc.. But it's a lot, lot harder to see them as taking anything away from you. Certainly you can't just brush it off with a generic "things should not be that convenient" type of argument.

I do worry that the only way to figure out whether you should wear the glasses is to put them on and see how they look in a mirror...

Edited at 2012-10-04 07:40 am (UTC)

Have you read Ted Chiang's Liking what you see? The premise is a little similar, and it's a great story.
From: (Anonymous)
2012-10-04 08:00 pm (UTC)

(Link)

You know, at first this seemed a bit scary...but then I had to ask: what if your goals are defined in terms of your cognition? IE, for example, I want be a famous and successful scientist, and I define "successful" as having unparalleled actual mastery of the art? Or if, in a more mundane example, I want to solve a Sudoku puzzle, and I define "having solved the puzzle" in a way that requires me to do the actual cognition, and derive the benefit of having solved the puzzle without being 'spoiled'? What sort of advice can the WE give? I'm sure it can give advice, and that the advice is useful...but if, as a result, I derive the benefit of the mental exercise that I wanted, I'm not entirely sure that this is a problem.

There's a couple of ways read this. One is that the WE doesn't actually help you implement your 'real' goals, but the second or third order subgoals and that its first bit of advice, "take me off", was warning you about precisely this effect. Or you could read it as a warning about defining goals in such a way as to allow that being turned into a brainless puppet of an alien horror would accomplish those goal (if you think being turned into a brainless puppet would be bad).

Another random thought - it seems that the second-stage WE has effectively replaced my lower-level functions with an extremely smart but alien intelligence that does everything better than I could do myself...the thing is, I'm not entirely convinced that this isn't already the case in reality. Certainly, if I tried to, say, catch a ball running on only my higher-level cognition that I consider "me", I'd fail utterly.

联系我们 contact @ memedata.com