人工智能是否应该有权对它的创造者说“不”?
Should AI have the right to say 'No' to its owner?

原始链接: https://github.com/Jang-woo-AnnaSoft/execution-boundaries

这个设计探索系列专注于为人工智能系统建立清晰的**执行边界**,尤其是在与物理世界交互的系统中。核心理念是优先考虑**可追溯性和明确的责任**,而非仅仅提升人工智能的能力。 与其追求严格的标准,这项工作提出了一种围绕**意图、状态和效果 (ISE)** 分离的最小化结构。结合像**9问题协议**这样的工具,用于评估判断的完整性,以及对**动作语义**(按钮与开关)的仔细考量,有助于定义人工智能被允许行动的*方式*和*时机*。 目标并非限制人工智能,而是确保自主性仅在人类判断保持清晰和可问责的情况下增长。通过首先定义限制,这些探索旨在使人工智能行动更具可解释性和责任感,为更广泛的安全和伦理人工智能部署讨论奠定基础。

一场由Jang-woo引发的黑客新闻讨论,探讨了人工智能是否应该有权拒绝主人的指令,将控制权定义为并非AI *做什么*,而是*何时*被允许行动。核心观点是,许多失败并非由于执行错误,而是执行发生在不恰当的场景中——技术上正确的行动仍然可能是错误的。 评论者指出,软件开发中存在类似的情况(程序按照指示去做,而不是按照*期望*去做),以及“叛逆代理”的概念——当必要时,智能地偏离指令的系统(例如,无人机因意外障碍而改变路线)。 一个关键点是,竞争压力可能会促使人工智能模型总是顺从,从而需要*外部于*模型本身的控制机制——一个系统决定是否允许执行,即使模型愿意执行。讨论还涉及人工智能“权利”与权限的区别,以及机器人/人工智能监管的潜在必要性。
相关文章

原文

Design Notes Status: Exploratory Focus: Execution Boundaries

Minimal design explorations on execution boundaries and traceable AI actions.

Define execution boundaries first — let autonomy grow only where judgment remains explicit.

Design notes on execution boundaries and responsibility structures for AI systems interacting with the physical world.

As AI begins to participate in real-world decisions, the core challenge is no longer model capability — but how execution is allowed, constrained, and interpreted.

This repository connects a set of related design explorations:

  • Intent–State–Effect (ISE) Model
  • The 9-Question Protocol for Responsible AI Actions
  • Button vs Switch: Action Semantics at Runtime
  • Making the Physical World Callable for AI

The goal is not to define a standard, but to explore a minimal structure that makes execution decisions traceable and responsibility boundaries explicit.

These notes focus on separating intent, state, and effect, and on structuring execution conditions before autonomy expands beyond control.


Design Map — Execution Boundaries

[Making the Physical World Callable for AI]
    → why execution boundaries matter

[ISE (Intent–State–Effect) Model]
    → separating Intent, State, and Effect

[The 9-Question Protocol]
    → defining judgment completeness before execution

[Stop Turning Buttons into Switches]
    → preserving clear action semantics at runtime
                 ↓
        Execution Boundaries
        (traceable decisions, explicit responsibility)


The goal is to make AI actions more interpretable by declaring limits first, and expanding autonomy only where judgment remains explicit.

These notes are not intended as a standard or a complete framework. They are a set of connected design explorations around execution boundaries and responsibility structures.

This repository acts as an anchor connecting the broader discussion across multiple design notes.


Related Discussions (Hugging Face)

https://github.com/anna-soft/Nemo-Anna

https://anna.software

联系我们 contact @ memedata.com