Show HN:扫描你的 AI 智能体,发现潜在的危险能力
Show HN: Scan your AI agents for dangerous capabilities

原始链接: https://github.com/makerchecker/MakerChecker

MakerChecker 是一个治理框架,旨在保护 AI 智能体,且无需更改你现有的基础设施(如 LangChain、CrewAI 等)。它充当工具调用的检查点,强制执行“制作者-核查者”(Maker-Checker)模式,确保智能体无法超越其分配的角色或批准自己的工作。 该系统包含三个主要组件: 1. **`mc scan`**:审核现有的智能体功能并识别高风险操作(如数据删除或敏感信息泄露),自动生成必要的治理代码。 2. **`@makerchecker/embedded`**:提供程序化控制以封装工具,确保未经授权的调用在执行前被拒绝。 3. **自托管服务器**:提供集中式仪表盘、人工审批工作流以及防篡改的审计追踪。每个决策都经过哈希链处理并使用 Ed25519 签名,支持独立的离线验证。 MakerChecker 与框架无关且专注于隐私,核心组件采用 Apache-2.0 许可证,可轻松集成到闭源项目中。它非常适合医疗、金融和监管报告等高风险环境,在这些领域,人工监督以及可验证、不可否认的审计日志对于自动化运营是必不可少的。

抱歉。
相关文章

原文

Your agents keep running in their existing framework (LangChain, Claude SDK, CrewAI). MakerChecker sits in front of every tool call as a checkpoint and behind it as a signed ledger: an agent acts only through a role, runs only the skills it was granted, cannot exceed its limits, and cannot approve its own work.


Find what your agent can already do on its own, classified by risk. No install, nothing leaves your machine:

It flags every consequential action — deleting data, moving money, running shell commands, exfiltrating secrets — names each against the real incident it resembles, and can write the governance code for you with --fix. → packages/scan

2 — Guarantee its behavior

Import the controls and wrap any tool. The agent can now only run what its role was granted — a call it isn't allowed is denied before it executes:

npm i @makerchecker/embedded
import { createGovernor, GovernanceDeniedError } from "@makerchecker/embedded";

const gov = createGovernor()
  .defineSkill("place-order@1", { riskTier: "high" })
  .defineRole("agent")
  .defineRole("risk-desk")
  .grant("risk-desk", "place-order@1")   // the agent is NOT granted it — deny by default
  .defineAgent("trader", "agent");

// Wrap your tool once. Now the agent structurally can't fire it.
const placeOrder = gov.governedTool("trader", "place-order@1", (order) => broker.submit(order));

try {
  await placeOrder({ symbol: "BTC", qty: 10 });
} catch (err) {
  if (err instanceof GovernanceDeniedError) console.log(err.code); // "skill_not_granted"
}

High-risk skills go to a separate role, so an agent can never approve its own work — and every decision, allowed or denied, commits to a signed audit log. → packages/embedded

3 — Working with auditors?

Step 2 already writes a signed log. When auditors need a durable, queryable, tamper-evident record — plus a human-approval inbox and a review console — run the self-hosted server:

Every decision is Ed25519-signed and hash-chained: change any row and verification breaks. Export a bundle and anyone verifies it offline — no database, no trust in the process that produced it. → full server setup below

These are three independent packagesmc scan, @makerchecker/embedded, and the server — that enforce the same controls and write the same signed audit format. Adopt any one on its own.


Runnable examples of agents doing consequential work behind a human gate:

  • Pharmacovigilance case processing — an agent triages adverse-event reports, but a medical reviewer signs before an expedited 15-day regulatory report transmits. examples/pv-icsr-processing
  • Medical-device (MDR) complaint triage — a regulatory officer decides reportability behind a gate before draft reports are generated. examples/mdr-reportability-triage
  • Oncology patient access — an agent handles benefit matching but is blocked from submitting copay enrollments without a specialist signing. examples/oncology-patient-access
  • Daily cash reconciliation — a finance agent reconciles transactions but locks at exception gates until a cash officer signs off. examples/daily-cash-reconciliation

🔌 Integrate With Your Framework

Drop-in connectors govern the tools you already have:

When you run the server, the SDK's governedTool routes each call through a proxy session for centralized authorization and recording:

import { createClient, governedTool, GovernanceDeniedError } from "@makerchecker/sdk";

const client = createClient({ baseUrl: "http://localhost:3000", apiKey: "mk_..." });
const { session } = await client.proxy.openSession({ label: "recon-run" });

const match = governedTool(
  client, session.id,
  "recon-preparer",   // agent whose role grants are evaluated
  "txn-match@1",       // skillRef: name@version
  (input) => matchTxns(input),
);

await match({ statement, ledger });   // throws GovernanceDeniedError if denied
await client.proxy.closeSession(session.id);

🖥️ Self-Hosted Server (optional)

Run the full gateway when you need centralized enforcement across many agents, a human-approval inbox, and a review console. docker compose up brings up Postgres, the server on localhost:3000, and a seeded demo, printing two API keys — an admin key (your agent authenticates runs) and an officer key (a human reviewer approves gated actions).

The seeded pharmacovigilance flow parks at a medical-review gate where the requester is refused as its own approver:

export H='authorization: Bearer mk_...'         # admin key
export OFFICER='authorization: Bearer mk_...'   # officer key

curl -X POST localhost:3000/api/flows/pv-icsr-processing/runs -H "$H" -H 'content-type: application/json' -d '{}'
curl localhost:3000/api/approvals -H "$H"

# The requester cannot approve their own run — rejected with 403
curl -X POST localhost:3000/api/approvals/<id>/decision -H "$H" -H 'content-type: application/json' \
  -d '{"decision":"approved","reason":"self-approval attempt"}'

# A separate officer signs; only now does the action proceed
curl -X POST localhost:3000/api/approvals/<id>/decision -H "$OFFICER" -H 'content-type: application/json' \
  -d '{"decision":"approved","reason":"Seriousness confirmed; file 15-day expedited ICSRs."}'

curl localhost:3000/api/audit/verify -H "$H"

Full setup, Kubernetes/Helm, and running with live models: docs/quickstart.md.


🔒 Verifiable Audit Trail

Every decision and tool call commits to a hash-chained log — each event a SHA-256 over the RFC 8785 canonical JSON of the event, chained through prev_hash from genesis and Ed25519-signed. Change any row and verification breaks. Anyone can verify an exported bundle offline — no database, and no trust in the process that produced it:

npx @makerchecker/proof-verifier verify bundle.json

Spec: docs/audit-spec.md.



📄 License & Contributing

  • Server, Web, Shared: AGPL-3.0.
  • mc scan, embedded, SDKs, connectors, examples: Apache-2.0 — embed them in closed-source agents freely.
  • Commercial (non-copyleft) licensing: [email protected].

Contributing: CONTRIBUTING.md · Security: SECURITY.md · Code of Conduct: CODE_OF_CONDUCT.md

联系我们 contact @ memedata.com