Wattage:一种用于 AI 智能体的 Token 消耗分析工具及成本回归控制门。
Wattage: A token-spend profiler and cost-regression gate for AI agents

原始链接: https://github.com/faizannraza/wattage

**Wattage** 是一款离线、隐私优先的诊断工具,可作为 AI 智能体的“能耗计”(Kill-A-Watt meter)。它通过分析 OpenTelemetry (OTel) 追踪数据,识别代币浪费,以真实货币计算成本,并针对冗余工具调用、提示词缓存不足以及智能体循环无法收敛等低效问题提供可操作的修复建议。 **核心功能:** * **收敛引擎:** 使用先进的检测技术捕捉智能体“抖动”和停滞的循环,这是标准精确匹配工具无法做到的。 * **可操作的报告:** 生成终端报告、HTML 火焰图和 README 徽章。它能识别模型不匹配和冗余输出等问题,并按成本和质量风险对修复优先级进行排序。 * **CI 集成:** 作为成本回归关卡。如果智能体的变更导致价格大幅上涨或评分下降,它可以中断 CI 构建,从而确保部署期间性能的稳定性。 * **零配置:** 完全离线运行,无需 API 密钥,通过摄取 JSON 追踪数据提供透明且基于证据的数据,而非估算值。 Wattage 非常适合希望通过自动化、可验证的反馈循环来优化生产环境 AI 智能体成本与性能的开发者。

抱歉。
相关文章

原文
wattage

CI PyPI npm Python versions License: Apache 2.0 Docs

A Kill-A-Watt meter for your AI agents. Point it at a trace and it tells you exactly where your tokens are being burned and wasted, prices each waste pattern in real dollars, prescribes a fix, and can fail your CI when a change makes your agent measurably more expensive.

wattage report demo

A real captured agent trace (see provenance) — Wattage catches a stable prompt prefix being re-sent instead of cached, prices the waste, and prescribes the fix. Regenerate this GIF with vhs docs/assets/demo.tape (see the tape file for the exact command).

uvx wattage report trace.json

No config file, no API key, fully offline — point it at an OTLP JSON trace export and it prices every call and runs every detector. Don't have a trace yet? Getting your first trace covers both "I already have OTel traces" and "I have zero instrumentation" (a runnable, 5-minute path from nothing to a real, priced report). Or try it right now against the fixture shipped in this repo:

git clone https://github.com/faizannraza/wattage
cd wattage && uv sync
uv run wattage report examples/sample_trace.json
╭──── ⚡ wattage — examples/sample_trace.json ────╮
│ Token Efficiency: A (100)   Total cost: $0.0602 │
│ quality: unmeasured                             │
╰─────────────────────────────────────────────────╯
      Token breakdown
┏━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ Category       ┃ Tokens ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ input          │  18450 │
│ output         │    320 │
│ cache_read     │      0 │
│ cache_creation │      0 │
│ reasoning      │      0 │
└────────────────┴────────┘
No findings — this trace looks efficient.
pricing: 2026-07-18-verified

Or get a self-contained, shareable HTML flame graph instead of the terminal view:

uv run wattage report examples/sample_trace.json --html report.html

The evidence, not a marketing claim

Wattage's standout feature is the convergence engine — the nonconvergence detector, which catches an agent thrashing through a loop without making real progress, including patterns a naive exact-match duplicate detector structurally cannot see (a retry with a fresh timestamp each time, an oscillation between two strategies, a "productive-looking" stall where every call is technically unique but nothing is actually learned).

Rather than assert that, we built a hand-reviewed set of 10 labeled synthetic loops and benchmarked Wattage's classifier against a real SHA-256 exact-match baseline implementation:

Classifier Precision Recall F1
Wattage 1.00 1.00 1.00
SHA-256 exact-match 1.00 0.14 0.25

Reproduce it yourself — no cherry-picking, no hidden setup:

uv run python -m benchmarks.harness

And on a genuine captured agent trace (not synthetic — see benchmarks/traces/README.md for provenance), Wattage's prefix_churn fix simulation shows a 44.7% cost reduction ($0.000199 → $0.000110) from enabling prompt caching on the stable prefix — small dollar figures because it's a 3-turn demo trace, but the mechanism is identical at production scale. Run it against your own traces for numbers that matter:

uv run python -c "from benchmarks.frontier import build_frontier; print(build_frontier())"

Full methodology: The Convergence Engine.

uv run wattage badge trace.json --out wattage-badge.svg
![Wattage](wattage-badge.svg)

Wire --badge-out into your CI job (see below) so it regenerates on every merge to your default branch, and the badge in your README stays live.

Three surfaces, one normalized data model underneath (sessions → tasks → loops → iterations → calls), built from OpenTelemetry GenAI semantic-convention traces:

  • wattage report — ingests a trace, prices every call against a vendored, dated pricing snapshot, and runs eight detectors:

    Detector Catches
    prefix_churn Stable context re-sent instead of cached
    cache_gap Caching attempted but under-redeemed by later reads
    verbosity Output far beyond what the step needed
    redundant_tool_calls The same tool call repeated (exact or fuzzy)
    nonconvergence Loops that thrash, oscillate, or stall without progress
    retrieval_thrash Repeated retrieval that never yields relevant results
    model_mismatch A pricier model doing work a cheaper one could handle
    reasoning_overspend Heavy reasoning-token spend on a simple step

    Every finding is priced in real dollars, includes a concrete fix, and is tagged with a quality_risk tier (none / low / review) — a fix that could plausibly change output quality (a model downgrade, less reasoning) only counts toward your score once a --quality map backs it with real evidence. Full detail: Detectors.

  • wattage score / wattage badge — a single 0–100 Token Efficiency grade for a README badge or a CI gate.

  • wattage ci — the cost-regression gate (below).

Wattage never fabricates a number: an unpriced model leaves that call's cost at zero (and fails wattage ci loudly, exit code 4) rather than guessing; an unmeasured quality signal is reported as unmeasured, not assumed fine.

# .github/workflows/wattage.yml
name: Wattage
on:
  pull_request:
    paths: ["agents/**", "prompts/**", "src/**"]
concurrency:
  group: wattage-${{ github.ref }}
  cancel-in-progress: true
jobs:
  token-efficiency:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Generate trace fixture
        run: python scripts/run_agent_fixture.py > trace.json
      - name: Wattage cost-regression gate
        uses: faizannraza/wattage/[email protected]
        with:
          source: trace.json
          baseline: .wattage/baseline.json
          fail-on: "score_below:80,cost_delta_pct_above:5,any_critical:true"
          pr-comment: "true"

Fails the build (exit code 1) when your agent regresses past the threshold you set, posts a per-detector delta table as a PR comment, and emits SARIF (shows up in GitHub's Security tab) and JUnit XML for any other CI system. The baseline is a small committed JSON file — noise-floor protection is structural, not statistical: it only ever updates on a run that actually passed the gate.

This is only half the setup. A PR job runs on a throwaway checkout, so it can't be the thing that updates .wattage/baseline.json on disk — that update needs a second workflow, triggered on push to your default branch, that commits the refreshed baseline (and badge) back after each merge. Skipping it means every PR compares against the same stale baseline forever. Full reference, with both workflows: CI Integration.

Detectors are discovered through a Python entry-point group, so adding one doesn't require touching this repo's core pipeline — see CONTRIBUTING.md for the full "write a detector" walkthrough, using cache_gap as the reference example.

Apache-2.0

联系我们 contact @ memedata.com