Persistent State Machines: LLM Attention with INT4 In-Memory Cells

原始链接: https://zenodo.org/records/21753002

近期的一场 Hacker News 讨论将论文《持久状态机：基于 INT4 内存单元的 LLM 注意力机制》标记为“AI 垃圾内容”和“伪术语大杂烩”。批评者认为该论文缺乏科学可信度，原因如下： 1. **缺乏同行评审：** 该论文是在 Zenodo 上自行发布的，而非通过认可的渠道。 2. **发布节奏可疑：** 作者在不到一周的时间内发布了八个版本，绕过了标准的迭代反馈流程。 3. **技术主张逻辑混乱：** 论文试图模糊地将有限自动机理论、深度学习和 FPGA 硬件等不同领域结合起来，却未能提供必要的数学逻辑支撑。 4. **指标不切实际：** 论文仅基于未经证实的模拟，就得出了夸张的性能结论（例如 2129 倍的加速），而缺乏实际物理测量或硬件综合支持。怀疑论者普遍认为，该论文看起来像是缺乏实质研究基础的生成式内容。他们建议社区对那些堆砌术语、令读者困惑，却无法提供具体证据或同行评审严谨性的论文保持警惕。

原文

Persistent State Machines: Complete Mathematical Proofs and Vivado Implementation Synthesis (Version 8.0)

ABSTRACT We present a formal discrete framework for attention operators in Large Language Models via Persistent State Machines (PSMs). Computation is broadcast to stationary in-memory cells that evaluate local deterministic state transitions. Complete mathematical proofs are given for quantization error bounds, a concrete multi-phase discrete Softmax construction under an explicit bounded-logits assumption, deterministic finite-automaton equivalence with spatial factorization, and membership in DSPACE(O(n)).

We validate the implementation feasibility of the architecture on contemporary programmable logic fabric through two distinct evaluation flows:

Low-Power Evaluation (Zynq-7000 xc7z020): A full 1024-cell array (d=128) out-of-context (OOC) block design implementation. By annotating the post-route netlist with a Switching Activity Interchange Format (SAIF) file obtained from a post-route functional simulation, we demonstrate that hardware-level self-activation gating restricts active dynamic switching to sparse cells. The dynamic power of the core logic is estimated below 1.0 mW, yielding a normalized dynamic energy of 3.81 × 10^-5 pJ/op.
System-on-Chip PCIe Integration (UltraScale+ xcvu9p): To verify system-level compatibility, a 256-cell sub-array (representing an integrated 1-head attention system) is integrated within a full SoC. This includes AMBA AXI4 interconnects and an AMD Xilinx PCIe Gen3 x1 Bridge (XDMA v4.2). The complete SoC successfully closed timing at a unified system clock of 62.5 MHz (Worst Negative Slack WNS = +1.854 ns) under the AWS Cloud FPGA Developer environment. The SoC occupies only 0.67% of the device's logic slices and 0.00% of DSP blocks, showing high scaling potential.

Functional simulation with more than one thousand random vectors confirmed bit-exact agreement with a fixed-point software reference. All energy figures are simulation-based tool estimates for the synthesised logic; no physical FPGA board measurement was performed, and system-level external memory energy is strictly excluded.

Japanese Patent Application No. 2026-177318 (Patent Pending)