无损模型压缩实验:GLM-5.2 内存占用降低 25%
Lossless model compression experiment: GLM-5.2 in 25% less memory

原始链接: https://brianbell-x.github.io/weight-compression/

本报告详细介绍了一种用于 BF16 模型权重的“K15”压缩方法。该方法通过利用符号位和指数位的重复性,实现了约 25% 的存储空间节省。 该技术将标准的 9 位符号-指数符号替换为映射至 15 项查找表的 4 位编码,同时保留原始的 7 位尾数。罕见符号通过精确的转义流进行处理。验证确认了 59,509 个张量的逐位重构准确性,实现了每个权重 11.173 位的布局密度。 在 A40 GPU 上使用 12 位密集原型进行的初步运行时间测试表明,其速度比标准 BF16 GEMV 操作提升了 0.733 倍。尽管压缩逻辑可靠,但研究指出,端到端服务集成、物理 GLM 规模的容器化以及完全融合的稀疏转义校正仍是待解决的问题。 此方法建立在 ZipNN、DFloat11 和 ZipServ 等现有技术的基础上,这些技术此前已证明了无损 BF16 指数压缩和直接寄存器重构的可行性。K15 实验的独特之处在于其特定的张量级 4 位编码结构以及对符号-指数冗余的处理方式。

``` Hacker News 新闻 | 往期 | 评论 | 提问 | 展示 | 招聘 | 投稿 登录 无损模型压缩实验:GLM-5.2 内存占用减少 25% (brianbell-x.github.io) hambandit 发布于 44 分钟前,15 点 | 隐藏 | 往期 | 收藏 | 1 条评论 帮助 Sanzig 16 分钟前 [–] 所以,我的理解没问题的话,这只是为了在网络传输中降低完整 BF16 权重的带宽占用,而不是用于推理服务,对吗?你们对比过它与 zstd 等先进通用压缩算法的性能表现吗? 另外,真的有很多人直接处理 BF16 权重吗?GLM-5.2 的参考部署是 FP8,许多厂商甚至直接使用 NVFP4 进行推理,且相比 FP8 参考部署,其性能损失似乎微乎其微。 回复 考虑申请 YC 2026 年秋季班!申请截止日期为 7 月 27 日。 准则 | 常见问题 | 列表 | API | 安全 | 法律 | 申请 YC | 联系 搜索: ```
相关文章

原文

[ FIG. 01 ]

01 / Where the K15 estimate comes from

Most weights share a few sign-exponent symbols

Every BF16 weight is 16 bits: a sign, 8 exponent bits, 7 mantissa bits. In a trained model the exponents are wildly repetitive; a handful of values cover nearly every weight.

The K15 accounting replaces the 9-bit sign-and-exponent symbol with a 4-bit code into a 15-entry table. Rare 9-bit symbols go to an exact escape stream; the 7-bit mantissa stays raw. With all side costs charged, the full scan prices this layout at 11.173 bits per weight.

02 / What was decoded exactly

The byte-split path round-trips

The streamed validator reconstructed the high byte from its codebook, indices, and escape stream, kept the low byte verbatim, and compared every BF16 tensor with the source.

All 59,509 tensors matched bit-for-bit. That decoded representation costs 12.005 bits per weight, or 24.967% less than BF16. The 30.168% K15 layout is separately charged accounting and was not independently decoded at GLM scale.

03 / Runtime evidence and open work

A dense prototype, not a serving result

A separate dense 12-bit prototype reconstructed codes in registers and measured 0.733 times BF16 GEMV time on an A40. Sparse escape correction was validated separately but was not fused into or included in that timing.

An exact lossless speedup, a physical GLM-scale K15 container, and end-to-end serving integration therefore remain open.

04 / Prior work and distinction

The exponent insight and fused concept are prior art

ZipNN and DFloat11 established lossless BF16 exponent compression in this size range. DFloat11 uses variable-length Huffman codes and restores BF16 weights before matrix multiplication. ZipServ is the closest prior runtime design: it already demonstrated fixed-length coding with direct register reconstruction. This experiment uses a different representation, per-tensor 4-bit codes over 15 joint sign-and-exponent symbols with sparse exact escapes, and does not claim the shared exponent redundancy, roughly 11-bit range, or fused concept as new.

联系我们 contact @ memedata.com