ALP：自适应无损浮点压缩

ALP：自适应无损浮点压缩
ALP: Adaptive lossless floating-point compression

发表于 ACM SIGMOD 2024 的 **ALP（自适应无损浮点压缩）** 是一种屡获殊荣的算法，旨在高效压缩 IEEE 754 浮点数据。它通过针对两种常见数据模式进行优化，从而提升存储效率： * **十进制浮点数：** ALP 通过乘以 10 的幂将其映射为整数，随后应用支持 SIMD 的 FastLanes 编码。 * **高精度数值：** ALP 对可预测的左侧位使用字典编码，同时保留随机的高精度右侧位不进行压缩，以确保数据完整性。 ALP 在压缩比、压缩速度和解码速度这三个关键领域持续优于现有的压缩方案。该项目因其可复现性而备受推崇，获得了全部三枚 ACM 可复现性徽章以及 **SIGMOD 最佳工件奖**。开源仓库（MIT 许可证）包含核心实现、基准测试工具以及清晰的文档，供用户将其集成到自己的系统中或针对自定义数据集进行测试。ALP 已成功集成到多种数据库系统中，被公认为浮点数据管理领域的尖端解决方案。

Sorry.

原文

Authors: Azim Afroozeh, Leonardo Kuffó, Peter Boncz
Conference: ACM SIGMOD 2024

This repository contains the source code and benchmarks for the paper ALP: Adaptive Lossless Floating-Point Compression, published at ACM SIGMOD 2024.

ALP is a state-of-the-art lossless compression algorithm designed for IEEE 754 floating-point data. It encodes data by exploiting two common patterns found in real-world floating-point values:

Decimal Floating-Point Numbers:
A large portion of floats/doubles in real-world datasets are decimals. ALP maps these values into integers by multiplying the number by a power of 10 and then compressing the result using a FastLanes variant of Frame-of-Reference encoding¹, which is SIMD-friendly.
Example: the number 10.12 becomes 1012 and is then fed to the FastLanes encoder.
High-Precision Floating-Point Numbers:
The remaining values are typically high-precision floats/doubles. ALP targets compression opportunities in only the left part of these values, which it compresses using FastLanes dictionary encoding. The right part is left uncompressed, as it is required to preserve high precision and is often highly random and incompressible.

📊 How does ALP perform?

These results highlight ALP’s superior performance across all three key metrics of a compression algorithm:
Decoding Speed, Compression Ratio, and Compression Speed—outperforming other schemes in every category.

🧪 How to Reproduce Results

Just run the following script:

./publication/script/master_script.sh

For more information on reproducing our benchmarks, refer to our guide here,
or read the official ACM reproducibility report:
https://dl.acm.org/doi/10.1145/3687998.3717057

🏅 ACM Artifacts & Awards

We are happy to share that we participated in the SIGMOD Availability & Reproducibility Initiative, and our paper earned all three badges:

🎉 We're also proud to share that ALP won the SIGMOD Best Artifact Award!

⏱️ Want to Benchmark Your Dataset?

Check out our guide: How to Benchmark Your Dataset
It explains how to run ALP on your own data.

🗂️ Repository Structure

src/: Core implementation of ALP and ALP_RD
benchmarks/: Benchmarking tools and datasets
include/: Header files for integration
scripts/: Utility scripts for data processing
test/: Unit tests
publication/: Publications and supplementary materials

This project is licensed under the MIT License. See the LICENSE file for details.

If you have questions, want to contribute, or just want to stay up to date with ALP and related projects, join our community on Discord:

ALP has been integrated into the following systems: