展示 HN:6cy – 实验性流媒体存档格式,具有每块的编解码器。
Show HN: 6cy – Experimental streaming archive format with per-block codecs

原始链接: https://github.com/byte271/6cy

## Sixcy:下一代容器格式 Sixcy 是一种正在开发的容器格式,专注于高性能数据存储和传输,优先考虑流式效率、数据可恢复性和灵活压缩。目前旨在用于基准测试、研究和原型设计——**不用于生产环境**——它提供了一种流式优先的设计,具有自描述块和定期检查点,以实现强大的数据恢复。 主要特性包括支持多种压缩算法(Zstd、LZ4 以及通过插件接口支持更多算法)、元数据优先索引以实现快速访问,以及内存安全的 Rust 实现。该项目结构清晰,模块化地划分了核心库函数、编解码器、索引、恢复和 I/O。 目前版本为 v0.1.x,Sixcy 提供了一个参考实现和插件接口。计划在 v0.2.0 中提供稳定的运行时包和扩展的编解码器支持。规范采用 CC-BY-4.0 许可,而实现则使用 Apache-2.0 许可。

Hacker News 新闻 | 过去 | 评论 | 提问 | 展示 | 招聘 | 提交 登录 展示 HN: 6cy – 实验性流式存档格式,具有每块的编解码器 (github.com/byte271) 34 分钟前,yihac1 发布,4 点赞 | 隐藏 | 过去 | 收藏 | 讨论 大家好, 我一直在研究存档格式设计,并构建了 6cy 作为研究项目。 目标不是取代 zip/7z,而是探索: • 块级别编解码器多态性(每个块不同的压缩) • 流式优先布局(无需全局查找) • 更好的崩溃恢复特性 • 基于插件的架构,以便专有编解码器可以在不更改格式的情况下存在 目前这是一个实验性的 v0.x 格式。 规范可能会发生变化,兼容性尚未保证。 我主要希望获得关于格式设计的反馈,而不是性能比较。 感谢您的关注。 帮助 指南 | 常见问题 | 列表 | API | 安全 | 法律 | 申请 YC | 联系方式 搜索:
相关文章

原文

Important

6cy is under active development and is not yet stable. The file format and APIs may change without notice.

Note

This project is intended for benchmarking, research, and ecosystem prototyping. It is not recommended for production use yet.

Sixcy is a modern, efficient, and robust container format designed for high-performance data storage and transmission. It is built with a focus on streaming efficiency, data recoverability, and flexible compression strategies.

  • Streaming-First Design: Optimized for single-pass read and write operations, making it ideal for network streams and large-scale data processing.
  • Data Recoverability: Includes self-describing blocks and periodic checkpoints to ensure data can be recovered even if the archive is truncated or partially corrupted.
  • Codec Polymorphism: Supports multiple compression algorithms (Zstd, LZ4, etc.) within a single archive, allowing for block-level optimization based on data type.
  • Plugin Architecture: Supports third-party and proprietary compression algorithms through a well-defined plugin interface, enabling closed-source extensions to integrate seamlessly.
  • Metadata-First Indexing: Provides a centralized index for fast random access and efficient file listing without scanning the entire archive.
  • Rust Reference Implementation: A high-performance, memory-safe implementation that serves as the canonical reference for the Sixcy specification.
  • src/: Core library and CLI implementation.
    • lib.rs: Library entry point.
    • superblock.rs: Fixed header management.
    • block.rs: Data block encoding and decoding.
    • codec/: Compression algorithm implementations and plugin interface.
    • index/: File index and metadata management.
    • recovery/: Data recovery and checkpointing logic.
    • io_stream/: High-level streaming I/O interfaces.
  • tests/: Integration tests for format validation.
  • benches/: Performance benchmarks for compression and I/O.
  • spec.md: Detailed binary format specification.
  • security.md: Public security profile and threat model.

Go to Environmental_preparation.md to view Environmental preparation

The compiled CLI tool will be available at target/release/6cy.

Note

What This Build Provides The open-source build includes:

  • .6cy container format implementation
  • Streaming I/O engine
  • Recovery logic
  • Standard codecs (Zstd / LZ4)
  • Plugin ABI for external codecs
  • Proprietary codecs and optimization
  • components are not part of
  • the open-source tree. They can be
  • integrated via the plugin interface.
cd C:\sixcy\target\release

Windows run

Note

Before running any commands, make sure you are in this directory

Linux&MacOS run

chmod +x 6cy
./6cy --help

You will see pack,unpack,list,info,optimize

  • Specification: The full binary format specification is available in spec.md.
  • Security: Security considerations and reporting procedures are detailed in security.md.

v0.1.x (current) Provides the reference implementation of the .6cy container format, streaming engine, and plugin interface.

This version is intended for:

  • Format validation
  • Integration testing
  • Research and benchmarking

v0.2.0 (planned) Will introduce the official runtime package and extended codec support.

If you are looking for a ready-to-use build, please wait for the v0.2.0 release.

This project uses different licenses for the specification and the implementation:

  • Specification (spec.md): Licensed under CC-BY-4.0.
  • Implementation (src/, tests/, benches/, etc.): Licensed under Apache-2.0.

Sixcy - Speed, Safety, and Flexibility in Data Containment.

联系我们 contact @ memedata.com