Jemalloc 5.4.0
Jemalloc 5.4.0

原始链接: https://github.com/jemalloc/jemalloc/releases/tag/5.4.0

此版本包含超过 160 次提交,主要致力于减少技术债务、代码重构及提升可移植性。 **主要特性:** * **固定内存 (Pinned Memory):** 增加了 `EXTENT_ALLOC_FLAG_PINNED`,允许为不可回收映射(如 HugeTLB)使用自定义钩子;并引入了广泛的 `mallctl` 接口,用于监控固定内存和互斥锁统计信息。 * **C++ 集成:** 将实验性的 `infallible_new` 运行时选项替换为编译时标志 `--enable-cxx-infallible-new`,以实现更好的编译器优化。 * **易用性:** 启用通过 `thread.arena` 恢复每个 CPU 的 arena 选择。 **不兼容变更:** * 将固定的 tcache 填充/刷新策略替换为基于垃圾回收 (GC) 事件间需求观测的动态模型。 * 移除了七个过时的非实验性 `mallctl` 控制项。 **优化与重构:** * 模块化了分配器前端和核心子系统。 * 引入了操作系统抽象层,将平台相关操作(I/O、线程、虚拟内存)与核心代码解耦。 * 改进了基块增长启发式算法,以防止虚拟内存耗尽。 **错误修复与可移植性:** * 解决了多种边界情况,包括潜在的死锁、`errno` 保留问题以及 C23 `NULL` 指针的正确性。 * 增强了 macOS、MinGW 和 GCC 16 的跨平台支持,并改进了线程局部存储 (TSD) 的生命周期管理。

**Jemalloc 5.4.0** 的发布在 Hacker News 上引发了热烈讨论,标志着这一广泛使用的内存分配器重新回到了积极开发的状态。 讨论的主要要点包括: * **项目状态:** 在从 Meta/Facebook 转回社区后,经历了一段沉寂期,Jemalloc 目前已恢复官方版本发布。 * **重要意义:** 用户称赞该分配器能够大幅降低资源密集型应用(如 Ruby on Rails 和各类 JVM 服务)的内存占用。许多开发者强调,即使是在使用具备垃圾回收机制的语言时,切换至 Jemalloc 也能解决持续存在的原生内存碎片和泄漏问题。 * **社区讨论:** 除了发布本身,讨论还涉及了由万亿级公司掌控开源项目的风险、malloc 命名惯例的历史(例如“Jason Evans malloc”),以及关于线程本地缓存与 CPU 本地缓存策略优劣的技术辩论。 该话题反映了人们对高性能通用系统工具的高度认可,同时也体现了“黑客”们在当前 AI 内容泛滥的背景下,更倾向于关注切实的工程改进。
相关文章

原文

This release contains over 160 commits, focusing on the technical debts
cleaning including refactorings, bug fixes, test coverage improvement, and
option cleanups. The release also includes portability improvements per
upstream issues report.

New features:

  • Add EXTENT_ALLOC_FLAG_PINNED so custom extent-allocation hooks can
    mark non-reclaimable mappings, such as HugeTLB pages, for preferential
    reuse outside the decay and purge pipeline. Add the mallctl interfaces
    stats.pinned, stats.arenas.<i>.pinned,
    stats.arenas.<i>.extents.<j>.npinned,
    stats.arenas.<i>.extents.<j>.pinned_bytes, and
    stats.arenas.<i>.mutexes.extents_pinned.{counter} to report
    pinned-memory usage and mutex statistics. (@binliu19: be2de8c)
  • Allow resuming per-CPU arena selection via thread.arena.
    (@Algunenano: 68c35f6)
  • Better align the contents of human-readable and JSON malloc statistics.
    (@spredolac: 68fe1be, 30b1a41, 04aad97)
  • Replace the runtime experimental_infallible_new option with the
    compile-time --enable-cxx-infallible-new option, enabling
    compiler-level optimizations and optimization in move constructors,
    and fix the new(std::nothrow) contract (@spredolac: 160ab9d, fe33667)

Incompatible changes:

  • Adapt tcache fill and retention targets per bin to demand observed
    between GC events, replacing the fixed refill/flush policy. Remove
    seven legacy non-experimental controls: lg_tcache_nslots_mul,
    tcache_nslots_small_min, tcache_nslots_small_max,
    tcache_nslots_large, tcache_gc_delay_bytes,
    lg_tcache_flush_small_div, and lg_tcache_flush_large_div.
    Corresponding malloc_conf settings are silently ignored, matching
    opt.* mallctls return ENOENT, and tcache_ncached_max remains
    supported. (@spredolac: d13fe91)

Bug fixes:

  • Preserve errno across free, free_sized, and free_aligned_sized,
    and across process_madvise-based page purging. (@spredolac: 3a77966,
    86f0582)
  • Fix numeric overflow checks in size classes. (@spredolac: 6b24522)
  • Accept NULL in free_sized() and free_aligned_sized() (C23
    correctness). (@bigbruno: 7ce8b91)
  • Fix TSD lifecycle edge cases by 1) initializing thread-cache bins before
    marking the cache enabled, preventing reentrant bootstrap allocations
    from using uninitialized state, and 2) avoiding TSD recreation for late
    deallocations after thread teardown on generic-TSD platforms.
    (@fzakaria: 54f22c8, @spredolac: fb5499a)
  • Use O_CLOEXEC when opening the THP sysfs file in init_thp_state.
    (@ibookstein: d070554)
  • Fix duplicate opt.stats_print and opt.stats_print_opts fields in
    malloc_stats_print output. (@spredolac: dfe3a2e)
  • Fix a potential deadlock during arena_reset. (@guangli-dai: 6957341)
  • Fix a prof-sampling / guard-page interaction bug in the SAN. (@gctony:
    e36a0fa)

Optimizations and refactors:

  • Modularize jemalloc's front end by extracting arena management,
    initialization, fork orchestration, and allocation dispatch from
    jemalloc.c; untangle tcache/arena ownership; and consolidate the
    internal header graph to eliminate circular dependencies. (@spredolac:
    ba1e2fe, 8874597, 9d75722, de9ad14, ...)
  • Simplify ctl dispatch by refactoring arena helpers, internalizing an
    implementation-only control, replacing control-flow macros with typed
    helpers, and organizing ctl.c by subsystem. (@spredolac: 9e14346,
    901365d, 4e903a0, 5bc8d6e)
  • Cap the base-block growth heuristic to avoid virtual-memory exhaustion
    under rare racy conditions. (@guangli-dai: 2f4db8c)
  • Move background-thread lifecycle and state operations into the
    background-thread module, clarifying ownership independently of PAC/HPA
    callers. (@guangli-dai: f1f0792)
  • Simplify the page-allocation boundary by replacing PAI vtable dispatch
    with direct PAC/HPA calls, removing the obsolete pai_t/pai.h
    abstraction, and moving deferred-work and decay orchestration out of the
    arena. (@guangli-dai: 1dfa6f7, 8edd101, d410f43)
  • Refactor statistics collection and rendering into separate
    gather/emission stages and descriptor-driven tables. (@spredolac:
    184f304, 80c8fcb)
  • Introduce an OS abstraction layer and move platform-dependent
    file/process I/O, time, synchronization, CPU, virtual-memory, atfork,
    error-handling, profiling, thread-yield, and configuration-access
    operations out of allocator core code. (@guangli-dai: c4158ac,
    c8e2e01, ...)

Portability improvements:

  • Replace the std::__throw_bad_alloc call with standard C++ (#2900).
    (@lexprfuncall: 1a15fe3)
  • Make arena_s use a flexible array member (bin_t all_bins[]) for C99
    or newer. (@grueninger: 300b58b)
  • Fix rdtscp detection with --with-lg-vaddr. (@xinydev: e8a0d2b)
  • Fix malloc_getcpu on macOS to read the current CPU number correctly.
    (@Algunenano: 5aabbc8)
  • Use CLOCK_MONOTONIC for background-thread sleep to prevent
    clock-rollback stalls, and detect monotonic-condvar support at configure
    time. (@antonio2368: ebacec3, 8361239)
  • Fix compilation warnings on macOS. (@gctony: abb0a8a)
  • Fix thread-exit TSD cleanup on MinGW builds. (@gctony: 1e92317)
  • Fix GCC 16 build warnings by removing -Wpedantic violations in macro
    and function syntax and explicitly NUL-terminating profiling thread-name
    copies to resolve -Wstringop-truncation. (@grueninger: 5acdcee,
    c22d929, @jasonangelov: 6a245e0)
  • Parse PID-namespace symlinks without glibc-dependent strtok/atol,
    and return identifiers as uint64_t. (@guangli-dai: 278d90a)
联系我们 contact @ memedata.com