Triforce——一款适用于苹果硅芯片笔记本电脑的波束成形器
Triforce – a beamformer for Apple Silicon laptops

原始链接: https://github.com/chadmed/triforce

Triforce是一个开源的最小方差无失真响应(MVDR)自适应波束形成器,专为特定苹果硅Macbook(Air/Pro型号)中的麦克风阵列而设计。其目标是使内置麦克风阵列能够在macOS之外使用,因为macOS依赖于专有的波束形成器。 由于可访问的数字信号处理(DSP)文献有限,Triforce是基于基本的工程数学原理构建的。它试图将所需信号与噪声分离,但承认它可能最初无法超越苹果的实现。 由于`nalgebra`的矩阵运算性能(缺乏显式SIMD指令)的限制以及为了降低计算成本,Triforce专注于单频带方法,避免了宽带分解。这一选择,以及创建立体声输出的计算成本,限制了输出为单声道。欢迎贡献以提高性能和波束形成质量。

Hacker News 上正在讨论 Triforce,一个针对苹果硅笔记本电脑的波束成形器。用户们对其应用感到好奇,一些人想知道它是否能改善桌面语音应用的性能,以及为什么苹果公司没有针对 Mac 上已有的麦克风阵列硬件公开更高级别的 API。 有评论澄清说,苹果的波束成形是一个软件功能,因此 Asahi Linux 用户需要这个实现。用户们指出,在 macOS 上,波束成形有效地消除了背景噪音。一些人推测苹果的做法,认为这可能是为了避免专利纠纷,类似于他们的扬声器实现方式。有评论解释说,苹果的麦克风设计用于波束成形,它们非常灵敏,但却是全向的,如果没有波束成形,其作用方式与传统的定向笔记本电脑麦克风不同。其他笔记本电脑也在 DSP 中使用类似的技术,但也可以进行修改。

原文

Triforce implements a Minimum Variance Distortionless Response adaptive beamformer for the microphone array found in the following Apple Silicon laptops:

  • MacBook Pro 13" (M1/M2)
  • MacBook Air 13" (M1/M2)
  • MacBook Pro 14" (M1 Pro/Max, M2 Pro/Max)
  • MacBook Pro 16" (M1 Pro/Max, M2 Pro/Max)
  • MacBook Air 15" (M2)

Triforce tries to keep dependencies to a minimum. On top of the crates specified in Cargo.lock, you will only require:

The Apple Silicon laptops mentioned above have an array of microphones arranged either in a triangle or in a straight line. Much like with the speakers, Apple are trying way too hard to be fancy here, and implement an adaptive beamformer in userspace to try and isolate the desired signal from background noise. Without beamforming, the array is far too sensitive and omnidirectional to be at all useful. Thus, to make it useful outside of macOS, we need a beamformer.

Finding accessible literature on any DSP-related topics, let alone something like wideband adaptive beamforming, is challenging to say the least. This is an attempt at a beamformer armed only with first year undergrad level engineering maths and some vague idea of the principles gleaned from various webpages and PDFs. Do not expect it to outperform Apple's implementation at this time. Patches to improve it are always welcome!

  • nalgebra does not do any explicit SIMD optimisation, relying only on LLVM auto-vectorisation. Performance and efficiency of matrix math routines are not very good.
  • Following from that, we are not doing wideband decomposition due to the added computational burden. Without SIMD/NEON support, this is simply too slow for a realtime audio plugin.
  • Output is mono only. Much like with wideband decomposition, adding additional matrix processing to fake stereo output would be too computationally intensive
联系我们 contact @ memedata.com