沙漠蚂蚁实验室:在设备端运行的本地化、高速模型
Desert Ant Labs: local, fast models that run on device

原始链接: https://desertant.com/blog/introducing-desert-ant-labs/

Desert Ant Labs 正式上线,提供一套专为端侧运行设计的 AI 模型,旨在速度、成本和隐私保护方面超越云端方案。Desert Ant 不走通用大模型路线,而是专注于小型、任务导向型模型,使开发者能够让手机和笔记本电脑本地运行智能功能,从而消除 Token 成本并降低延迟。 该平台首发 18 款模型(涵盖音频增强、转录和个人敏感信息去标识化等功能),并提供适用于 Swift、Kotlin 和 JavaScript 的统一 SDK。这些模型经过效率优化,即使在五年前的硬件上也能流畅运行。Desert Ant 倡导“主权”AI,确保用户数据留存在设备端,为云端 API 提供了一种安全的替代方案。 通过利用消费级硬件中未被充分挖掘的庞大算力,该公司旨在摆脱行业对昂贵数据中心的依赖。Desert Ant 正在改变 AI 开发范式:开发者无需再为每次 API 调用付费,而是可以将“小巧的大脑”直接嵌入产品中,在每个交互点实现实时、零成本的智能。对于每月活跃设备数在 10 万台以内的应用,这些模型均可免费使用。

关于“Desert Ant Labs”的 Hacker News 讨论反映了人们对其最初的好感与随后的怀疑。虽然用户被其在设备端快速运行人工智能模型(用于 PDF 处理和内容标记等任务)的承诺所吸引,但社区也指出了几个实际的局限性。 批评主要集中在三个方面: 1. **平台限制:** 开发人员指出,这些模型主要针对 iOS/macOS 进行了优化,这使得它们在标准 Web 环境或低成本 VPS 托管中并不实用。 2. **技术透明度:** 敏锐的用户指出,该公司提供的产品(如“Voz”、“Clear”和“Ear”)似乎只是现有开源模型(Parakeet、DeepFilterNet 和 Whisper-tiny)的重命名版本,并捆绑了私有的推理代码。 3. **实用性:** 虽然市场对用于内容管理系统(CMS)集成和数据提取的轻量级设备端 AI 有明显需求,但参与者认为目前的实现方式缺乏广泛 Web 开发用例所需的跨平台兼容性。 总体而言,社区认为该项目只是在成熟的开源技术之上覆盖了一层精美的营销包装,且受限于其目前针对特定硬件的定位。
相关文章

原文

Today we're launching Desert Ant Labs, a European frontier AI lab building opinionated on-device intelligence. We believe the best path to efficient intelligence starts on-device.

We're building small, specialized models for audio, vision, and text – each model answers in milliseconds, and costs nothing to run, so you can put intelligence in every product interaction, without being limited by token cost or inference speed. Small enough to run on a five-year-old phone, fast enough to use on every frame or keystroke, and better than the API call you're already paying for.

The first 18 models are live today (12 stable and six in beta), accessible via one SDK for Swift, Kotlin, and JavaScript. One model per task, each built to be the fastest way to complete that task on a device:

  • Voz: transcribe 10 minutes of audio in two seconds on an iPhone – 4.7x faster than Whisper – with a start and end time on every word.
  • Clear: a 9MB model that can turn a five-minute laptop recording into studio quality audio in one second.
  • Redact: mask names, addresses, and card numbers, in real time, in 27 languages, so they never reach your servers.
  • Tongue: identify 84 languages from three words, with a 2MB model.
Language ID accuracy, three words in

Tongue · 2MB 0.933

293MB detector 0.887

Tongue names the language from three words, scoring 0.933 at 2MB against 0.887 for a 293MB detector.

And that's just to name a few. You can find full specs and benchmarks for the other fourteen, on desertant.com/models and Hugging Face. Every model is free up to 100k monthly active devices. No tokens, no logins.

Personal data caught, by system

Redact · 12MB 88.8

GLiNER-PII · 2.3GB 91.1

Rampart · 14.7MB 61.4

OpenAI filter · 3GB 60.2

Redact catches 88.8% of the personal data in a text, close to the 2.3GB GLiNER-PII, from a 12MB model.

We're building this in Europe, where "on-device" is the sovereign default. The data never leaves your customer's hands, the feature never depends on someone else's cloud, and what's never been uploaded can never be compelled.

How we got here

For five years we've been building our video app, Detail, with an on-device first approach. But when we introduced features like Auto Edit to create short clips, or audio enhancement for podcasts, we had to fall back to cloud APIs. And as the popularity of Detail grew, so did our infrastructure bills.

Every few months I'd hunt for useful on-device models. I'd surf Hugging Face for a model that could find filler words or clean up a recording. And, every June, we'd get great new tools to build with but the industry wasn't moving fast enough. The foundation was there: the chips, Core ML, the research. What was missing was everything between that foundation and actually implementing a feature in your app: a model you could drop in and ship with a few lines of code.

So, we trained the models ourselves. It turns out training a model is a product design challenge, and product is what we know. We designed models and local inference that beat cloud services on speed, quality, and cost, and outperform other local and cloud models on the task itself, at a fraction of their size.

We replaced Dolby for better, faster audio enhancement with Clear, and made our on-device transcriptions 5x faster with Voz. We also replaced Claude Sonnet with Clips, our 284MB model that turns a 10-minute video into a dozen clips in 5 seconds – 10x faster and using 470x less energy than Sonnet, with the same quality.

Clear audio enhancement speed, 5 minutes of audio

iPhone 16 Pro 302x

MacBook Pro (M5) 345x

Clear enhances, masters, and re-encodes a clip on the device, best of three, from a 9MB model. 302x realtime on a phone.
Transcription speed, 30 minutes of audio

Voz 319x

Apple SpeechAnalyzer 78x

Whisper large-v3-turbo 50x

Realtime factor over 30 continuous minutes on an M3 Ultra. Voz reaches 298x on an iPhone 17 Pro.

Detail 6, which will launch with iOS 27, replaces all of our cloud APIs with our own models, running entirely on the device.

We've all spent the past few years building with LLMs as if they were just another API. And, amid the hype around generalist frontier brains, we almost forgot they're not the only option.

Every developer I talk to has a wishlist of on-device models they'd build if cost wasn't a factor, or a feature they're bleeding tokens on that they'd happily swap for a local model. A call that runs the same way a hundred thousand times a day: cleaning a recording, tagging a photo, pulling a date out of a sentence, catching a name before the text hits your servers. None of these needs a frontier model.

NVIDIA's own researchers pulled apart three agent systems and estimated that 40 to 70% of their calls to a large model could go to a small, specialized one instead.

The compute is already paid for

The industry will spend about $450 billion on data centers this year. Meanwhile, the world ships more than a billion phones, tablets, and laptops with increasingly capable chips, perfectly suited to these kinds of tasks. There's more compute available in people's hands than in every AI data center on earth.

We have an unfair advantage with free inference. No per-call cost, so a feature runs on every message instead of the ones you can afford to check. No round-trip, and your customer's data never leaves the device. When inference costs nothing, the way we build products changes entirely.

Little brains in every product

To build with local models, the developer experience has to get a lot better. You need models you can use commercially, that beat the alternatives on your task in speed and quality, that you can drop into your app with a few lines of code, and are easy to discover.

Think of the first hundred models as the cerebellum, the little brain. The little brain handles the always-on work – balance, timing, the skills you never think about, so the rest of the brain is free to think. That's what we're building first: fast, specialized models for the work that runs all day, on the device, for free.

Then comes the cortex, the layer that decides which model answers. A small local model first, a bigger one when the job requires it, and the cloud only when the work has to leave the device. As open research advances and device silicon becomes more capable, the local models grow, and we'll train larger ones ourselves. Frontier intelligence, built from the small end up.

Cloud labs ship neutral models because per-token pricing needs a neutral model. Every Desert Ant model ships with a default we choose, and the levers you need to change that default. We optimize the model and the runtime together: on an iPhone, Clear and Voz run on the Neural Engine, and in the browser, Clear's same weights run through WebAssembly.

The SDK

Ready to get started? You can implement Desert Ant models in your app with our native Swift, Kotlin, and JavaScript SDK, available on GitHub.

Our docs are written for developers and agents and you can try the models on your Mac with the CLI, or in your browser on Hugging Face.

Building something cool with our models, or want to build them with us? Get in touch.

联系我们 contact @ memedata.com