Local-First Software Is Easier to Scale

原始链接: https://elijahpotter.dev/articles/local-first_software_is_easier_to_scale

Harper, an "edge-first" grammar checker, experienced a massive surge in traffic and users after hitting the Hacker News front page. This influx highlighted a key advantage of local-first software: scalability without server infrastructure. Unlike competitors like LanguageTool, which relies on server-side processing and would need to scale up resources to handle a sudden increase in demand, Harper runs directly on the user's device. This eliminates the need for expensive cloud architecture expertise and increased AWS bills. The author emphasizes the joy of not having to scale, illustrating the efficiency of Harper's optimized and lean code. While cloud providers boast about scalable solutions, Harper's architecture avoids scaling altogether, providing a seamless experience even with a significant user spike. The author only noticed the spike when visiting Hacker News the following morning, because there were no issues at all with the Harper system.

This Hacker News thread discusses the merits of "local-first" software, where data resides primarily on the user's device instead of a remote server. Commenters highlight several advantages: reduced latency, increased privacy, built-in scalability (as each user handles their own setup), and resilience against service outages or data loss due to provider issues. One user shared an anecdote about a forgotten iPad constantly resyncing old data. However, the thread also acknowledges challenges. One is the business model, as subscriptions are more easily justified with cloud services. Another is ensuring data integrity and security, as users are responsible for their own backups and protection. Local-first AI applications face hurdles related to the high VRAM requirements of many models. Some consider "local-first" a rebrand of the "box-product" concept. Others debate the practicality of scaling collaborative features in a fully local-first environment, or the security implications of trusting client-side logic. Ultimately, the appeal of local-first software lies in its user control and independence from centralized services, even if financial or technical complexities remain.

The title of this post is somewhat misleading. Local-first software rarely needs to be scaled at all.

Harper recently received a massive increase in both traffic and user count. How? By making it to the front page of Hacker News. If you couldn't tell by the extent to which I talk about this, I'm thrilled. This amounted to a tremendous amount of free advertising, if nothing else.

I've been thinking a lot about the wondrous benefits of working at the edge. That's just one term of many that people use to describe Harper: "edge-first". Others include "local-first" or simply "on-device". They are all just jargon for a simple idea: We run all the relevant grammar checking code as physically close to the users as possible. That way, we can serve up suggestions faster than anyone else. To achieve our goal, we make sure that everything is well-optimized and lean. Harper barely uses any CPU or memory at all.

What does all this mean in the context of the recent uptick in user acquisition? Imagine, for a moment, we are LanguageTool. Our software is written in Java, with questionable attendance to the efficiency of our code. Since our software requires a big server to run, we have a certain number of servers on standby to handle a certain amount of expected load.

If suddenly the number of requests we received were to double (as it did for the Chrome extension), we would be big trouble. To avoid increasing the latency on requests (or dropping requests all together), we would need to scale up the number of running servers. This not only takes hiring an expert in cloud architecture, but also additional funding to pay the AWS (or whatever) bill at the end of the month.

Because Harper runs at the edge (no server required), we don't have to worry about that. In fact, I only noticed that our user count had spiked after I visited Hacker News the morning after. No hiccups at all.

Lots of cloud providers like to brag about being able to scale with their users. I like to brag about not having to scale at all.