August 24, 2026 · Anup Talwalkar · Polign

I've been thinking about agent memory for quite some time. There are really two things that bother me. The first is how we represent memory in the agent space. The second is where that memory should live when the agent is running on a hardware constraints. Somehow my work converged both of the concerns into a single answer.

1. The memory needs to be deterministic when dealing with LLMs

What I've observed in my own agent use is that there are certain behaviors and patterns that LLMs can't follow and sometimes miss or forget. I would correct a fact and the agent would quote the old version a week later. I would change a preference and recall would return both versions, and the model had to guess which one I meant. I realized something. Right now we make the LLM sort this out by rereading old text, which costs tokens when it works and accuracy when it tries to rely on semantic retrieval. A schema with types and supersession rules answers the same question instantly.

When a model manages its own memory, it has to decide whether the recalled text is new or a correction, whether the instructions contradict something already stored. These are mostly schema decisions, and shouldn't be left to the model to figure out.

A typed memory store moves those decisions out of the model and into the schema. The model extracts a fact; the database determines what that fact means.

2. Serverless/cold-first database that must be accessible on a small hardware

The edge angle comes from something I keep returning to. During my time at Google Cloud Storage, I saw how much tooling and infrastructure was needed to support simple semantic storage use cases. The operational cost became obvious once we realized how the index and semantic storage would scale with the growing corpus. For example, hot vector index (along with availability guarantees via replication) in RAM or even SSDs is expensive, and isn't really needed for users with infrequent and cold queries. Every memory design I had seen assumed a search cluster or a managed API on the other side of a network link. Meanwhile embedding models were already running fine on CPU cycles and small models are getting better every few months.

I ended up building polign_db as a typed database on top of the hybrid vector + BM25 engine and living in an object store. The polign_db server holds nothing durable, so processes and machines can die and restart and the memory stays intact. Initially while working on polign_db, I wanted it to serve search from object storage with as little resident state as possible, and with query costs that do not grow with the corpus size. Those choices were about cloud bills, but they also happened to be what small devices need.

As a side note, the Wikipedia search demonstrates polign_db. The demo serves 12.5 million passages from S3 while the server idles at about 37 MiB RSS, and the whole demo, embedder and web app included, fits on a 2 GB ARM machine. Most of that memory goes to embedding generation and query concurrency, not the database.

Running the agent memory demo

The footprint for this demo is small. You need three things to run it:

• polign-server: curl -fsSL https://get.polign.com | sh
• Go, since the demo runs with go run
• an Anthropic or OpenAI API key, exported as ANTHROPIC_API_KEY or OPENAI_API_KEY

Then point the demo at a local directory:

./run-demo.sh fs:./demo-bucket

Claude models are the default. To use an OpenAI model instead, pass the model id and the provider is inferred from it:

./run-demo.sh fs:./demo-bucket -model gpt-5

Tell it something:

you> I use Vim as my editor.
  → remember_preference({"subject":"user","predicate":"prefers_editor","value":"vim"})
  ← {"stored":{"id":"m-...","value":"vim","status":"active",...}}

Numbers work as numbers:

you> Is my step goal above 8000?
  → recall({"subject":"user","predicate":"daily_step_goal","value_min":8000})
  ← {"count":1,"records":[{"value":9000,...}]}

That comparison is running in the database. The model never had to look at a paragraph and decide whether 9000 is more than 8000.

Semantic recall sits next to the filters. The demo embeds queries with a small local model and searches the same records. The indexing, validation and storage is handled by polign_db.

The local agent stack

The stack I expect to see more of is a pile of small, boring parts:

┌─────────────────────────────┐
│         Local LLM           │
│         conversation        │
└──────────────┬──────────────┘
               │
┌──────────────▼──────────────┐
│        Typed memory         │
│                             │
│ validation                  │
│ supersession                │
│ structured filtering        │
│ semantic recall             │
└──────────────┬──────────────┘
               │
┌──────────────▼──────────────┐
│      Durable local store    │
│        /var/lib/agent       │
└─────────────────────────────┘

The embedder and the database can already run on small hardware today. As local models improve, the whole agent will be able to move onto the device, and at that point memory does not need to be an afterthought or a service subscription. It is a typed interface over storage you own.

I kept the agent memory demo open source, but the underlying polign_db is closed source. The demo and the downloads are not gated and freely available to download and install. The getting started guide covers the installation steps.

Leave a note and your email on the signup page and I will follow up as soon as I can (usually within a few hours/a day).