Aurora DSQL:可扩展、多区域 OLTP
Aurora DSQL: Scalable, Multi-Region OLTP

原始链接: http://muratbuffalo.blogspot.com/2026/07/aurora-dsql-scalable-multi-region-oltp.html

最近发布的 Aurora DSQL 论文揭示了一种将传统单体数据库“解构”为五个专门化、可独立扩展服务的架构:查询处理器(Query Processors)、存储节点(Storage Nodes)、裁决器(Adjudicators)、日志(Journals)和交叉开关(Crossbars)。 该系统基于几项大胆的工程技术选择: * **同步时钟:** 利用 AWS TimeSync 实现无需协调的读取。 * **乐观并发控制(OCC):** 消除悲观锁以避免“慢锁持有者”问题,尽管这在高争用情况下可能导致事务中止。 * **线性一致性(Linearizability):** 优先考虑强一致性而非最终一致性,以简化业务逻辑。 * **事务护栏:** 对事务大小设置硬性上限,以确保可预测的尾部延迟。 * **线性化两阶段提交(Linearized 2PC):** 优化分布式事务以最小化网络往返次数。 其核心收益在于构建了一个计算、提交逻辑与存储可独立扩展的系统,实现了近乎即时的强一致性读取和高效的提交。通过复用 PostgreSQL 引擎和现有的 AWS 基础设施而非从头构建,团队成功实现了这一复杂的全球化设计。尽管在处理“热点键”时可能存在冲突,但 Aurora DSQL 代表了向解耦、高性能数据库架构的一种成熟转型。

抱歉。
相关文章

原文

The Aurora DSQL paper finally dropped. Reading it yesterday was an interesting experience, because I spent two years (2022-23) working with the AWS team that designed and built Aurora DSQL. Since I have been very familiar with the architecture, the paper's overview description of the system didn't excite me much. And if I am being honest, reading the paper also felt a bit dry, most likely because I am not doing the usual extra thinking to explore/understand the ideas in the paper. But taking a step back today, and leaving my subjective experience aside, I will try to elaborate on how the DSQL architecture is actually built on a set of aggressive and opinionated engineering bets. 

If I had to explain this architecture in a single sentence (just as I used to do for other teams at AWS) it would be this: We took a traditional monolithic database and blew out every single component into an independent, horizontally scalable service.


Exploding the Monolith

DSQL divides the database into the following specialized services:

  • Query Processors (QP): Stateless virtual machines that run a custom PostgreSQL engine to parse queries and buffer writes locally.
  • Storage Nodes: Sharded nodes that hold the data and use multiversion concurrency control (MVCC) to serve historical data to QP instantly.
  • Adjudicators: The conflict-resolution layer that checks if a transaction handed of to them by QP is safe to commit.
  • Journals: A highly available replication log that durably saves transactions across zones or regions. (Good for short term durability before this reaches the storage nodes. See my MemoryDB review.) 
  • Crossbars: The routing layer that reads the updates from the Journals and sends changes to the right Storage Nodes.

The Big Architectural Bets in DSQL Design

Well, in addition to fully committing to the disaggregation principle, here are the other big bets in DSQL design.

The Synchronized Clock Bet: To get reads without coordination, DSQL relies entirely on highly synchronized physical clocks, in this case AWS TimeSync. A QP just checks its local clock and asks storage for data from that exact microsecond.

No Pessimistic Locking: Optimistic Concurrency Control (OCC) may lead to high abort rates for databases, but DSQL makes it work by pairing it with MVCC under Snapshot Isolation. Because readers look at a snapshot of the past, read-write conflicts are impossible. Awesome, but what about write-skew? When needed, customers should just use FOR UPDATE, and also design their schemas to force write-write conflicts for business logic violations.

Eventual Consistency is Dead: DSQL provides strong consistency (linearizability), arguing that developers simply cannot write correct business logic on eventually consistent systems. It’s a subtle point, but linearizability (a guarantee about single-object real-time operations) and snapshot isolation (a guarantee about multi-object transaction visibility) control different things, as Jepsen's consistency models outline. DSQL offers a consistent snapshot for your snapshot-isolated transactions, and that individual key operations strictly respect real-time ordering.

Forcing Guardrails: DSQL hard-caps transactions at 3,000 rows and 10MiB. The paper cites Little's Law to justify this, essentially forcing users to accept smaller transactions in exchange for highly predictable stable tail latency.

Linearized 2PC: For transactions that span multiple Adjudicators, traditional Two-Phase Commit (2PC) is too slow over wide area networks as it requires 2RTTs. DSQL uses a "Warp-inspired" trick where Adjudicators vote, but only the leader writes the final commit to its single Journal. This avoids coordinating multiple logs.

The Payoff

Independent Scalability: Compute, commit logic, and storage are completely separated. If you need more read capacity, you add storage nodes. If you have a spike in connections, the system spins up more QPs. You can also tune/optimize them separately, for example, potentially reconfigure adjudicator-range placement based on access patterns.

0-RTT Consistent Reads: Because the QP assigns a local timestamp and storage handles the rest, reading data requires zero coordination with a leader. It is almost as fast as your network latency to the storage node. This is a big win because in OLTP SQL, reads are significantly more common than writes. Even most writes (like UPDATES or INSERTS with unique indexes) are actually reads first.

1-RTT (or 1.5 RTT) Commits: Whether you write one row or a hundred, coordination only happens once at commit time, costing just 1 RTT (or slightly more for a multi-shard commit).

No "Slow Lock Holder" Problem: Because there are no pessimistic locks, a developer going to lunch with an open transaction terminal (exact quote from the paper) cannot bring down the database. Readers never block writers, and writers never block readers.

While it is hard to critique my old team, this won't be a true Murat Buffalo (now Bay Area?) review without pointing out the tradeoffs and shortcomings in the paper. Because of the long read-modify-commit duration during a transaction, DSQL may be prone to write-write conflicts on hot keys. This limits how many back-to-back operations you can squeeze into a single row (which is bad application design anyway). Under heavy contention, transactions will abort, whereas a traditional database may have queued them up. If two regions write to the same key, OCC will abort one of them, and unfortunately this conflict is only detected at commit time, meaning you pay the WAN network latency penalty before finding out you have to retry.

While the paper couldn't provide extensive evaluation, quantitative data on user adoption or business payoff, it does provide a candid Lessons Learned section which talks about some friction with respect to Foreign Key Constraints and high-locality sequences.

Building at Scale

I have one final takeaway that is somewhat counterintuitive: building a completely novel global production database didn't actually feel like as much effort as it should have.

Yes, it was a lot of work, but the execution felt surprisingly smooth after the team got going. I attribute this to two things: a great upfront design, and a deliberate choice to avoid reinventing the wheel. Rather than building everything from scratch, the team used the PostgreSQL engine for SQL parsing, execution, and the client protocol, while discarding its local storage and transaction processing layers. We didn't build a new replication log; we used AWS's existing internal Journal service. We were also equipped with hard-earned lessons from preceding AWS database projects, like JournalDB and the unfortunately named QLDB (Quantum Ledger Database).  Beyond the tooling, the team dynamics were excellent. Marc Brooker is technically brilliant, and he did a masterful job leading a talented team of principal engineers. Our weekly whiteboard thinking sessions were a lot of fun. Because the foundational architecture was so well-designed, the actual development felt significantly easier than at least what I would expect for a product of this magnitude. (But then again, I wasn't with the team for the last year of the project!)

联系我们 contact @ memedata.com