Unicode的转写规则是图灵完备的。
Unicode's transliteration rules are Turing-complete

原始链接: https://seriot.ch/computation/uts35/

Nicolas Seriot 证明了 Unicode 技术标准 #35 (UTS #35) 中的音译规则(通常用于将“é”转换为“e”等简单的文本转换)是**图灵完备的**。 通过利用该规范的“重访游标”(revisiting cursor)功能,这些规则可以实现复杂的计算模型,包括 2-标签系统、Collatz 函数、110 号规则元胞自动机以及素数生成算法。由于音译引擎在规范中没有限制,因此无法判定给定的规则文件是否会终止。 虽然广泛使用的 ICU 库包含一个务实的“循环限制”来防止无限执行,但其底层的规则本质上是可执行代码。Seriot 的研究发现,在大多数操作系统、浏览器和数据库中被隐式信任并内置的区域设置(locale)数据文件,实际上具备执行任意计算的能力。这揭示了一个重大的安全隐患:接受来自不受信任来源的音译规则文件,等同于执行任意代码,因此应以同等的安全严谨度进行对待。

一项近期的发现指出,用于文本处理的 Unicode UTS #35 音译规则系统是图灵完备的。研究人员通过证明仅需三条重写规则即可计算考拉兹猜想(Collatz conjecture),再次揭示了一个在看似平凡的数据处理格式中出现的“意外”图灵完备性案例。 Hacker News 上的讨论阐明了几个关键点: * **安全影响:** 虽然该规则引擎在技术上是图灵完备的,但它并不构成现实的安全威胁。用户无法通过标准输入执行任意代码;该引擎需要加载自定义的、不可信的规则集,而这本身就意味着主机系统已经遭到入侵。 * **实用性:** 诸如 ICU 库(在操作系统和 .NET 等运行时中广泛使用)之类的实现,已包含重写次数限制等安全机制,以防止无限循环并确保程序能够终止。 * **更广泛的趋势:** 评论者指出,图灵完备性经常出现在配置繁杂的复杂系统中——从 Microsoft Word 的自动更正到基于字体的语言模型(LLM)。专家认为,随着数据处理规范为了应对全球语言需求而变得日益灵活,防止它们成为计算通用系统变得越来越困难,使得“可判定性”反而成了例外,而非惯例。
相关文章

原文
Unicode's Transliteration Rules Are Turing-Complete

Nicolas Seriot

Computation > Unicode's Transliteration Rules Are Turing-Complete

July 2026

See also: Jira is Turing-Complete

Table of Contents

  1. Transliteration Rules
  2. 2-Tag Systems
  3. The Collatz Function
  4. Correctness and Universality
  5. ICU's Rewrite Guard
  6. Rule 110
  7. Prime Numbers
  8. Conclusion
  9. Appendix: Files

I've been wondering for a while whether Unicode allows universal computation. The core Unicode algorithms (normalization, casing, bidi, collation) are deliberately bounded, but UTS #35 transliteration rules, under their natural unbounded semantics, are not. This is a result I haven't found published before.

These rules ship as locale data in ICU, the widely used Unicode/globalization library used in most operating systems, browsers, runtimes, and databases. Whether a given rule file terminates on a given input is undecidable.

1. Transliteration Rules

A transliterator typically turns "é" into "e", using a list of ordered rewrite rules:

L { x } R > y ;

The substring x is replaced by y when it sits between (optional) contexts L and R. The revisiting feature allows | in the replacement, which places the cursor inside the new text so that newly written material can trigger further rules.

Example:

x > y | z ;
za > w ;

xa rewrites to y|za (cursor before z). The engine rescans and za matches, producing yw.

Using the Python PyICU module:

from icu import Transliterator as T
t = T.createFromRules("", "x > y|z; za > w;")
print(t.transliterate("xa")) # yw

Here is the Latin-Katakana transform. It uses contexts, capture groups, quantifiers and the cursor. Before i or e, c rewrites to s and the cursor backs up so the s rules re-fire. Same revisiting trick as above, shipped in production locale data.

c } i → | s ;
c } e → | s ;

2. 2-Tag Systems

To prove UTS #35 universality, we compile a 2-tag system (Post, 1943) into transliteration rules, a model proven universal (Cocke & Minsky, 1964).

A 2-tag system has one production per letter. Each step removes the first two letters and appends the production of the first one. It halts when fewer than two letters remain.

3. The Collatz Function

Our example is Liesbeth De Mol's 2-tag system for the Collatz function (even n → n/2, odd n → (3n+1)/2): a → bc, b → a, c → aaa, on the unary word aaa...a. We prefix the word with a read marker M, which pins the machine to the front. When no rule matches at M, no rule matches anywhere.

The construction uses one rule per letter:

M a [abc] ([abc]*) > | M $1 b c ;
M b [abc] ([abc]*) > | M $1 a ;
M c [abc] ([abc]*) > | M $1 a a a ;

The first rule matches the marker, the letter a, one more letter, then captures everything else. The replacement writes the next configuration and puts the cursor back before the marker so the next step fires immediately.

One rule application

The character class, the capturing group and $1, the quantifier and the cursor are standard rule syntax (the spec's Transform Syntax Characters table).

You can run this machine with uts35.pycollatz.txt is the rules above with the | deleted, so each pass performs exactly one tag step. From aaa, the run replicates the worked example on the Wikipedia tag system page (aaa, abc, cbc, caaa, aaaaa, ...) with the values appearing as runs of as. The same rules also run with no Python at all, through ICU's stock uconv (uts35.sh). test.sh checks the machine against expected outputs.

% python3 uts35.py collatz.txt aaa
ICU 78.3
  0 - Maaa         # 3
  1 - Mabc
  2 - Mcbc
  3 - Mcaaa
  4 - Maaaaa       # 5
  5 - Maaabc
  6 - Mabcbc
  7 - Mcbcbc
  8 - Mcbcaaa
  9 - Mcaaaaaa
 10 - Maaaaaaaa    # 8
 11 - Maaaaaabc
 12 - Maaaabcbc
 13 - Maabcbcbc
 14 - Mbcbcbcbc
 15 - Mbcbcbca
 16 - Mbcbcaa
 17 - Mbcaaa
 18 - Maaaa        # 4
 19 - Maabc
 20 - Mbcbc
 21 - Mbca
 22 - Maa          # 2
 23 - Mbc
 24 - Ma           # 1

4. Correctness and Universality

  1. At most one rule matches. There is exactly one marker. The letter after it selects the rule. ([abc]*) captures all remaining letters.
  2. One rewrite is exactly one tag step. The rule for letter x matches precisely when the marker faces x plus at least one more letter. The replacement constructs the next configuration.
  3. Halting corresponds. Every rule requires two letters after the marker, so Ma and M are fixed points. The transform terminates exactly when the tag system halts.

Together, by induction: after k rewrites the string is exactly M followed by the tag system's word after k steps, and the transform reaches a fixed point exactly when the tag system halts. Nothing here is specific to Collatz. One rule per letter compiles any 2-tag system, so a universal one yields a fixed rule file that simulates any Turing machine, encoded in the initial word.

5. ICU's Rewrite Guard

ICU stops each transliterate() call after 16 rewrites per input code point (loopLimit = span << 4 in rbt.cpp; the Java port has the same guard). However, the specification itself defines no limit. The guard is ICU's pragmatic addition to prevent infinite computation, as termination is undecidable. Here, each rewrite performs a full tag step, so iterating until the string stabilizes is safe.

6. Rule 110

The runner is not limited to tag systems. Any rule file is a program. rule110.txt implements the Rule 110 cellular automaton in 14 rules. Cells are written . (0) and * (1). A head carries the previous two cells and rewrites each cell in place. One pass is one generation. One fuel g per generation, spent into s; the run halts by itself when the fuel is out.

python3 uts35.py rule110.txt "ggggggggg*"
ICU 78.3
  0 - Mggggggggg*
  1 - Mggggggggs**.
  2 - Mgggggggss***..
  3 - Mggggggsss**.*...
  4 - Mgggggssss*****....
  5 - Mggggsssss**...*.....
  6 - Mgggssssss***..**......
  7 - Mggsssssss**.*.***.......
  8 - Mgssssssss*******.*........
  9 - Msssssssss**.....***.........

7. Prime Numbers

primes.txt is Wolfram's real-time prime-generating cellular automaton (A New Kind of Science, p. 640); 16 states (0-f) and 223 transform rules. The first cell after the fuel is 0 exactly at prime ticks.

% python3 uts35.py primes.txt gggggggggggg0a048
ICU 78.3
  0 - Mgggggggggggg0a048
  1 - Mgggggggggggs9604d7
  2 - Mggggggggggss06f5d80
  3 - Mgggggggggsss0ad3d870
  4 - Mggggggggssss96fc0d700
  5 - Mgggggggsssss0adb008000
  6 - Mggggggssssss96fad087000
  7 - Mgggggsssssss0a960f870000
  8 - Mggggssssssss9af6f01700000
  9 - Mgggsssssssss9adad018000000
 10 - Mggssssssssss96f60f187000000
 11 - Mgsssssssssss0a06f02870000000
 12 - Mssssssssssss96fad02d700000000

8. Conclusion

Transliteration rules were designed to turn "é" into "e". Three lines of them can compute the Collatz function.

Unbounded rewriting with a revisiting cursor is an old recipe for universality. The surprise is that it lives in a data format for locale files, shipped in every OS, whose specification doesn't mention the possibility.

The above discussion demonstrates that a transliteration rule file is not just data, it's a program. If you accept transform rules from outside, you are accepting code, which should be reviewed and bound at runtime, as ICU already does.

Appendix: Files

  • collatz.txt — the three-rule Collatz machine (one tag step per pass)
  • rule110.txt — Rule 110 in 14 rules
  • primes.txt — Wolfram's prime-generating cellular automaton in 223 rules
  • uts35.py — runner, PyICU
  • uts35.sh — runner, ICU's stock uconv, no Python
  • test.sh — self checks

Environment: ICU 78.3, PyICU 2.16.2, macOS; also verified with ICU 72.1 on Debian 12; July 2026

联系我们 contact @ memedata.com