我把杰夫变成了一个(糟糕的)聊天机器人。
I turned Jev into a (lousy) chatbot

原始链接: https://github.com/kyle-pena-nlp/jevchat/

**jevchat** 是一个实验性的、高度可定制的聊天界面,通过要求 Jev 模型预测序列中的下一个符号来生成文本。该工具通过从概率分布中反复采样,逐个字符或标记地构建响应,直到达到停止信号。 该项目提供了多种查询模型的策略,从简单的 **choice**(请求下一个符号)到 **bisect**(使用是非拆分)以及 **buckets**(管理大型词汇表)。用户可以切换字母表、采样策略和展示模式——例如“假设”模式,该模式通过让模型对完整的字符串而非孤立的符号进行排序,显著提高了准确性。 尽管运行起来既不切实际且成本高昂,但该工具专为娱乐和实验而设计。用户可以通过实时终端界面进行交互,该界面会显示实时的生成统计数据和概率分布。代码库经过充分测试,提供了用于性能比较的 `bench` 命令,并可通过 `.env` 文件进行灵活配置。

最近在 Hacker News 上,一位用户分享了他利用 Jev 构建“糟糕”聊天机器人的实验,引发了关于人工智能开发本质及大语言模型(LLM)实验当前趋势的热烈讨论。 参与者将该聊天机器人反复试验的行为比作《瑞克和莫蒂》中的“死亡水晶”场景,并指出,业余爱好者将此类概念转化为项目的高速模式,营造出了一种竞争激烈、“快节奏”的氛围。尽管一些用户认为该项目是微不足道或“毫无意义”的练习,但另一些人则为其辩护,认为这是必经的过程,也是了解模型底层架构的宝贵途径。 讨论重点突出了以下几个主题: * **性能与效率:** 虽然主流大语言模型功能更强大,但 Jev 提供的速度和成本效益对特定的小众用例更具吸引力。 * **人工智能的“杰文斯悖论”:** 用户指出,工具的快速进步导致了“跑分至上”和实验性代码的激增,且往往是在重复过去的迭代。 * **递归式调试:** 该讨论还涉及了利用一个大语言模型去解释或调试另一个大语言模型的元体验,并将其比作“给狗按按钮”。
相关文章

原文

We know Jev.

jevchat turns that into a chat model. At every step it asks Jev one question:

Given the user's question and the reply written so far, which symbol comes next?

The options are an alphabet plus an option to stop emitting. Jev returns a probability for each one, and the sampler draws the next symbol from that normalised distribution. Append, repeat, and stop when STOP is drawn.

There are several alphabets and sampling strategies available.

The idea is for fun, the cost is somewhat impractical, and the results are hilarious.

image

This was a Claude accelerated experiment. I described the sampling algorithms, strategies, and so on, and it implemented them.

Put your Jev key in .env next to pyproject.toml (git-ignored):

JEV_API_KEY and TYPESAFE_API_KEY are also accepted. Values in .env win over exported ones, so editing the file is enough to switch keys.

poetry run jevchat                          # interactive chat
poetry run jevchat ask "do people need water?"
poetry run jevchat alphabets                # what you can sample from
poetry run jevchat bench                    # compare every mode (table below)

The reply appears as it is sampled, in a panel with a live readout of the generation rate — symbols/s, characters/s, milliseconds per API call, elapsed time — and the top few symbols Jev scored at the last step, so you can watch the distribution the sampler is drawing from.

Ctrl-C cancels. The first press stops generation once the in-flight request returns and keeps the partial reply; a second press aborts immediately. In chat, the partial reply stays in the conversation history. ask exits 130 when cancelled.

Chat commands: /help, /alphabet [name], /temp <v>, /stop-bias <v>, /reset, /stats, /exit.

Two things are swappable: how the distribution over the next symbol is obtained (-s/--strategy), and what it is over (-a/--alphabet). Every combination below is a runnable command.

choice asks one question over the whole alphabet. bisect sorts the alphabet and asks earlier/later yes-no questions until the group is small, then asks one choice question inside it.

# choice — one question over the whole alphabet (the default)
poetry run jevchat -s choice ask "how many eyes do people have?"

# ...without the re-ordering that cancels Jev's position bias (worst mode)
poetry run jevchat -s choice --no-shuffle-criteria ask "how many eyes do people have?"

# ...averaging 4 re-orderings, sent as 4 parallel questions in one request
poetry run jevchat -s choice --ensemble 4 ask "how many eyes do people have?"

# bisect — earlier/later down to groups of 20, each split asked both ways
poetry run jevchat -s bisect ask "how many eyes do people have?"

# ...cheaper: bigger groups, each split asked once
poetry run jevchat -s bisect --bisect-cutoff 32 --no-bisect-swap ask "how many eyes do people have?"

# buckets — the alphabet split across many questions, each with an OTHER escape.
# The only strategy that can hold more than 255 symbols.
poetry run jevchat -a words1k -s buckets ask "what colour is snow?"
poetry run jevchat -a bpe5k -s buckets --bucket-size 127 ask "what is the capital of france?"

# refine — buckets, then a question over the winners, then a rescored nucleus.
# Twice the probability on the right symbol and ~19x the vocabulary resolved.
poetry run jevchat -a words1k -s refine ask "where do fish live?"
poetry run jevchat -a words1k -s refine --refine-nucleus 6 --refine-rounds 2 ask "…"
# hypothesis — options are the resulting texts (the default)
poetry run jevchat -p hypothesis --window 40 ask "what colour is snow?"

# symbol — options are the bare symbols, as the first version of this did
poetry run jevchat -p symbol ask "what colour is snow?"
# keep 3 candidate replies alive instead of committing symbol by symbol
poetry run jevchat -b 3 ask "what is the opposite of hot?"

Costs one score per live beam per step. Above width 1, temperature, top_p and top_k stop applying — beams are ranked by probability, not drawn from.

poetry run jevchat -a lower26 -t 0 ask "what is 2+2?"   # a-z and space only
poetry run jevchat -a ascii   -t 0 ask "what is 2+2?"   # spells anything
poetry run jevchat -a tokens  -t 0 ask "do people need water?"   # whole words

# these three exceed 255 options, so they need --strategy buckets
poetry run jevchat -a words1k -s buckets -t 0 ask "what colour is grass?"
poetry run jevchat -a bpe2k   -s buckets -t 0 ask "where do fish live?"
poetry run jevchat -a bpe5k   -s buckets -t 0 ask "what do bees make?"
poetry run jevchat -a tokens -s bisect --bisect-cutoff 20 ask "do people need water?"
poetry run jevchat -a ascii -s choice --ensemble 12 -t 0.2 --repetition-penalty 1.0 \
    ask "what colour is grass?"

There are two ways to ask Jev the same question. Under --presentation symbol the options are the symbols themselves — 'a', 'i', ' the' — and Jev has to append the option to the reply in its head before judging it. The instructions used to say exactly that: "judge grammar and spelling on the concatenation, not on the option on its own."

Under --presentation hypothesis the options are the resulting texts:

answer_so_far = "The capital of France is Par"

symbol      options:  'a'  'i'  's'  …  STOP
hypothesis  options:  '…he capital of France is Para'
                      '…he capital of France is Pari'
                      '…he capital of France is Pars'
                      '…he capital of France is Par'     <- unchanged: this is STOP

The append is already done, so Jev only ranks finished strings — which is what a decision model is built for. It is the single largest improvement in the project: on character alphabets it roughly triples top-1 and doubles the probability mass landing on the right symbol, for fewer input tokens than symbol options with their per-option descriptions.

158 tests, all offline — a scripted fake client for the generation loop and an httpx.MockTransport for the HTTP layer. No API key and no network needed. jevchat bench is the part that does hit the API.

联系我们 contact @ memedata.com