```LLM 结构化输出手册```

```LLM 结构化输出手册```
LLM Structured Outputs Handbook

原始链接: https://nanonets.com/cookbooks/structured-llm-outputs

大型语言模型（LLM）在自动化和智能体开发方面潜力巨大，但它们偶尔未能持续产生结构化输出（如JSON、XML或代码）阻碍了实际应用。虽然LLM *通常* 生成有效的语法，但其概率性本质给依赖它们执行数据提取和工具调用的开发者带来了可靠性问题。本手册旨在解决这一挑战，提供一份关于确保确定性、结构化LLM输出的全面且*更新*的资源。它涵盖了底层机制、最佳工具和技术、系统构建/扩展、速度和成本优化以及输出质量改进。本手册由Nanonets-OCR和docstrange团队创建，旨在整合当前分散在研究论文、博客和代码仓库中的快速发展信息，既作为学习指南，也作为开发者的实用参考。

## LLM 结构化输出：摘要最近的 Hacker News 讨论强调了“LLM 结构化输出手册”（nanonets.com）作为开发者有价值的资源。该指南详细介绍了确保大型语言模型 (LLM) 产生可预测格式输出的技术，这对于可靠地集成到更大的系统（如代理和管道）中至关重要。主要收获包括“受约束的非确定性”的优势——即使内容不总是准确，也能保证语法的正确性。这对于资源受限的环境特别有用，例如在 Raspberry Pi 等设备上本地运行模型，在严格的输出约束下（例如“是”或“否”），即使是小型模型也可以有效。讨论还指出了几个辅助结构化生成的库：Outlines、Guidance 和 XGrammar。TOML 和 TOON 等 JSON 的替代方案也被探索，以期获得更好的可靠性和效率，但支持它们优越性的证据尚有争议。基于正则表达式的 XML 模式强制执行是另一种可行的方法，尤其是在代码生成方面。最终目标是避免解析错误，并确保从 LLM 获得一致且机器可读的输出。

原文

LLMs mostly produce syntactically valid outputs when we try generating JSON, XML, code, etc., but they can occasionally fail due to their probabilistic nature. This is a problem for developers as we use LLMs programmatically, for tasks like data extraction, code generation, tool calling, etc.

LLMs came with the promise of agents and automation. But without structured outputs, it’s just a pipe dream.

There are many deterministic ways to ensure structured LLM outputs. If you are a developer, this handbook covers everything you need.

What happens under-the-hood?
What are the best tools & techniques?
How to pick the right tools & techniques?
How to build, deploy, and scale systems?
How to optimize for latency and cost?
How to improve the quality of output?

Motivation

Structured generation is moving too fast. Most resources you find today are already outdated. You have to dig through multiple academic papers, blogs, GitHub repos, and other resources.

This handbook brings it all together in a living document that updates regularly.

How to use this

You can read it start-to-finish, or treat it like a lookup table.

Who are we?

We're the maintainers of Nanonets-OCR models (VLMs to convert documents into clean, structured Markdown) and docstrange (open-source document processing library).

Subscribe to our newsletter
Updates from the LLM developer community in your inbox. Twice a month.
Developer insights
Latest breakthroughs
Useful tools & techniques

```LLM 结构化输出手册``` LLM Structured Outputs Handbook

```LLM 结构化输出手册```
LLM Structured Outputs Handbook