从头开始构建法学硕士：3 小时的编码研讨会 Building LLMs from the Ground Up: A 3-Hour Coding Workshop

原始链接: https://magazine.sebastianraschka.com/p/building-llms-from-the-ground-up

这是一个时长 3 小时的视频教程，介绍了理解大语言模型 (LLM) 的实践方法。 该视频涵盖了这些模型的实施、训练和使用，首先介绍了它们的基础知识。 它提供了分步指南，包括创建自定义分词器类、基于 GPT-2 和 Llama 等现有模型设置模型架构、加载预训练权重、执行指令微调、基准评估和测试会话性能。 该视频使用Python编程语言，所有必要的材料都可以在其随附的GitHub存储库和Lightning Studio资源中找到。 此外，研讨会还利用了 LitGPT 库，该库有一个单独的 GitHub 存储库。 本教程允许观看者从头开始构建自己的法学硕士。

This is a 3-hour video tutorial presenting a hands-on approach to understanding large language models (LLMs). The video covers implementing, training, and utilizing these models, starting with an introduction to their basics. It provides a step-by-step guide through creating a custom tokenizer class, setting up model architecture based on existing models such as GPT-2 and Llama, loading pretrained weights, performing instruction fine-tuning, benchmark evaluations, and testing conversational performance. The video uses Python programming languages, and all necessary materials can be found in its accompanying GitHub repository and Lightning Studio resources. Additionally, the workshop also leverages the LitGPT library which has a separate GitHub repository. This tutorial allows viewers to build their own LLMs from scratch.

---