<LLM: Learning by Building from Scratch> Commentary Lecture
This course covers the GitHub notebooks and bonus content for <Building an LLM from Scratch> (Gilbut, 2025). GitHub: https://github.com/rickiepark/llm-from-scratch/
<Building an LLM from Scratch> is the Korean translation of the bestseller <Build a Large Langauge Model (from Scratch)> (Manning, 2024) written by Sebastian Raschka. Starting from scratch and building a complete version of OpenAI’s GPT-2 model, this book provides a way to learn how large language models work and how to use them.
I studied in the following order:
Challenge lecture attendance - Book reading - Source analysis lecture attendance
I will continue with Large Language Models Core Concepts Quickly.
5.0
김진철
14% enrolled
Working hard.....😊
5.0
든든한꼬마
65% enrolled
Thank you for the great lecture.
What you will gain after the course
Starting from scratch, implement a complete LLM entirely in code.
Learn the core components that make up LLMs, including transformers and attention.
Learn how to pretrain an LLM similar to GPT.
Learn how to fine-tune an LLM for classification.
Learn how to fine-tune an LLM to respond according to human instructions.
Discount news
If you register for <Deep Learning from the Keras Creator, 3rd Edition> Complete Reading Challenge (https://inf.run/DPba3), you will receive a 50% discount coupon for this course!
<Hands-On Machine Learning with Scikit-Learn and PyTorch> Complete Reading Challenge (https://inf.run/5SDwX), we'll give you a 50% discount coupon for this course!
This course explains the example code provided with <Learn LLMs by Building Them from Scratch>. The GitHub repository (https://github.com/rickiepark/llm-from-scratch/) includes not only the example code from the book but also a variety of supplementary materials. Explanations of this supplementary content are also provided.
Anyone can take this course without purchasing the book. However, it is most effective when taken alongside the book. Some code explanations may be difficult to understand without referring to the book. The prerequisite knowledge required is Python programming. Experience with deep learning and PyTorch will be helpful. If you are new to both concepts, please read Appendix A first.
Follow the code line by line, and you’ll build your own GPT! A hands-on guide to implementing GPT from scratch and mastering the principles of LLMs through practice
Complex concepts are explained through illustrations, and you learn by building an LLM yourself. This book is a hands-on introduction to LLMs that guides you through learning the structure and operating principles of large language models by implementing them from start to finish. Rather than simply explaining concepts, it begins with text preprocessing, tokenization, and embeddings, then gradually builds self-attention, multi-head attention, and Transformer blocks. It goes on to integrate these components into a working GPT model and explores firsthand the key elements of modern architecture design, including the number of model parameters, techniques for stabilizing training, activation functions, and normalization methods. It also provides an in-depth guide to pretraining and fine-tuning. You will practice pretraining on unlabeled data, tuning the model for downstream tasks such as text classification, and applying instruction-based learning techniques that have recently gained attention. The book also covers cutting-edge topics such as LoRA-based parameter-efficient fine-tuning (PEFT), offering broad guidance on how to connect LLMs to real-world services and research. Every concept is implemented in PyTorch code and optimized for hands-on practice in a standard laptop environment. By following the implementation process in this book, you will naturally understand what happens inside an LLM and gain an intuitive, hands-on grasp of how the mechanisms of large language models work.
Recommended for these people
Who is this course right for?
Those who want to gain a detailed understanding of how large language models (LLMs) work
Those who want to pretrain and fine-tune LLMs using PyTorch and the transformers package
People who want to know the architecture of OpenAI’s GPT-2 model
For those who can’t rest until they’ve tried making everything themselves!
Need to know before starting?
Basic knowledge of Python programming is required.
Although I majored in mechanical engineering, I have worked continuously reading and writing code since graduating. I am a Google AI/Cloud GDE. I run the TensorFlow blog (tensorflow.blog) and explore the fascinating boundary between software and science by writing and translating books about machine learning and deep learning.
I have written 『Deep Learning You Study by Building It Yourself』 (Hanbit Media, 2025), 『Self-Study Machine Learning + Deep Learning (Revised Edition)』 (Hanbit Media, 2025), 『Self-Study Data Analysis with Python』 (Hanbit Media, 2023), 『The Art of Conversing with ChatGPT』 (Hanbit Media, 2023), and 『Do it! Introduction to Deep Learning』 (Aegis Publishing, 2019).
『Hands-On Machine Learning with Scikit-Learn and PyTorch』(Hanbit Media, 2026), 『Deep Learning from the Keras Creator (3rd Edition)』(Gilbut, 2026),『LLM Fine-Tuning, Fast and Focused!』(Insight, 2026), 『LLM & AI with PyTorch』(Hanbit Media, 2026), 『Large Language Models, Fast and Focused!』(Insight, 2025), 『Machine Learning, Fast and Focused!』(Insight, 2025), 『Learn LLMs by Building from Scratch』(Gilbut, 2025), 『Hands-On LLM』(Hanbit Media, 2025), 『Machine Learning Q & AI』(Gilbut, 2025), 『Mathematics for Developers』(Hanbit Media, 2024), 『Practical ML Problem Solving with Python』(Hanbit Media, 2024), 『Machine Learning Textbook: PyTorch Edition』(Gilbut, 2023), 『Stephen Wolfram's ChatGPT Course』(Hanbit Media, 2023), 『Hands-On Machine Learning, 3rd Edition』(Hanbit Media, 2023), 『Generative Deep Learning by Building, 2nd Edition』(Hanbit Media, 2023), 『Python That Awakens Your Coding Brain』(Hanbit Media, 2023), 『Natural Language Processing with Transformers』(Hanbit Media, 2022), 『Deep Learning from the Keras Creator, 2nd Edition』(Gilbut, 2022), 『Machine Learning&Deep Learning for Developers』(Hanbit Media, 2022), 『Gradient Boosting with XGBoost and Scikit-Learn』(Hanbit Media, 2022), 『Learn Deep Learning with the Google Brain Team with TensorFlow.js』(Gilbut, 2022), and 『Machine Learning with Python Machine Learning Libraries (2nd Revised Edition)』(Hanbit Media, 2022), among several dozen books translated into Korean.
I studied in the following order:
Challenge lecture attendance - Book reading - Source analysis lecture attendance
I will continue with Large Language Models Core Concepts Quickly.