inflearn logo

Understanding LLM Architecture and GPU Utilization Strategies for AI Masters

Understand Transformer-based LLM architectures and GPU utilization strategies, and get hands-on experience with the actual serving process using vLLM. This course covers the entire practical workflow, from building AI system pipelines to monitoring and utilizing multiple GPUs, and is designed to help you understand everything intuitively through diagrams and hands-on exercises—without complex formulas.

(5.0) 28 reviews

535 learners

Level Basic

Course period Unlimited

GPU
GPU
attention-model
attention-model
AI
AI
transformer
transformer
LLM
LLM
GPU
GPU
attention-model
attention-model
AI
AI
transformer
transformer
LLM
LLM

Reviews from Early Learners

5.0

5.0

WonJune Lee

43% enrolled

I am not in the deep learning industry, but I work in the field of computer vision (rule-based). Since my company requires LLM and vision-related deep learning technologies, I have been studying these topics. I have only completed about 40% of the course, but I felt compelled to leave a review now. I have taken many deep learning courses, including those by famous and highly-rated instructors, but I haven't found any course as clean and clear as this one. The best part is the quality of the lecture materials. The instructor recorded every single matrix calculation in Excel, which is incredibly helpful when reviewing. The Python code is also well-commented in many places. The quality of the lectures themselves is excellent; the instructor reminds you of parts you might have forgotten, ensuring you don't miss anything. While most other courses show calculations once or twice and then move on, this course goes through the calculations together until the end, which provides great clarity. It seems like the Q&A is monitored frequently, as I received immediate answers to my questions. The lectures seem to have been filmed this year, so it's great that they include a lot of the latest trends. It feels like this course hasn't gained much word-of-mouth yet, but I highly recommend it to anyone who needs to study these topics.

5.0

램쥐뱅

100% enrolled

It’s the best. Thank you.

5.0

김민형

31% enrolled

I was surprised by how professional and much more detailed the lecture was than I expected. It feels like it's been a long time since I've seen a lecture of this quality. Since my job involves AI-related engineering, I purchased this thinking I would just roughly study the basics of LLM Architecture, but I am extremely satisfied.

What you will gain after the course

  • Understanding the encoder-decoder architecture and core operating principles of Transformer models

  • Understanding the evolution of modern attention mechanisms such as MHA, MQA, GQA, and MLA

  • Hands-on practice using the vLLM engine, the de facto standard for AI serving today

  • Monitor key performance metrics such as TTFT and TPOT in a vLLM serving environment

  • Design and implementation of a multi-GPU architecture using Tensor/Pipeline/Data Parallelism

  • Understanding the Core Concepts of Agent AI and How Tool Calling Works

  • Experience building AI system pipelines and monitoring performance from a real-world industry perspective

  • Understanding the latest LLM trends, such as MLA, MTP, and Engram, based on the latest research papers

In the age of AI Agents,
practical skills for understanding AI systems are becoming increasingly important.

From Transformer-based LLM architecture
to GPU utilization, vLLM serving, and multi-GPU strategies

Practical LLM Architecture Course

In the era of autonomous AI agents,
you can utilize various agent tools and public APIs, including OpenAI, Claude, and Codex.

However, in real-world service environments,
data security, network costs, token costs, and GPU resource management must all be taken into consideration.

Therefore, what matters is
understanding how to appropriately combine Public APIs and self-hosted GPU-based LLMs
in a Hybrid AI architecture.


If so, is using only Public APIs always the best option?

Not necessarily.

These days, numerous LLMs comparable to public APIs (ChatGPT, Claude, Sonnet, etc.) are being developed both domestically and internationally.



Three models selected based on the first evaluation of domestic sovereign AI


However, understanding and using LLMs well is not easy.
Purchasing expensive GPUs and
using LLMs with an understanding of them versus using them without understanding them makes
a significant difference.

So now is the time to learn the architecture for serving LLMs yourself.


🌟 From LLM Architecture to Serving


In the age of agents, we are now entering the era of inference rather than training. While it is important to know how to use public APIs effectively, many companies prefer to build local serving environments for various reasons, including security, governance, and cost. Learn everything from understanding LLM architectures for building a local LLM serving environment to configuring the architecture and exploring trends in LLM development.


Course Core Structure

Core 1. Understanding Hugging Face Models


You need to know and use the many LLMs published on Hugging Face.
However, the config.json file that provides the specifications of LLM models is no different from an encrypted message to beginners. That’s because you need to understand transformer models to be able to read it.

But don’t worry. After taking this course, you’ll become an expert who can look at and understand the key specifications.

Learn how to decode a config.json file through this course.

(This is the content of Chapter 3-5. Learn all about the remaining key parameters.)


Core 2. Mastering Attention

The beginning and end of the transformer model underlying current LLMs are attention.

The attention model emerged in 2017,
but it still reigns as the most powerful algorithm nearly 10 years later.
Many efforts have been made to move beyond the Transformer architecture,
but no architecture has yet emerged that completely replaces Transformer attention.

⚠️ You absolutely cannot afford to understand attention only superficially.


Fully understand how attention works and learn how it has evolved.

(This is the content of Chapter 5-4. The evolution of attention is the evolution of LLMs.)


Core 3. Mastering Multi-GPU Architectures

Multi-GPU configurations are essential for running large-scale LLMs and achieving fast inference.
But did you know there are several ways to configure multiple GPUs?


We’ll teach you the essential GPU utilization strategies—the crucial gateway to becoming a core AI engineer.




😄 Recommended for people who:

AI beginners

Those who wanted to study the Transformer and attention structures but found them difficult because of the complex formulas and concepts

AI beginners

Those who have used ChatGPT or generative AI services but want to understand how LLMs actually work
and the principles behind them

AI Engineer

AI engineers who need to understand LLM architecture and GPU environments and build and operate real-world AI systems

💡 What You’ll Learn in the Course

Step 1. Foundation

  • Understanding Transformer Models

  • Tokenizer & Embeddings

  • Encoder vs. Decoder

  • View the model source code

Step 2. Attention

  • Mastering Decoder Models

  • Mastering Attention

  • Masked Attention

  • KV Cache

Step 3. Serving

  • vLLM Serving

  • Paged Attention

  • OpenAI Compatible

  • SSE Protocol

Step 4. Tool Call

  • Understanding Tool Calls

  • Tool Response Architecture

  • Chat Template

  • Tool call parser

Step 5. Optimazation

  • Performance Testing

  • vLLM Monitoring

  • Multi-GPU & Parallelism

  • Additional vLLM Features

Step 6. Advanced

  • Multi Token Prediction

  • mHC

  • Engram

  • Efforts to overcome limitations

💡 Key Points of the Lecture

Point 1

The core principles of attention learned without formulas


Learn various attention techniques intuitively through Excel without using formulas (MHA → MQA → GQA, Sliding Window attention)

Point 2

Implementing an AI Architecture with a 3-Tier Structure


Understand the basic structure of a 3-tier architecture consisting of OpenWebUI, FastAPI, and vLLM, and learn the basic flow of tool integration.

Point 3

Measuring Concurrent Users and Tips for Operating vLLM

Using jMeter, conduct FastAPI → vLLM load testing to examine metrics such as TTFT and TPOT based on the number of concurrent users.

Point 4

Monitoring vLLM Services

Build a Prometheus & Grafana dashboard pipeline to learn the basic principles of operating a vLLM service.

Point 5

Single-GPU / Multi-GPU Testing

Through hands-on practice with the three basic multi-GPU approaches (Pipeline Parallel, Tensor Parallel, and Data Parallel), you will see firsthand why multi-GPU is necessary.

Point 6

Mastering LLM Development Trends

We introduce the latest LLM development trends, including DeepSeek’s MTP, Shared MoE, MLA, and Engram, as well as efforts to improve inference efficiency.

✅ Tools Used in the Course




✅ Server Hands-On Environment Guide

The vLLM system will be built using Runpod. Hands-on exercises using Google Colab’s T4 GPU will also be conducted in parallel. Since the T4 GPU provides 15GB of GPU memory, exercises that can be done in Colab will be conducted there.

Runpod

We will configure a hands-on environment based on the OpenWebUI → FastAPI → Runpod flow. We will deploy vLLM on a GPU server in the Runpod cloud and conduct various hands-on exercises.

Practice costs approximately $10 to $20.


Google Colab

Google Colab, which also serves as the standard environment for AI practice, is used for simple exercises that do not require a Runpod environment. The exercises are conducted on the regular free tier, not Pro, using a T4 GPU.

✅ Local Practice Environment Guide

The vLLM service runs on Runpod, but
OpenWebUI and FastAPI will also run on the local computer you use to take the course.
Therefore, please check that your learning environment meets the requirements below!



Runpod and Colab are used as the primary practice environments, but
You will practice by running OpenWebUI and FastAPI in your local environment.

⚠️ This course will also be updated whenever vLLM is updated.

vLLM is updated very quickly. However, its Major version is still in the 0.x range.
Nevertheless, many companies use vLLM as their inference engine as a de facto standard.
vLLM supports not only the transformer models that currently form the backbone of LLMs, but also the Mamba architecture, which has emerged as an alternative. When new features are added to models, such as Multi Token Prediction, vLLM is updated each time to support them.
This course will also be updated whenever new vLLM features or new model types are released.

Don't miss the latest trends in LLMs.


Recommended for
these people

Who is this course right for?

  • Practitioners who use ChatGPT and generative AI but want to understand how LLMs actually work

  • A beginner who wants to systematically learn LLM serving and system architecture with the goal of becoming an AI engineer

  • Developers who want to understand Transformer and attention architectures from a practical perspective without complex equations

  • Backend and infrastructure engineers who want to understand GPU optimization and the practical process of building AI systems in multi-GPU environments

  • PMs and planners who want to understand LLM architecture and GPU utilization strategies during the planning and development of AI services

Need to know before starting?

  • Understanding of basic Python syntax (variables, functions, conditional statements, etc.)

  • Basic Git Usage

Hello
This is hyunjinkim

Inflearn Verified

2,042

Learners

137

Reviews

251

Answers

4.9

Rating

3

Courses

Hello.

I am a 17-year veteran currently working in the Data & AI field at a large corporation.

Since obtaining my Professional Engineer Information Management certification, I have been creating content to share the knowledge I've gained with as many people as possible.

Nice to meet you. :)

 

Contact: hjkim_sun@naver.com

More

Curriculum

All

53 lectures ∙ (14hr 27min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

28 reviews

5.0

28 reviews

  • kjunekjune0812님의 프로필 이미지
    kjunekjune0812

    Reviews 3

    ∙

    Average Rating 5.0

    Edited

    5

    43% enrolled

    I am not in the deep learning industry, but I work in the field of computer vision (rule-based). Since my company requires LLM and vision-related deep learning technologies, I have been studying these topics. I have only completed about 40% of the course, but I felt compelled to leave a review now. I have taken many deep learning courses, including those by famous and highly-rated instructors, but I haven't found any course as clean and clear as this one. The best part is the quality of the lecture materials. The instructor recorded every single matrix calculation in Excel, which is incredibly helpful when reviewing. The Python code is also well-commented in many places. The quality of the lectures themselves is excellent; the instructor reminds you of parts you might have forgotten, ensuring you don't miss anything. While most other courses show calculations once or twice and then move on, this course goes through the calculations together until the end, which provides great clarity. It seems like the Q&A is monitored frequently, as I received immediate answers to my questions. The lectures seem to have been filmed this year, so it's great that they include a lot of the latest trends. It feels like this course hasn't gained much word-of-mouth yet, but I highly recommend it to anyone who needs to study these topics.

    • hyunjinkim
      Instructor

      Hello Wonjune Lee, Thank you for your thoughtful review! I put a lot of thought into improving the quality of the lecture materials so that students could receive meaningful resources and review them effectively even later on. I also spent a lot of time thinking about how to effectively convey operations like Attention. The conclusion I reached was that it shouldn't be explained through formulas alone, nor through simple metaphors, nor just through torch code. Believing that it can only be understood by following the flow visually, I tried my best to explain it using Excel, and I'm glad to hear that it was conveyed well :) I hope you gain great insights from the remaining parts of the course. Keep it up!

  • mhkim님의 프로필 이미지
    mhkim

    Reviews 4

    ∙

    Average Rating 4.8

    5

    31% enrolled

    I was surprised by how professional and much more detailed the lecture was than I expected. It feels like it's been a long time since I've seen a lecture of this quality. Since my job involves AI-related engineering, I purchased this thinking I would just roughly study the basics of LLM Architecture, but I am extremely satisfied.

    • jusungpark님의 프로필 이미지
      jusungpark

      Reviews 33

      ∙

      Average Rating 4.5

      5

      100% enrolled

      It’s the best. Thank you.

      • hyunukjeong8334님의 프로필 이미지
        hyunukjeong8334

        Reviews 2

        ∙

        Average Rating 4.5

        5

        61% enrolled

        Thank you for explaining it in an easy-to-understand way.

        • nova7tr1173님의 프로필 이미지
          nova7tr1173

          Reviews 11

          ∙

          Average Rating 4.8

          5

          11% enrolled

          It's my first time taking the course, and it's very helpful. ^-^

          • hyunjinkim
            Instructor

            Hello nova7tr, Thank you for your review. I am glad to hear that it was helpful. I hope you enjoy the rest of the course :)

        Similar courses

        Explore other courses in the same field!

        25% off for new members

        $80.00

        25%

        $110.00