Realtime Datalake Using Kafka & Spark
hyunjinkim
Beginner's Kafka & Spark Real-time Pipeline Course. All-in-one: Master concepts to architecture.
Basic
Kafka, Apache Spark, pyspark
Understand Transformer-based LLM architectures and GPU utilization strategies, and gain hands-on experience with the actual serving process using vLLM. This course covers the entire practical workflow, from building AI system pipelines to monitoring and multi-GPU utilization, and is designed for intuitive understanding through diagrams and practice without complex formulas.
214 learners
Level Basic
Course period Unlimited
Reviews from Early Learners
5.0
WonJune Lee
I am not in the deep learning industry, but I work in the field of computer vision (rule-based). Since my company requires LLM and vision-related deep learning technologies, I have been studying these topics. I have only completed about 40% of the course, but I felt compelled to leave a review now. I have taken many deep learning courses, including those by famous and highly-rated instructors, but I haven't found any course as clean and clear as this one. The best part is the quality of the lecture materials. The instructor recorded every single matrix calculation in Excel, which is incredibly helpful when reviewing. The Python code is also well-commented in many places. The quality of the lectures themselves is excellent; the instructor reminds you of parts you might have forgotten, ensuring you don't miss anything. While most other courses show calculations once or twice and then move on, this course goes through the calculations together until the end, which provides great clarity. It seems like the Q&A is monitored frequently, as I received immediate answers to my questions. The lectures seem to have been filmed this year, so it's great that they include a lot of the latest trends. It feels like this course hasn't gained much word-of-mouth yet, but I highly recommend it to anyone who needs to study these topics.
5.0
jjanec
It's challenging, but I'll do my best to follow along. I like that the content contains only the essential core information.
5.0
김민서
It's very helpful.
Understanding the encoder-decoder structure and core operating principles of the Transformer model
Understanding the evolution of the latest attention mechanisms, including MHA, MQA, GQA, and MLA
Hands-on practice on how to utilize the vLLM engine, the de facto standard for current AI serving
Monitoring key performance metrics such as TTFT and TPOT in a vLLM serving environment
Design and implementation of multi-GPU architecture utilizing Tensor/Pipeline/Data Parallelism
Understanding the Core Concepts of Agent AI and the Principles of Tool Calling
Experience in building AI system pipelines and monitoring performance from a real-world industry perspective
Understanding the latest LLM trends such as MLA, MTP, and n-grams based on the latest research papers
In the era of autonomous AI Agents,
you can utilize various Agent tools and Public APIs such as OpenAI, Claude, and Codex.
However, in a real service environment, you must also consider
data security, network costs, token costs, and GPU resource management.
Therefore, what is important is
an understanding of the Hybrid AI architecture, which combines Public APIs and self-hosted GPU-based LLMs
according to the situation.
sao cho phù hợp với từng tình huống.
In that case, is using only Public APIs always the best option?
Not necessarily.
These days, many LLMs comparable to public APIs (ChatGPT, Claude, Sonnet, etc.) are being developed both domestically and internationally.
However, knowing and using LLMs well is not easy.
Purchasing expensive GPUs and using LLMs with an understanding of them versus using them without understanding
leads to
a significant difference.
In the era of agents, we have moved from the age of training to the age of inference. While using public APIs effectively is necessary, many companies prefer building local serving environments for various reasons such as security, governance, and cost. Learn everything from understanding LLM architecture for building local serving environments to architectural configuration and the latest trends in LLM development.
You need to know how to use the numerous LLMs released on Hugging Face.
However, the config.json file, which provides the specifications of an LLM model, is no different from a secret code to beginners. This is because you need to understand the Transformer model to be able to read it.
But don't worry. After taking this course, you will become an expert who can look at and understand the key specifications.
(This is the content for Chapter 3-5. Make sure to take away all the remaining key parameters.)
Attention is the beginning and end of the Transformer model, which serves as the foundation for current LLM models.
The attention-model emerged in 2017, but
it has remained the most powerful algorithm for nearly a decade.
While many efforts are being made to move beyond the Transformer structure,
no architecture has yet emerged that completely replaces the Transformer's attention.
⚠️ You must never have just a vague understanding of attention.
(This is the content for Chapter 5-4. The evolution of Attention is synonymous with the evolution of LLMs.)
Multi-GPU configuration is essential for running large-scale LLMs and achieving fast inference.
However, did you know that there are several different ways to configure multi-GPU setups?
Those who wanted to study Transformer and Attention structures,
but found them difficult due to complex formulas and concepts
Those who have used ChatGPT or generative AI services,
but want to understand the principles of how LLMs actually work
AI engineers who need to understand LLM architecture and GPU environments,
and possess the capability to build and operate actual AI systems
Understanding the Transformer Model
Tokenizer & Embedding
Encoder vs Decoder
View model source code
Mastering the Decoder Model
Mastering Attention
Masked Attention
KV Cache
vLLM Serving
Paged Attention
OpenAI Compatible
SSE Protocol
Understanding Tool Calls
Tool Response Architecture
Chat Template
Tool call parser
Performance Testing
vLLM Monitoring
Multi-GPU & Parallelism
vLLM Additional Features
Multi Token Prediction
mHC
Engram
Efforts to overcome limitations
Core Principles of Attention Learned Without Formulas
Learn various attention techniques intuitively through Excel without complex formulas (MHA → MQA → GQA, Sliding Window Attention)
Implementation of a 3-Tier AI Architecture
Understand the basic structure of a 3-tier architecture connecting OpenWebUI, FastAPI, and vLLM, and learn the fundamental flow of tool integration.
Measuring Concurrent Users and Tips for vLLM Operation
Using jMeter, perform a load test from FastAPI to vLLM to check metrics such as TTFT and TPOT according to the number of concurrent users.
Monitoring vLLM Services
Learn the basic principles of vLLM service operation by building a Prometheus & Grafana dashboard pipeline.
Single GPU / Multi-GPU Testing
Through hands-on practice with the three basic multi-GPU methods (Pipeline Parallel, Tensor Parallel, and Data Parallel), you will see firsthand why multi-GPU setups are necessary.
Mastering LLM Development Trends
We introduce the latest techniques from DeepSeek, such as MTP, Shared MoE, MLA, and Engram, as well as current LLM development trends focused on inference efficiency.
The vLLM system construction will be carried out using Runpod. In addition, hands-on sessions using Google Colab's T4 GPU will be conducted in parallel. Since the T4 GPU provides 15GB of GPU memory, exercises that are feasible on Colab will be conducted there.
We will configure a practice environment based on the OpenWebUI → FastAPI → Runpod flow. We will conduct various exercises by deploying vLLM on GPU servers in the Runpod cloud.
Hands-on practice will incur a cost of approximately $10 to $20.
Google Colab, which is like the standard environment for AI practice, is used for simple exercises that do not require a Runpod environment. We will use the standard free tier, not Pro, and utilize the T4 GPU.
The vLLM service will be hosted on Runpod, but
OpenWebUI and FastAPI will also be running on your local computer.
Therefore, please check if your environment meets the requirements below!
vLLM's update speed is very fast. However, the major version is still in the 0.x range.
Nevertheless, many companies are using vLLM as their inference engine as a de facto standard.
vLLM supports not only the Transformer models that currently form the backbone of LLMs but also alternative architectures like Mamba , and it is updated every time new features are added to models, such as Multi Token Prediction, to support them.
This course will also be updated as new vLLM features or new model types are released.
Who is this course right for?
Practitioners who use ChatGPT and generative AI but want to understand how LLMs actually work
Beginners who aim to become AI engineers and want to systematically learn LLM serving and system architecture.
Developers who want to understand Transformer and Attention structures from a practical perspective without complex formulas.
Backend and infrastructure engineers who want to understand GPU optimization and the actual workflow of building AI systems in multi-GPU environments
PMs and planners who want to understand LLM architecture and GPU utilization strategies during the AI service planning and development process.
Need to know before starting?
Understanding of basic Python syntax (variables, functions, conditional statements, etc.)
Basic usage of git
Inflearn Verified
1,672
Learners
117
Reviews
246
Answers
4.9
Rating
3
Courses
Hello.
I am a 17-year veteran currently working in the Data & AI field at a large corporation.
Since obtaining my Professional Engineer Information Management certification, I have been creating content to share the knowledge I've gained with as many people as possible.
Nice to meet you. :)
Contact: hjkim_sun@naver.com
All
54 lectures ∙ (14hr 27min)
Course Materials:
All
16 reviews
5.0
16 reviews
Reviews 1
∙
Average Rating 5.0
5
It's very helpful.
Thank you for the review, Minseo Kim. I hope the lecture was very helpful ^-^
Reviews 3
∙
Average Rating 5.0
Edited
5
I am not in the deep learning industry, but I work in the field of computer vision (rule-based). Since my company requires LLM and vision-related deep learning technologies, I have been studying these topics. I have only completed about 40% of the course, but I felt compelled to leave a review now. I have taken many deep learning courses, including those by famous and highly-rated instructors, but I haven't found any course as clean and clear as this one. The best part is the quality of the lecture materials. The instructor recorded every single matrix calculation in Excel, which is incredibly helpful when reviewing. The Python code is also well-commented in many places. The quality of the lectures themselves is excellent; the instructor reminds you of parts you might have forgotten, ensuring you don't miss anything. While most other courses show calculations once or twice and then move on, this course goes through the calculations together until the end, which provides great clarity. It seems like the Q&A is monitored frequently, as I received immediate answers to my questions. The lectures seem to have been filmed this year, so it's great that they include a lot of the latest trends. It feels like this course hasn't gained much word-of-mouth yet, but I highly recommend it to anyone who needs to study these topics.
Hello Wonjune Lee, Thank you for your thoughtful review! I put a lot of thought into improving the quality of the lecture materials so that students could receive meaningful resources and review them effectively even later on. I also spent a lot of time thinking about how to effectively convey operations like Attention. The conclusion I reached was that it shouldn't be explained through formulas alone, nor through simple metaphors, nor just through torch code. Believing that it can only be understood by following the flow visually, I tried my best to explain it using Excel, and I'm glad to hear that it was conveyed well :) I hope you gain great insights from the remaining parts of the course. Keep it up!
Reviews 11
∙
Average Rating 5.0
5
I've completed the course! Thank you so much for providing such high-quality education!! Except for the notes I left in the Q&A, I had no issues performing all the exercises on a Windows-based system!
Hello logt! You left this review after completing 100% of the course! I will make sure to update the part regarding container issues on Windows OS that you mentioned in the Q&A. Thank you.
Reviews 1,010
∙
Average Rating 4.9
5
Thank you for the great lecture!
Hello Jang jaehoon, Thank you for the course review 👍 I see you've completed 7%. I hope you enjoy the rest of the lessons and find them very helpful. You can do it!
Reviews 4
∙
Average Rating 5.0
Check out other courses by the instructor!
Explore other courses in the same field!