AI has written all the code for you, but
you’ve probably been frustrated wondering how it actually works, right?
In the AI era, we need people who can give AI precise instructions, control it, and orchestrate its work, rather than being led by AI. Such people will not be replaced by AI.
Whether you’re coding with AI or preparing for a job or career change, I hope you’ll build those skills through this course.
It covers the principles of GPUs made as easy as possible to understand in this world, along with the necessary computer science knowledge. We teach you what you couldn't learn at university or even on the job.
We help you get AI to do its job better.
In this course, you’ll learn how the GPU actually executes code. We explain from the fundamentals how threads are grouped and run, and why the execution time can vary by dozens of times depending on where values are placed. To ensure these principles don’t remain merely theoretical, we use vector search—finding the five most similar documents among one million—and show on screen what changes and how when running it on a CPU, on a GPU, and after optimization. Along the way, we also teach CS concepts such as caches, threads, synchronization, and floating-point numbers. We’ll teach you what you couldn’t learn at university or in the industry.
AI writes the code. Your job is to understand how that code works, enabling you to develop higher-performance features and services in less time than others.
MEASURED ON RTX 4090
| CPU similarity calculation | 558.83 ms |
| First GPU Kernel | 11.736 ms 47.6x |
| GPU-optimized kernel | 3.237 ms Approximately 173× |
Recommended for people like these
College Students Developing with AI
People who know how to build things with AI but don’t know how the computer executes that code
Whether it’s for a class, a club, or a project you built on your own, AI writes the code for you. But in interviews or presentations, what people ask about isn’t the end product—it’s your understanding. "Why did you implement it this way?" "What’s the difference between a CPU and a GPU?" "Why does the way memory is accessed affect performance?" If you’ve ever found yourself at a loss for words when faced with these questions, what you’re lacking right now isn’t the number of projects you’ve done, but the ability to explain the system you built.
Junior developer
A developer who can quickly build features with AI but doesn’t know how to improve performance
Features are delivered quickly when you use AI. The problem comes next. When the code is slow, memory usage grows, or traffic surges, you need to improve it. AI offers multiple solutions, but ultimately, it is up to you to choose the right one. Without criteria for making those decisions, you may gain experience, but you will not build much problem-solving skill.
AI Engineer
Those who need to give AI precise prompts to build better-performing solutions faster than their competitors
If you only tell AI to "make it faster," you end up spending time changing this and that without knowing why the returned code is the way it is. If you understand the principles, you can give precise instructions from the start. For example, you can say, "Check whether this kernel’s memory accesses are coalesced," or "First determine whether the current bottleneck is computation or memory." Then, instead of spending days on trial and error, you receive code that delivers the desired performance in one go. This is where the difference in getting your product out faster than competitors is decided.
After completing the course,
- You can ask AI to do more advanced tasks. "Make it faster with the GPU" is no longer where you stop—you can ask, "Check whether Memory Coalescing is working properly in this kernel," or "Analyze whether registers or shared memory are causing the low Occupancy," and verify the answers you receive.
- You will build a solid foundation in computer science, so you will be able to fully understand any course you take in the future. Arrays and contiguous memory, the heap, sorting, time complexity, caches and the memory hierarchy, latency hiding, threads and scheduling, synchronization and race conditions, floating-point errors, and more will become easy to understand.
- You can answer interview and presentation questions like "Why did you implement it this way?" with solid reasoning. On the first page, you record 20 questions about the underlying principles that you cannot answer, and on the last page, you answer all twenty. It remains in a form that can be used directly as interview answers.
- You can say exactly which part became faster and by how much. You will learn how to measure kernel time precisely using the GPU clock, calculate achieved bandwidth and compare it with the theoretical upper bound, and determine what to use as an alternative when the profiler is unavailable.
- You will know where to look when performance falls short. You will have a basis for determining whether the bottleneck is computation or memory.
Features of This Course
Key Strengths (1)
There is no course that makes the principles this easy to understand.
There are many resources explaining GPUs. However, most explain terminology using terminology. After reading the sentence, "A warp is a group of 32 threads and executes using SIMT," you may know one more term, but you still cannot visualize it.
This course does not start with definitions. It asks first: What can’t you determine if you don’t know this? If you don’t understand warps, you can’t decide why the block size should be set to a multiple of 32, and if you don’t understand the memory hierarchy, you won’t know where to start looking when faced with slow code. We first create that sense of uncertainty, then provide precise definitions, and finally show how these concepts actually play out in code that searches one million documents.
We define each term when it first appears and include the original English term alongside it. We do the same for concepts that are easy to gloss over with a “you probably know this” assumption, such as arrays, pointers, time complexity, and caches. For those who already know them, it may be just a passing sentence, but for those who don’t, that one sentence is the bridge that enables them to follow the course all the way to the end.
Key Strengths (2)
We provide the practice code as-is to show that it works according to the actual principles.
It doesn’t end with explanations. We provide the complete source code and Makefile for 15 hands-on programs: cpu_search, hello_kernel, index_check, device_info, warp_waste, divergence, spill, gpu_search, timing, sync_experiment, reduction, coalesced, stride, bandwidth, and occupancy. Each program was created to measure one specific principle.
In the course, we show you the screen as we build and run these programs exactly as they are. We don’t skip a single command or line of output. This means that even if you only watch the course without setting up a practice environment, you can see with your own eyes that “it works according to the principles,” and if you want to run it yourself, you can use the exact same code.
You’ll learn the following.
All 47 lectures form a single cohesive flow. Starting with why CPUs and GPUs are designed differently, we will examine the principles one by one in order, including how threads are grouped and executed and why where a value is placed determines the time it takes. We will see how these principles actually work in a program that searches 1 million documents.
Therefore, we recommend taking the lessons in order. This is because the numbers examined in the previous lesson become the criteria for making performance improvement decisions in the following lesson. GPU principles and CS knowledge are not taught separately, either. We explain them right there when they become necessary.
SECTION 1 · Lessons 1–5
Why Learn GPU Principles and CS Knowledge Now?
We begin by addressing why you need to understand the fundamentals in an era when AI writes code for you. First, we’ll check what you’ll gain from this course, then prepare the practice environment. You can skip setting up the practice environment without affecting your ability to follow the lectures.
- 1Why GPU principles and computer science knowledge are essential in the AI era (1)
- 2Why GPU principles and computer science knowledge are needed in the AI era (2)
- 3Why GPU principles and computer science knowledge are needed in the AI era (3)
- 4What you will learn in this course
- 5Practice environment — You may skip this.
SECTION 2 · Lessons 6–11
How Are CPUs and GPUs Different?
“GPUs are fast because they have many cores” cannot explain cases where a GPU is actually slower. We’ll look at where the two devices spent the same transistor budget. We’ll separately cover the questions that actually come up in interviews for CPUs and GPUs.
- 6Differences in CPU and GPU Design (1)
- 7Differences in CPU and GPU Design (2) — Easy Enough for an Elementary School Student to Understand
- 8CPU vs. GPU Design Differences (3) — CPU Interview Questions
- 9Differences in CPU and GPU Design (4) — GPU Interview Questions
- 10Differences in CPU and GPU Design (5) — 3 Calculation Methods
- 11Differences in CPU and GPU Design (6)
SECTION 3 · Lessons 12–21
Build a foundation in computer science and establish a baseline with the CPU
We cover why vectors are used, how much GPU memory there is, and the relationship between threads and memory. Then we verify dot products, norms, and cosine similarity by hand, move on through sorting and heap data structures, and build it with the CPU first to establish a baseline time. This value becomes the denominator for all subsequent speedup ratios.
- 12 CS Knowledge — Why Use Vectors?
- 13CS Knowledge — GPU Memory Size
- 14CS Knowledge — GPU Threads and Memory
- 15Measuring on a GPU server
- 16CS Knowledge — Dot Product, Norm, and Cosine Similarity (1)
- 17CS Knowledge — Dot Product, Norm, and Cosine Similarity (2)
- 18Why vector search is slow on the CPU
- 19CS Knowledge — Full Sorting Algorithm
- 20CS Knowledge — Partial Sorting Algorithms and Heap Data Structures
- 21Measuring with the CPU
SECTION 4 · Lessons 22–32
How Does the GPU Execute Code?
We distinguish between what GPUs do well and what they do poorly, then examine how CPUs and GPUs work together. After defining what a kernel is and measuring it in practice, we delve into how GPUs work internally over five lessons, starting with threads.
- 22What GPUs do well
- 23What GPUs are not good at doing
- 24Computer Science Knowledge — How the CPU and GPU Work Together
- What Is a GPU Kernel? 25
- 26Measuring Kernel Performance in Practice
- 27GPU Threads
- 28GPU Principles (1)
- 29GPU Principles (2)
- 30GPU Principles (3)
- 31GPU Principles (4)
- 32GPU Principles (5)
SECTION 5 · Lessons 33–42
You need to understand memory to see performance.
Where a value resides determines the time it takes. Over three lessons, we examine the memory hierarchy and cover data movement between the CPU and GPU. This is where you’ll see why speeding up only the kernel isn’t enough. We then move on to accurately measuring time, memory optimization, and synchronization, race conditions, and floating-point arithmetic.
- 33GPU Memory Hierarchy (1)
- 34GPU Memory Hierarchy (2)
- 35GPU Memory Hierarchy (3)
- 36Principles of Data Transfer Between the CPU and GPU (1)
- 37CPU-GPU Data Transfer Principles (2)
- 38Optimizing CPU-GPU Performance — Keeping Data in Memory
- 39Performance Measurement — Timing the Execution
- 40 Optimizing GPU memory
- 41CS Knowledge — Synchronization and Race Conditions
- 42CS Knowledge — Reduction and Floating-Point Numbers
SECTION 6 · Lessons 43–47
Become Someone Who Measures and Makes Judgments
We cover the remaining GPU principles and learn how to assess performance using utilization metrics and profiling tools. We wrap up by also organizing what evidence you can present when applying for jobs, changing jobs, and working on real-world projects.
- 43GPU Principles (6)
- 44GPU Principles (7)
- 45Performance Measurement Method — Utilization
- 46Performance measurement tools to use in projects when getting a job or changing jobsパフォーマンス測定ツール
- 47Let's avoid being replaced by AI and earn higher salaries.
You learn CS knowledge because it is necessary.
Vectors, inner products and norms, sorting algorithms, heap data structures, memory hierarchy, synchronization and race conditions, reduction, and floating-point errors. These are the lectures marked “CS Knowledge” in the table of contents. They come up when you need to explain why something works the way it does out of necessity. That’s why they stick in your memory so clearly.
The person who created this course
TECHLEAD CRO
Techlead CRO
I am an AI and cloud expert with 20 years of experience, having started my career through SK Group’s entry-level recruitment.
As an NTIS-registered national R&D researcher, I carried out six AI-related R&D projects and filed seven patents, three of which were registered in the fields of computer vision, search engines, and e-commerce recommendation systems. I have written papers on the readability of AI-generated code, AI-mediated service ecosystems, and information accessibility in the AI era. In 2019, I received an award from the Head of the Incheon Regional Office of Small and Medium Enterprises and Startups.
I teach courses on Agentic AI, MLOps, machine learning and deep learning, and cloud security at organizations including KT Cloud, Hyundai Mobis, Hyundai AutoEver, Isu Group, Korea University of Technology and Education, and the Health Insurance Review & Assessment Service.
Why I Created This Course
In the field, when providing AI-related consulting for large enterprises, I often see similar situations. People are familiar with having AI write code, but they often get stuck at the stage of determining whether the code was written correctly. When AI says, "Performance has improved," it is difficult to determine why it improved, and conversely, when the code slows down, they feel at a loss as to where to start looking for the problem.
To make this judgment, you ultimately need to understand how GPUs work and the basic computer science principles underlying them. However, what you learned at university often remains knowledge for passing exams, and even in the workplace, there are few opportunities to relearn these concepts in connection with practical work.
In fact, the material covered in this course is based on what I have explained whenever needed in large-enterprise AI consulting and corporate training settings over the years. It also includes a great deal of content that is not typically covered in introductory courses and is difficult to encounter unless you participate in actual projects or consulting.
This time, rather than limiting that content to specific companies or projects, I reorganized it into a single course that anyone can follow and learn from the very beginning. This is not a course that only people specializing in GPUs can understand; it connects everything from basic concepts to real-world code and performance measurements so that any developer who wants to understand and evaluate AI-generated code firsthand can follow along.
So I created all 23 chapters of the textbook, 24 video lessons, and 15 hands-on programs myself, and ran every example directly on a GPU server to verify it. The performance figures shown in the lectures are also the results of actually running and measuring the code.
While creating this course, I established three principles.
1. Explain concepts starting with the most basic definitions.
2. Include directly measured results when explaining performance.
3. Do not hide experiments that produced results different from what was expected; show them as they are.
Do you have any questions?
Q. I’m not a computer science major and have never taken a CS class. Will I be able to follow along?
This course was designed with people who have never studied computer science as the default audience. Arrays, contiguous memory, pointers, time complexity, and caches are defined from the very first time they appear. Rather than simply giving definitions and moving on, we also provide the formal names, such as "this is commonly called the heap," so you can search for them on your own and explore them further later.
Q. In an age when AI even writes code for us, is there still any point in learning how GPUs work?
Exactly the opposite.
We are not learning the principles because AI cannot write code. We need to learn them because AI has started writing code too well, so people must now be able to understand that code and assign it higher-level tasks.
Without understanding the principles, your request ends with “Make it faster using the GPU.” Once you understand the principles, you can ask, “Check whether this kernel’s Memory Coalescing is working properly,” “Analyze whether the cause of the low Occupancy is registers or shared memory,” or “First determine whether this kernel is memory-bound or compute-bound,” and verify whether the answer you get is correct. The stronger AI becomes, the more those who understand the principles will survive.
Q. Do I have to practice on a GPU computer? Can I understand it by just watching the lectures without doing the exercises?
Yes, even if you do not follow along with the exercises, the lectures show everything that was actually done in the exercises exactly as it was, so you can properly learn the principles of GPUs and computer science without setting up an environment for hands-on practice yourself. The lectures are designed to be understandable even if you simply listen.
Q. My computer does not have an NVIDIA GPU. Should I rent and use a GPU server as described in the course?
This course was created on the premise that “my laptop doesn’t have a GPU.” Therefore, you will rent a server with an NVIDIA GPU for the hands-on exercises.
In the course, we use RunPod to rent a GPU server by the hour and connect via SSH so you can follow along easily. We walk through each step on screen, including creating an account, creating a server, creating an SSH key, connecting, uploading files, and restoring the environment. Guides for both macOS and Windows are included.
Things to Note Before Taking the Course
Practice Environment
- Your computer — Both macOS and Windows are supported. You do not need an NVIDIA GPU. Simply connect via SSH from the terminal, and instructions are provided for both operating systems.
- GPU server — Rent one by the hour from RunPod. The course is based on one RTX 4090 (24 GB of VRAM) and approximately 60 GB of disk space. You can use a different GPU; the principles are the same, only the numbers will differ, and the course also covers how to interpret those differences.
- Software — We use a server image that includes the CUDA Toolkit as is. The course is based on CUDA 12.8 and the 570 series drivers. Since the server image handles the installation, there is no separate installation process.
- Cost — Server rental costs are separate from the course fee and are incurred only for the time the server is running. At the time the course was created, an RTX 4090 cost $0.74 per hour. We also cover creating the server when practicing, deleting it when finished, and scripts for restoring the environment.
Learning Materials
- 23 web textbook chapter links provided — They follow the same order as the videos and include more detailed definitions, execution procedures, interpretations, review questions, and summaries.
- 15 hands-on programs (project.zip) — cpu_search, hello_kernel, index_check, device_info, warp_waste, divergence, spill, gpu_search, timing, sync_experiment, reduction, coalesced, stride, bandwidth, occupancy. We provide the complete source code and Makefile.
- Data generation script — Embeds 1 million paragraphs to create a 768-dimensional float32 vector file (3.07 GB).
- Environment restoration script — even if you delete and recreate the server, you can return to the same environment in just a few minutes.
Prerequisite Knowledge and Notes
- Anyone curious about how GPUs actually work is welcome, regardless of major, C/C++ experience, CUDA experience, or whether you have taken data structures and algorithms courses. We will explain the principles of GPUs and the CS knowledge needed for them step by step.
- We recommend taking the lessons in order. Since the course is structured around continuously improving a single program, the measurements from earlier chapters become the baselines for later chapters.
- What this course does not cover — memorizing CUDA syntax, how to use PyTorch and vLLM, or library tuning options. This course covers the principles underlying those topics.
- The copyright to the course materials belongs to the knowledge provider. You may freely modify and use the practice code for learning purposes, but redistribution and sharing of the course materials are prohibited.
To become a developer who can understand, instruct, and verify code created by AI.
That is the purpose of this course.
![[AI Real-world Method] Creating a Service Planning Figma UX Flow in just 10 minutesCourse Thumbnail](https://cdn.inflearn.com/public/files/courses/340323/cover/01kev8vd8z1tkyrehkk9k5s1x8?w=420)
![[Suminjeongeum] Basic Mathematics for AI - An Introduction to Mathematics Starting from ConceptsCourse Thumbnail](https://cdn.inflearn.com/public/files/courses/342179/cover/01krw77g9yv13wy852v9na3fq1?w=420)
![[OT Security - ③ Chemical Industry] Understanding Manufacturing Security in the AI Era: Focusing on Field Case StudiesCourse Thumbnail](https://cdn.inflearn.com/public/files/courses/342536/cover/ai/0/8b97b1f6-6b48-4beb-922e-67b6b0624fb2.png?w=420)
![[Deep-Up_PASS] AWS Certified AI Practitioner Exam Question ExplanationsCourse Thumbnail](https://cdn.inflearn.com/public/files/courses/337055/cover/01jxh5hzmh89eztnpmr0hbssfz?w=420)
![[Beginner] Learn how to create 3D animated videos with no-code using CreateStudio3, even if it's your first time [Super Easy]Course Thumbnail](https://cdn.inflearn.com/public/files/courses/340594/cover/01kfwdvk88zkg41ewrb9tg73xa?w=420)
![[OT (Manufacturing) Security] Master Class for the 4 Major Manufacturing IndustriesCourse Thumbnail](https://cdn.inflearn.com/public/files/courses/342823/cover/ai/3/d8a24ae4-a719-4167-b6c9-88c40dbbf133.png?w=420)
![[Hi,AI!] Perfect Your Resume & Cover Letter with AI All-Purpose Prompts for Job SeekersCourse Thumbnail](https://cdn.inflearn.com/public/files/courses/339883/cover/01kbpceh0bvaj9437v9mafswyf?w=420)
