강의

N
챌린지

멘토링

N
클립

로드맵

BEST
Data Science

/

Data Engineering

Kafka Complete Guide - Core

From the core of Kafka to in-depth content on internal mechanisms, the course is structured so that even beginners can quickly reach the expert level through detailed theoretical explanations, hands-on practice, and practical Kafka application development practice.

(4.9) 91 reviews

1,796 learners

  • dooleyz3525
Kafka
Data Engineering

Reviews from Early Learners

What you will learn!

  • Understanding Kafka's core architecture and key components

  • Understanding Kafka's internal core mechanisms

  • Practice operating Kafka with various Kafka commands based on CLI

  • Implement various functions of Producer and Consumer directly using Java

  • Understanding Kafka's internal mechanisms and key environment parameters through hands-on practice

  • Implementing a practical Kafka application using Producer and Consumer

  • Hands-on and understanding of replication and availability architecture in a multi-broker Kafka cluster

  • Practice and understand the startup mechanism of Zookeeper and Kafka and the leader election process of the controller

  • Monitoring Kafka with UI using Confluent Control Center

  • Serialization/Deserialization of custom objects

Distributed streaming platforms, Kafka is the trend!
Learn it thoroughly, starting from the core.

For large-scale data processing
Representative solution, Kafka!

Kafka is the answer to building event streaming applications that handle increasingly massive data processing and complex business requirements.

Kafka, a messaging system, has become an indispensable choice for collecting large amounts of streaming data, building large-scale data pipelines, and building event-driven microservice architectures (MSAs). Consequently, many companies have been rushing to adopt Kafka in recent years, and the demand for Kafka experts is steadily increasing.

Why Kafka?

great
Performance & Availability

forcible
Scalability

formidable
open source ecosystem

distinguished
Mission Stability

Wide
User Community

rich
Online Resources

This course will not only provide a theoretical explanation of Kafka, but will also provide detailed explanations of the internal workings of its core mechanisms, hands-on practice, and implementation of real-world Kafka application development to help you become a Kafka operation and development expert .

Of course the essentials
High level skills are OK.

'Kafka Complete Guide - Core Edition'

Kafka
First time

Kafka skills
Those who want to improve

Theory and Practice
Anyone who wants to take care of everything

This course is designed for those who are learning Kafka for the first time, as well as those who are already applying Kafka to their work but want to build a more solid foundation and improve their skills at a higher level.

  • Core mechanisms of Kafka's main components
  • Hands-on Kafka training using CLI commands
  • Implementing various functions of Producer and Consumer using Java-based Kafka client library
  • Understanding and practicing the high-level internal mechanisms of Kafka's core elements.
  • Implementing a Real-World Kafka Application
  • UI practice for operating and monitoring Kafka using Confluent Control Center
  • Serialization/deserialization of custom objects

It consists of contents such as etc., and you will be able to learn everything from the essential elements required for operating and developing Kafka to the more advanced core mechanisms through this lecture.

Key concepts and practice all in one

The course consists of 30% theory and 70% practice, and the curriculum is filled with various practical classes to help students learn the theory more naturally through practice.

Most of the hands-on classes are conducted through live coding . By following the videos step-by-step, you'll soon find yourself confidently handling Kafka from development to operations.

💡 Please note before taking the class!

  • This course does not cover Kafka connectors or Kafka Streams/KSQ separately. These topics will be covered in a future series of lectures.

Unique features of this course
Check it out.

Intuitive understanding of core mechanisms through illustrations and animations

You can visually understand the core mechanism operation process.

To help you intuitively understand the core mechanisms of Kafka, including brokers, topics, partitions, consumers, replication, and Zookeeper, we have organized the theoretical lectures with numerous illustrations and PPT animations.

Detailed explanation and practice of Kafka's internal deep learning mechanisms and various environment parameters.

In theory, theory and practice are the same. In practice, they are not.
In principle, theory and reality are the same, but in practice, they are quite different. (Albert Einstein)

This lecture goes beyond a conceptual understanding of Kafka's key components and delves into the essential elements of Kafka's internal mechanisms, along with the core environment parameters of Broker, Producer, and Consumer, at a level never before seen in any other lecture.

Additionally, the lectures are structured in a sequence that begins with an explanation of the internal mechanism theory and then moves on to practical training, which will allow you to more firmly master what you have learned in theory.

From basic to practical level, various Java-based practice codes are explained in detail line by line.

Most of the hands-on training will be conducted through live coding. For any potentially challenging code, I'll provide detailed, line-by-line explanations. Beyond basic Kafka producer and consumer client code, you'll learn to implement challenging, real-world Kafka applications alongside me, further enhancing your skills.

Practice is the only way to learn! Learn by repeating the code.
We also provide a PDF of the lecture materials, which are over 200 pages long.

Practice Environment 💾

Server OS

The Kafka server OS is Ubuntu Linux 20.04, running on an Oracle VirtualBox VM. Although it uses Linux, it runs on a virtual machine, making it suitable for both Windows and macOS environments.

VirtualBox can be installed on most Windows and macOS platforms. However, for Macs, VirtualBox is not installed on the latest M1 models. Therefore, you must install Ubuntu using a virtual environment such as UTM. For M1 models, please ensure that Ubuntu can be installed in a virtual environment before selecting a course.

Confluent Kafka
Community Edition

Kafka uses Confluent Kafka Community Edition version 7.1.2, not Apache Kafka.

Confluent, founded by the core team behind Kafka, provides enterprise-grade Kafka with enhanced performance and convenience for enterprise customers. It offers 100% compatibility with Apache Kafka while also offering access to a wider range of Kafka modules and integrated binaries. With Confluent, you can leverage the powerful distributed Kafka system in a more elastic and scalable form. This reduces infrastructure deployment and maintenance burden and accelerates development.

Client Development

Kafka client development is Java- based, using IntelliJ IDEA . Kafka offers client libraries for various languages, but the Java-based client is the most widely supported, offering performance, stability, and the latest features.

Recommended PC Specifications

A full lab environment configuration may require a PC environment with 20-30GB of storage capacity and 4GB or more of RAM .

Check out the Q&A 💬

Q. Why should I learn Kafka?

Kafka has established itself as a leading solution for real-time, high-capacity event streams platforms. Thousands of companies worldwide use Kafka due to its high message processing performance and reliability, diverse feature support, and large ecosystem and community. It is used for a wide range of purposes, from building data pipelines to microservices.

Therefore, many companies are currently seeing a growing demand for Kafka-savvy individuals. This demand will only continue to grow, so mastering Kafka will be a significant advantage for your career.

Q. Can I take the course even if I don't know Kafka?

Yes, this course is structured step-by-step, from basic to advanced, so even beginners can easily follow along. By slowly following the numerous hands-on, live-coding sessions, even beginners will soon become proficient in Kafka.

Q. Can I take the course even if I don't have any Java development experience?

While Kafka knowledge is optional, basic Java programming skills, including an understanding of threads, are required for the practical exercises. The Java code used in the exercises can be found on GitHub .

If you do not have basic Java programming experience , you may have difficulty with the practical exercises. Therefore, please review the practical Java code in advance before taking the course and make sure that it is appropriate for your level.

Q. Is prior knowledge of Linux and IntelliJ required?

Linux and IntelliJ are explained in detail from installation to environment configuration, so you don't need to know them, but it may be more helpful for your practice if you know them.

Q. Will the course also cover Kafka Connect and Kafka Streams/KSQL?

This lecture is the first in a series of Kafka lectures, covering the essential architecture of Kafka. The series consists of three lectures. The second lecture will cover Kafka Connect and Schema Registry, and the third will cover Kafka Streams/KSQL. (The detailed release schedule has not yet been determined.)

Kafka Series!! Great lectures to listen to together 📖

Recommended for
these people

Who is this course right for?

  • For those who want to learn Kafka for the first time

  • Anyone who is using Kafka but wants to gain a more solid foundation of knowledge

  • Anyone who needs a solid understanding of Kafka's Advanced Internal mechanism and key environment parameters

  • For those who want to implement Producer and Consumer functions using Java

  • For those who want to develop Kafka applications closer to real-world situations

  • Data engineers considering adopting Kafka for large-scale data collection and data pipeline construction

  • Solution architects considering implementing a messaging system in their enterprise environment

  • Anyone interested in Kafka's distributed messaging system architecture

Need to know before starting?

  • You don't need to know Kafka, but you need basic Java programming knowledge to understand Threads.

Hello
This is

26,235

Learners

1,306

Reviews

3,973

Answers

4.9

Rating

13

Courses

(전) 엔코아 컨설팅

(전) 한국 오라클

AI 프리랜서 컨설턴트

파이썬 머신러닝 완벽 가이드 저자

Curriculum

All

149 lectures ∙ (22hr 53min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

91 reviews

4.9

91 reviews

  • yebinkang3286님의 프로필 이미지
    yebinkang3286

    Reviews 4

    ∙

    Average Rating 4.8

    5

    99% enrolled

    This is a really good lecture. I was curious because there was no material that explained the detailed internal mechanisms of Kafka beyond the basics, so thank you :)

    • dooleyz3525
      Instructor

      Thank you for the great compliment ^^

  • humbledv님의 프로필 이미지
    humbledv

    Reviews 25

    ∙

    Average Rating 5.0

    5

    100% enrolled

    I am a non-Kafka legacy developer, but Kafka is so popular these days that I took the course to join in on the chats with my colleagues. It was very helpful that you explained the principles. Thank you.

    • dooleyz3525
      Instructor

      I'm glad you enjoyed it. Thank you for your kind review.

  • hjj199110565님의 프로필 이미지
    hjj199110565

    Reviews 22

    ∙

    Average Rating 5.0

    5

    100% enrolled

    I took the course in advance to study for the company I am transferring to because it seems like it will use Kafka. Although the course is over 20 hours long, I was able to finish it without getting bored, and I think I have a grasp of the concept of Kafka. I think I should try to apply it to my work, use it, review it, and utilize it. Thank you.

    • dooleyz3525
      Instructor

      Congratulations on completing the course. I'm glad that you were able to complete it without getting bored. Thank you for your kind review.

  • palmseung님의 프로필 이미지
    palmseung

    Reviews 4

    ∙

    Average Rating 5.0

    5

    56% enrolled

    This is a lecture that properly explains the principles of Kafka, not just a superficial explanation of how to use Kafka!

    • dooleyz3525
      Instructor

      Thank you for the course review, it really gets to the point.

  • bom20065님의 프로필 이미지
    bom20065

    Reviews 2

    ∙

    Average Rating 5.0

    5

    61% enrolled

    I haven't finished the class yet, but I really like how you explained Kafka in detail! Thank you for preparing such a great class! I'm looking forward to the next lecture too ㅎㅎ

    • dooleyz3525
      Instructor

      I'm glad that the lecture was helpful. Enjoy the remaining lectures ^^

Limited time deal ends in 7 days

$58.30

24%

$77.00

dooleyz3525's other courses

Check out other courses by the instructor!

[Revised Edition] The Complete Guide to Python Machine Learning강의 썸네일

[Revised Edition] The Complete Guide to Python Machine Learning

dooleyz3525

We will help you easily understand the core concepts of machine learning and acquire the ability to implement practical machine learning applications by moving away from theory-based machine learning courses.

초급

초급

Python, Machine Learning(ML), Statistics

[Revised Edition] The Complete Guide to Python Machine Learning강의 썸네일

[Revised Edition] The Complete Guide to Python Machine Learning

dooleyz3525

FastAPI Complete Guide강의 썸네일

FastAPI Complete Guide

dooleyz3525

This course is designed to help you learn the core functions of FastAPI, as well as the entire process of web service development. Through this course, we will help you become a FastAPI expert developer needed in the field.

중급이상

중급이상

Python, FastAPI, SQL

FastAPI Complete Guide강의 썸네일

FastAPI Complete Guide

dooleyz3525

SQL data analysis learned through various cases강의 썸네일

SQL data analysis learned through various cases

dooleyz3525

By implementing various practical data analysis cases using SQL, you can simultaneously improve your data analysis and SQL utilization skills.

중급이상

중급이상

SQL, PostgreSQL, DBMS/RDBMS

SQL data analysis learned through various cases강의 썸네일

SQL data analysis learned through various cases

dooleyz3525

Deep Learning CNN Complete Guide - Pytorch Version강의 썸네일

Deep Learning CNN Complete Guide - Pytorch Version

dooleyz3525

From core theories of deep learning and CNN to various CNN model implementation methods, and practical deep learning development know-how through real-world problems, If you want to become a deep learning CNN technology expert based on Pytorch, join us in this lecture :)

초급

초급

Deep Learning(DL), PyTorch, Computer Vision(CV)

Deep Learning CNN Complete Guide - Pytorch Version강의 썸네일

Deep Learning CNN Complete Guide - Pytorch Version

dooleyz3525

Data Analysis SQL Fundamentals강의 썸네일

Data Analysis SQL Fundamentals

dooleyz3525

Through detailed lectures and hands-on practice on the core elements of SQL, we will provide you with a solid foundation to grow into a SQL analysis expert.

초급

초급

SQL, PostgreSQL, DBMS/RDBMS

Data Analysis SQL Fundamentals강의 썸네일

Data Analysis SQL Fundamentals

dooleyz3525

Oracle Performance Analysis and Instance Tuning Core Guide강의 썸네일

Oracle Performance Analysis and Instance Tuning Core Guide

dooleyz3525

It provides a key guide to understanding the internal mechanisms of Oracle DB architecture and growing as a performance tuning and performance analysis expert.

중급이상

중급이상

Oracle, DBMS/RDBMS

Oracle Performance Analysis and Instance Tuning Core Guide강의 썸네일

Oracle Performance Analysis and Instance Tuning Core Guide

dooleyz3525

Kaggle Advanced Machine Learning Practical Crash Course강의 썸네일

Kaggle Advanced Machine Learning Practical Crash Course

dooleyz3525

This course is designed to upgrade your skills as a practical machine learning development expert by implementing the machine learning model of the Home Credit Default Risk competition on Kaggle.

중급이상

중급이상

Machine Learning(ML), Kaggle

Kaggle Advanced Machine Learning Practical Crash Course강의 썸네일

Kaggle Advanced Machine Learning Practical Crash Course

dooleyz3525

[Revised Edition] A Complete Guide to Deep Learning Computer Vision강의 썸네일

[Revised Edition] A Complete Guide to Deep Learning Computer Vision

dooleyz3525

This course will help you become a deep learning-based computer vision expert who is needed in the field by providing a deep theoretical explanation of object detection and segmentation and practical examples that can be used immediately in the field.

중급이상

중급이상

Machine Learning(ML), Tensorflow, Python

[Revised Edition] A Complete Guide to Deep Learning Computer Vision강의 썸네일

[Revised Edition] A Complete Guide to Deep Learning Computer Vision

dooleyz3525

Kafka Complete Guide - Connect강의 썸네일

Kafka Complete Guide - Connect

dooleyz3525

Through in-depth theoretical explanations of Kafka Connect and detailed practical training that can be used immediately in the field, we will help you grow into an expert in building data linkage and data pipelines based on Kafka Connect that are needed in the field.

중급이상

중급이상

Kafka, Data Engineering

Kafka Complete Guide - Connect강의 썸네일

Kafka Complete Guide - Connect

dooleyz3525

A Complete Guide to Deep Learning CNN - TensorFlow Keras Version강의 썸네일

A Complete Guide to Deep Learning CNN - TensorFlow Keras Version

dooleyz3525

From core theories of Deep Learning and CNN to implementation methods of various CNN models, and practical Deep Learning development know-how through real-world problems, If you want to become a Deep Learning CNN technology expert, join us in this lecture :)

초급

초급

Deep Learning(DL), CNN, Tensorflow

A Complete Guide to Deep Learning CNN - TensorFlow Keras Version강의 썸네일

A Complete Guide to Deep Learning CNN - TensorFlow Keras Version

dooleyz3525

Kafka Complete Guide - ksqlDB강의 썸네일

Kafka Complete Guide - ksqlDB

dooleyz3525

This course is designed to help you learn the use of ksqlDB and its core mechanisms through various hands-on exercises. After completing the course, you will be able to easily and quickly build a real-time streaming data analysis system based on Kafka.

중급이상

중급이상

Kafka, ksqlDB, Data Engineering

Kafka Complete Guide - ksqlDB강의 썸네일

Kafka Complete Guide - ksqlDB

dooleyz3525

The Complete Guide to Spark Machine Learning - Part 1강의 썸네일

The Complete Guide to Spark Machine Learning - Part 1

dooleyz3525

If you want to be recognized as a machine learning expert based on large-scale data, from understanding the core framework of Spark machine learning, to SQL-based data processing through difficult practical problems, to data analysis through business domain analysis, and to the ability to implement optimized machine learning models, please join this course.

중급이상

중급이상

Apache Spark, Machine Learning(ML), Big Data

The Complete Guide to Spark Machine Learning - Part 1강의 썸네일

The Complete Guide to Spark Machine Learning - Part 1

dooleyz3525

Similar courses

Explore other courses in the same field!

Kafka Complete Guide - ksqlDB강의 썸네일

Kafka Complete Guide - ksqlDB

dooleyz3525

This course is designed to help you learn the use of ksqlDB and its core mechanisms through various hands-on exercises. After completing the course, you will be able to easily and quickly build a real-time streaming data analysis system based on Kafka.

중급이상

중급이상

Kafka, ksqlDB, Data Engineering

Kafka Complete Guide - ksqlDB강의 썸네일

Kafka Complete Guide - ksqlDB

dooleyz3525

Kafka Complete Guide - Connect강의 썸네일

Kafka Complete Guide - Connect

dooleyz3525

Through in-depth theoretical explanations of Kafka Connect and detailed practical training that can be used immediately in the field, we will help you grow into an expert in building data linkage and data pipelines based on Kafka Connect that are needed in the field.

중급이상

중급이상

Kafka, Data Engineering

Kafka Complete Guide - Connect강의 썸네일

Kafka Complete Guide - Connect

dooleyz3525

Apache Flink with Silicon Valley Engineers강의 썸네일

Apache Flink with Silicon Valley Engineers

altoformula

Real-time data processing is now essential, not optional! Learn to handle real-time streaming smartly with Apache Flink. Batch? Streaming? Even complex concepts can be understood easily and quickly. Hands-on practice configuration where you'll get the hang of it by directly working with Kafka and DB integration! In a world driven by data, start first with Flink.

입문

입문

flink, Big Data, Data Engineering

Apache Flink with Silicon Valley Engineers강의 썸네일

Apache Flink with Silicon Valley Engineers

altoformula

Resignation Time if You Don't Know? The Standard Guide to Data Engineering강의 썸네일

Resignation Time if You Don't Know? The Standard Guide to Data Engineering

altoformula

Uncover true value in the age of data! 📊 Designing applications focused on data has now become essential. Develop the insight and skills your company wants with the latest trends and practical case studies. Start now with efficient data processing and design secrets! Your next step, leap into the data-driven world!

초급

초급

Big Data, Data Engineering, Architecture

Resignation Time if You Don't Know? The Standard Guide to Data Engineering강의 썸네일

Resignation Time if You Don't Know? The Standard Guide to Data Engineering

altoformula

SQL Basics from a Silicon Valley Data Leader강의 썸네일

SQL Basics from a Silicon Valley Data Leader

keeyonghan9539

If you work with data, SQL is a fundamental skill you must know. In this course, we will learn SQL from a data analysis perspective, focusing on hands-on practice. The practice will be conducted in Google Colab using DuckDB.

초급

초급

SQL, Data literacy, Data Engineering

SQL Basics from a Silicon Valley Data Leader강의 썸네일

SQL Basics from a Silicon Valley Data Leader

keeyonghan9539

Building a stateful service for Daeyongrang Chat TPS강의 썸네일

Building a stateful service for Daeyongrang Chat TPS

July

We will tell you everything about how to build a server and perform uninterrupted deployment for stateful services such as Chijijik and AfreecaTV.

초급

초급

Go, Kafka, Node.js

Building a stateful service for Daeyongrang Chat TPS강의 썸네일

Building a stateful service for Daeyongrang Chat TPS

July

Kafka Intro: Hands-on강의 썸네일

Kafka Intro: Hands-on

jscode

I've created a 'Kafka Introduction' course that is easy for non-technical users to understand and immediately applicable in practice!

초급

초급

Kafka, EDA, MSA

Kafka Intro: Hands-on강의 썸네일

Kafka Intro: Hands-on

jscode

Complete Mastery of Azure Data Fundamentals for Data Beginners강의 썸네일

Complete Mastery of Azure Data Fundamentals for Data Beginners

daniellee

This is a special lecture that can lay the theoretical foundation to simultaneously prepare for the Microsoft AZ-900 certification, and as the latest content reflecting the scope of questions as of May 2025, by providing content related to core data concepts, Azure's relational data, Azure's non-relational data, and Azure's analytical workloads in a form combining theory and practice, it can be utilized as a meaningful educational opportunity to not only obtain the certification but also take the first step towards becoming a data professional.

입문

입문

SQL, Big Data, Data Engineering

Complete Mastery of Azure Data Fundamentals for Data Beginners강의 썸네일

Complete Mastery of Azure Data Fundamentals for Data Beginners

daniellee

Redis with Silicon Valley Engineers강의 썸네일

Redis with Silicon Valley Engineers

altoformula

Redis is a fast and scalable open-source database. This technology is used to maximize performance in real-time applications, games, social media platforms, and more. However, to truly understand the power of Redis and use it effectively, you need a deep understanding. Now is your chance to learn Redis in a professional way. "Redis with Silicon Valley Engineers" covers a wide range of topics, from the core concepts of Redis to advanced features. This course is designed for everyone, from beginners who are new to Redis to advanced users.

초급

초급

Redis, database, Big Data

Redis with Silicon Valley Engineers강의 썸네일

Redis with Silicon Valley Engineers

altoformula

Learn Docker and CI environment by following [2023.11 update]강의 썸네일

Learn Docker and CI environment by following [2023.11 update]

John Ahn

Through this course, you can learn about Docker and set up a CI environment.

초급

초급

GitHub, AWS, Docker

Learn Docker and CI environment by following [2023.11 update]강의 썸네일

Learn Docker and CI environment by following [2023.11 update]

John Ahn

Understand and gain insight into backend multithreading강의 썸네일

Understand and gain insight into backend multithreading

captainmomo

Understand multithreaded programming and see how this knowledge can help you analyze other projects.

초급

초급

multithread, MySQL, Kafka

Understand and gain insight into backend multithreading강의 썸네일

Understand and gain insight into backend multithreading

captainmomo

Data Analysis SQL Fundamentals강의 썸네일

Data Analysis SQL Fundamentals

dooleyz3525

Through detailed lectures and hands-on practice on the core elements of SQL, we will provide you with a solid foundation to grow into a SQL analysis expert.

초급

초급

SQL, PostgreSQL, DBMS/RDBMS

Data Analysis SQL Fundamentals강의 썸네일

Data Analysis SQL Fundamentals

dooleyz3525

[Renewal] First-time MongoDB and NoSQL (Big Data) Database Bootcamp [From Beginner to Application] (Updated)강의 썸네일

[Renewal] First-time MongoDB and NoSQL (Big Data) Database Bootcamp [From Beginner to Application] (Updated)

funcoding

Learn NoSQL technology, which can handle Big Data, one of the fundamental technologies in full-stack and data science utilized by modern startups. MongoDB is the easiest and quickest technology to utilize among NoSQL databases. In this lecture, you will learn the basics of MongoDB in a short time, and also acquire the skills to manipulate and utilize MongoDB with Python.

초급

초급

Python, MongoDB, DBMS/RDBMS

[Renewal] First-time MongoDB and NoSQL (Big Data) Database Bootcamp [From Beginner to Application] (Updated)강의 썸네일

[Renewal] First-time MongoDB and NoSQL (Big Data) Database Bootcamp [From Beginner to Application] (Updated)

funcoding

IntelliJ IDEA Conf 2025강의 썸네일

IntelliJ IDEA Conf 2025

JetBrains

IntelliJ IDEA Conf 2025는 JetBrains에서 매년 주최하는 온라인 컨퍼런스로, Java와 Kotlin 개발 개발자를 위한 유익한 세션을 제공합니다. AI 도입, Spring Boot, Gradle, Kafka, 데이터베이스 관리 등 다양한 주제를 다루며, 클라우드 통합, 보안 코딩, 개발자 성능 향상 및 고급 개발 기법에 대한 인사이트도 공유됩니다. 이 컨퍼런스는 전문 개발자는 물론 초보 개발자도 참여가능하며, 더 빠르게 고품질의 코드를 작성할 수 있는 생산성 팁과 함께 최신 기술을 접할 수 있는 기회를 제공합니다.

초급

초급

Java, Spring, Kotlin

IntelliJ IDEA Conf 2025강의 썸네일

IntelliJ IDEA Conf 2025

JetBrains