강의

멘토링

커뮤니티

BEST
Data Science

/

Data Engineering

Kafka Complete Guide - Core

From the core of Kafka to in-depth content on internal mechanisms, the course is structured so that even beginners can quickly reach the expert level through detailed theoretical explanations, hands-on practice, and practical Kafka application development practice.

(4.9) 95 reviews

1,841 learners

  • dooleyz3525
Kafka
Data Engineering

Reviews from Early Learners

What you will gain after the course

  • Understanding Kafka's core architecture and key components

  • Understanding Kafka's internal core mechanisms

  • Practice operating Kafka with various Kafka commands based on CLI

  • Implement various functions of Producer and Consumer directly using Java

  • Understanding Kafka's internal mechanisms and key environment parameters through hands-on practice

  • Implementing a practical Kafka application using Producer and Consumer

  • Hands-on and understanding of replication and availability architecture in a multi-broker Kafka cluster

  • Practice and understand the startup mechanism of Zookeeper and Kafka and the leader election process of the controller

  • Monitoring Kafka with UI using Confluent Control Center

  • Serialization/Deserialization of custom objects

Distributed streaming platforms, Kafka is the trend!
Learn it thoroughly, starting from the core.

For large-scale data processing
Representative solution, Kafka!

Kafka is the answer to building event streaming applications that handle increasingly massive data processing and complex business requirements.

Kafka, a messaging system, has become an indispensable choice for collecting large amounts of streaming data, building large-scale data pipelines, and building event-driven microservice architectures (MSAs). Consequently, many companies have been rushing to adopt Kafka in recent years, and the demand for Kafka experts is steadily increasing.

Why Kafka?

great
Performance & Availability

forcible
Scalability

formidable
open source ecosystem

distinguished
Mission Stability

Wide
User Community

rich
Online Resources

This course will not only provide a theoretical explanation of Kafka, but will also provide detailed explanations of the internal workings of its core mechanisms, hands-on practice, and implementation of real-world Kafka application development to help you become a Kafka operation and development expert .


Of course the essentials
High level skills are OK.

'Kafka Complete Guide - Core Edition'

Kafka
First time

Kafka skills
Those who want to improve

Theory and Practice
Anyone who wants to take care of everything

This course is designed for those who are learning Kafka for the first time, as well as those who are already applying Kafka to their work but want to build a more solid foundation and improve their skills at a higher level.

  • Core mechanisms of Kafka's main components
  • Hands-on Kafka training using CLI commands
  • Implementing various functions of Producer and Consumer using Java-based Kafka client library
  • Understanding and practicing the high-level internal mechanisms of Kafka's core elements.
  • Implementing a Real-World Kafka Application
  • UI practice for operating and monitoring Kafka using Confluent Control Center
  • Serialization/deserialization of custom objects

It consists of contents such as etc., and you will be able to learn everything from the essential elements required for operating and developing Kafka to the more advanced core mechanisms through this lecture.

Key concepts and practice all in one

The course consists of 30% theory and 70% practice, and the curriculum is filled with various practical classes to help students learn the theory more naturally through practice.

Most of the hands-on classes are conducted through live coding . By following the videos step-by-step, you'll soon find yourself confidently handling Kafka from development to operations.

💡 Please note before taking the class!

  • This course does not cover Kafka connectors or Kafka Streams/KSQ separately. These topics will be covered in a future series of lectures.

Unique features of this course
Check it out.

Intuitive understanding of core mechanisms through illustrations and animations

You can visually understand the core mechanism operation process.

To help you intuitively understand the core mechanisms of Kafka, including brokers, topics, partitions, consumers, replication, and Zookeeper, we have organized the theoretical lectures with numerous illustrations and PPT animations.

Detailed explanation and practice of Kafka's internal deep learning mechanisms and various environment parameters.

In theory, theory and practice are the same. In practice, they are not.
In principle, theory and reality are the same, but in practice, they are quite different. (Albert Einstein)

This lecture goes beyond a conceptual understanding of Kafka's key components and delves into the essential elements of Kafka's internal mechanisms, along with the core environment parameters of Broker, Producer, and Consumer, at a level never before seen in any other lecture.

Additionally, the lectures are structured in a sequence that begins with an explanation of the internal mechanism theory and then moves on to practical training, which will allow you to more firmly master what you have learned in theory.

From basic to practical level, various Java-based practice codes are explained in detail line by line.

Most of the hands-on training will be conducted through live coding. For any potentially challenging code, I'll provide detailed, line-by-line explanations. Beyond basic Kafka producer and consumer client code, you'll learn to implement challenging, real-world Kafka applications alongside me, further enhancing your skills.

Practice is the only way to learn! Learn by repeating the code.
We also provide a PDF of the lecture materials, which are over 200 pages long.

Practice Environment 💾

Server OS

The Kafka server OS is Ubuntu Linux 20.04, running on an Oracle VirtualBox VM. Although it uses Linux, it runs on a virtual machine, making it suitable for both Windows and macOS environments.

VirtualBox can be installed on most Windows and macOS platforms. However, for Macs, VirtualBox is not installed on the latest M1 models. Therefore, you must install Ubuntu using a virtual environment such as UTM. For M1 models, please ensure that Ubuntu can be installed in a virtual environment before selecting a course.

Confluent Kafka
Community Edition

Kafka uses Confluent Kafka Community Edition version 7.1.2, not Apache Kafka.

Confluent, founded by the core team behind Kafka, provides enterprise-grade Kafka with enhanced performance and convenience for enterprise customers. It offers 100% compatibility with Apache Kafka while also offering access to a wider range of Kafka modules and integrated binaries. With Confluent, you can leverage the powerful distributed Kafka system in a more elastic and scalable form. This reduces infrastructure deployment and maintenance burden and accelerates development.

Client Development

Kafka client development is Java- based, using IntelliJ IDEA . Kafka offers client libraries for various languages, but the Java-based client is the most widely supported, offering performance, stability, and the latest features.

Recommended PC Specifications

A full lab environment configuration may require a PC environment with 20-30GB of storage capacity and 4GB or more of RAM .


Check out the Q&A 💬

Q. Why should I learn Kafka?

Kafka has established itself as a leading solution for real-time, high-capacity event streams platforms. Thousands of companies worldwide use Kafka due to its high message processing performance and reliability, diverse feature support, and large ecosystem and community. It is used for a wide range of purposes, from building data pipelines to microservices.

Therefore, many companies are currently seeing a growing demand for Kafka-savvy individuals. This demand will only continue to grow, so mastering Kafka will be a significant advantage for your career.

Q. Can I take the course even if I don't know Kafka?

Yes, this course is structured step-by-step, from basic to advanced, so even beginners can easily follow along. By slowly following the numerous hands-on, live-coding sessions, even beginners will soon become proficient in Kafka.

Q. Can I take the course even if I don't have any Java development experience?

While Kafka knowledge is optional, basic Java programming skills, including an understanding of threads, are required for the practical exercises. The Java code used in the exercises can be found on GitHub .

If you do not have basic Java programming experience , you may have difficulty with the practical exercises. Therefore, please review the practical Java code in advance before taking the course and make sure that it is appropriate for your level.

Q. Is prior knowledge of Linux and IntelliJ required?

Linux and IntelliJ are explained in detail from installation to environment configuration, so you don't need to know them, but it may be more helpful for your practice if you know them.

Q. Will the course also cover Kafka Connect and Kafka Streams/KSQL?

This lecture is the first in a series of Kafka lectures, covering the essential architecture of Kafka. The series consists of three lectures. The second lecture will cover Kafka Connect and Schema Registry, and the third will cover Kafka Streams/KSQL. (The detailed release schedule has not yet been determined.)

Kafka Series!! Great lectures to listen to together 📖

Recommended for
these people

Who is this course right for?

  • For those who want to learn Kafka for the first time

  • Anyone who is using Kafka but wants to gain a more solid foundation of knowledge

  • Anyone who needs a solid understanding of Kafka's Advanced Internal mechanism and key environment parameters

  • For those who want to implement Producer and Consumer functions using Java

  • For those who want to develop Kafka applications closer to real-world situations

  • Data engineers considering adopting Kafka for large-scale data collection and data pipeline construction

  • Solution architects considering implementing a messaging system in their enterprise environment

  • Anyone interested in Kafka's distributed messaging system architecture

Need to know before starting?

  • You don't need to know Kafka, but you need basic Java programming knowledge to understand Threads.

Hello
This is

26,921

Learners

1,367

Reviews

4,009

Answers

4.9

Rating

14

Courses

(전) 엔코아 컨설팅

(전) 한국 오라클

AI 프리랜서 컨설턴트

파이썬 머신러닝 완벽 가이드 저자

Curriculum

All

149 lectures ∙ (22hr 53min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

95 reviews

4.9

95 reviews

  • yebinkang님의 프로필 이미지
    yebinkang

    Reviews 4

    Average Rating 4.8

    5

    99% enrolled

    정말 좋은 강의 입니다. 카프카에 대해서 기본적인 내용을 뛰어 넘어 자세한 내부 메커니즘을 알려주는 자료가 없어서 궁금했는데 감사합니다:)

    • 권 철민
      Instructor

      훌륭한 칭찬 감사합니다 ^^

  • HDJWDKN8265님의 프로필 이미지
    HDJWDKN8265

    Reviews 26

    Average Rating 5.0

    5

    100% enrolled

    논카프카 레가시 개발자지만 카프카가 요새 너무 유행이어서 동료들과의 잡담에 끼기 위하여 수강하였습니다. 원리까지 설명해주셔서 너무 유익했습니다. 감사합니다.

    • 권 철민
      Instructor

      잘 들으셨다니, 저도 기쁩니다. 좋은 수강평 감사드립니다.

  • 황재정님의 프로필 이미지
    황재정

    Reviews 22

    Average Rating 5.0

    5

    100% enrolled

    이직하려는 회사에서 카프카를 사용 할 것 같아 미리 공부겸 수강하였는데 20시간 넘는 강의지만 지루하지 않게 완강 할 수 있었고 카프카에 대한 개념이 잡힌 것 같습니다. 업무에 적용하고 사용해보며 복습하여 활용 할 수 있도록 노력해야할 것 같습니다. 감사합니다.

    • 권 철민
      Instructor

      완강을 축하드립니다. 지루하지 않게 완강하셨다니 저도 기쁩니다. 좋은 수강평 감사합니다.

  • 홍홍홍님의 프로필 이미지
    홍홍홍

    Reviews 5

    Average Rating 5.0

    5

    56% enrolled

    kafka 사용에 대한 겉핥기가 아니라 kafka 원리를 제대로 설명해주면서 알려주는 강의!

    • 권 철민
      Instructor

      핵심을 관통하는 수강평, 감사합니다.

  • bom20065님의 프로필 이미지
    bom20065

    Reviews 2

    Average Rating 5.0

    5

    61% enrolled

    아직 수업은 다 못 들었지만, 카프카에 대해서 자세히 설명해주셔서 너무 좋습니다! 좋은 수업 준비해주셔서 감사드립니다! 다음 강의도 기대가 됩니다 ㅎㅎ

    • 권 철민
      Instructor

      강의가 도움이 되었다니 저도 기쁘군요. 남은 강의도 즐강하십시요 ^^

Limited time deal ends in 15:15:52

$74,250.00

25%

$77.00

dooleyz3525's other courses

Check out other courses by the instructor!

Similar courses

Explore other courses in the same field!