강의

멘토링

로드맵

BEST
Data Science

/

Data Engineering

Kafka Complete Guide - ksqlDB

This course is designed to help you learn the use of ksqlDB and its core mechanisms through various hands-on exercises. After completing the course, you will be able to easily and quickly build a real-time streaming data analysis system based on Kafka.

(4.8) 17 reviews

353 learners

  • dooleyz3525
카프카
Kafka
ksqldb
데이터 엔지니어링
streaming
sql
ksqlDB
Data Engineering

Reviews from Early Learners

What you will learn!

  • From KSQLDB's basic concepts to advanced architecture

  • Difference between Stream and Table, and Stateful Streaming Processing Mechanism

  • Information on the creation and management of KSQLDB's main objects, understanding of various data types

  • RocksDB operation mechanism in KSQLDB

  • Understanding various query syntax and functions of KSQLDB

  • Understanding and using Group by and Mview, and its specificity and limitations

  • Understanding and utilizing various types of KSQLDB joins, as well as their specificities and limitations

  • Understanding of various types of Windows and the operation mechanism of Time-based Window Aggregation and Window Join

  • Using Connect in KSQLDB

  • Integration of KSQLDB and Elasticsearch and visualization of analysis results through Kibana

Large-scale real-time streaming analysis system,
Easy and powerful with Kafka + ksqlDB!

Real-time streaming data analysis
Want to achieve efficiency and scalability?

If you are using Kafka , the easiest and fastest way to implement a large-scale real-time streaming analytics system is to use ksqlDB .

ksqlDB, which is installed and operated integrated with Kafka, can process/transform/analyze real-time streaming data simply with a few lines of SQL code , without using complex streaming APIs.

Excerpt from the official Confluent page (link)

Leading domestic and international companies are already facing the need to analyze streaming data in real time with incomparable large volumes and fast latency compared to the past and to immediately reflect the results, and are actively introducing ksqlDB to achieve this.

In the past, the complex Kafka Streams API was used to process/transform/analyze real-time streaming data based on Kafka, but now ksqlDB, which allows for easy and fast construction of streaming analysis systems with just simple queries, is becoming the trend for building real-time streaming data analysis systems.

ksqlDB is growing rapidly, but experts are scarce.

ksqlDB is rapidly replacing the existing Kafka Streams API due to its many advantages, including its easy and convenient SQL-based implementation.

However, finding professionals skilled enough to utilize ksqlDB in practical applications is extremely difficult. This is because ksqlDB is a relatively new solution, and most materials and lectures covering it are superficial and conceptually insufficient to build the skills necessary for practical use.


Real-time, high-volume streaming data processing applications made easy and fast.

'Kafka Complete Guide - ksqlDB'

This course is designed to help you grow into a ksqlDB expert with practical, hands-on content. Our goal is to help you take a leap forward as the ksqlDB expert your company desires .

Those who have repeatedly been blocked by the high wall of ksqlDB

Those who want to understand the core mechanism of ksqlDB

✅ Anyone who wants to immediately use ksqlDB for work

Accordingly, this lecture is filled with the contents that students must acquire in order to utilize ksqlDB in the field.

Unique features of this course
Check it out.

Learn the core operating mechanisms of key ksqlDB components through detailed visual aids and hands-on exercises.

While ksqlDB shares some similarities with a typical RDBMS, it also differs significantly in many aspects. Therefore, to effectively use ksqlDB, a detailed understanding of its operating mechanisms, including its key components—Stream, Table, Query, MView, and RocksDB—is essential. This course will help you master the core mechanisms of ksqlDB through detailed visualizations and hands-on exercises.

Various functions unique to ksqlDB, including joins, group by, window utilization, and practical training for practical use.

To effectively utilize ksqlDB in practice, you must be familiar with the various functions it provides, including JOIN, GROUP BY, and WINDOW. In particular, ksqlDB's JOIN, GROUP BY, and WINDOW operations have limitations that differ from those of SQL, and without understanding them, you cannot fully utilize ksqlDB. This course will provide hands-on practice in implementing these elements, allowing you to clearly understand these differences.

Additionally, through a separate "Online Shoe Shop" practical training section, we will guide you to a level where you can efficiently apply various real-time analytics in ksqlDB in your field.

From leveraging Connect in ksqlDB to Elasticsearch integration and Kibana visualization.

We'll explain how to integrate and leverage Connect in ksqlDB. You'll also learn how to store ksqlDB analysis results in Elasticsearch via Connect and visualize them using Kibana.

We will guide you from beginner to expert level with ksqlDB.

⚙️

Core mechanisms of ksqlDB's main components

🔎

Differences between Streams and Tables, how to use them, and how to create and manage key objects.

🧰

Understanding various query syntax and key functions of ksqlDB through practice

📊

Understanding and utilizing ksqlDB's unique Group by, MView, and Join, as well as differences and limitations compared to RDBMS.

🔐

Understanding of various types of windows and practicing the operation mechanism of time-based window aggregation and window join.

💾

Integration with Connect and collection and visualization of analysis results through Elasticsearch and Kibana

We've gone beyond a superficial overview of ksqlDB, devoting significant effort to covering core content and practical applications not found in any previous lectures or online resources. Furthermore, we've filled the curriculum with a variety of hands-on exercises to ensure a more natural understanding of the theory through hands-on practice.

After completing this course, you will find yourself becoming a ksqlDB expert who can compete with anyone else.

We have structured the content so that you can fully understand it through detailed explanations and various practical exercises.
(We provide students with a PDF of the lecture material, which is over 100 pages long.)

💡 Please note before taking the class!

  • This lecture only covers ksqlDB and not Kafka Streams.
  • Virtualbox VM compatibility may be difficult to practice on Macbook M1/M2 model PCs.

Practice environment

Server OS

The Kafka server OS is Ubuntu Linux 20.04, running on an Oracle VirtualBox VM. Although it uses Linux, it runs on a virtual machine, making it suitable for both Windows and macOS environments.

VirtualBox can be installed on most Windows and macOS platforms. However, for Macs, VirtualBox is not installed on the latest M1 models. Therefore, you must install Ubuntu using a virtual environment such as UTM. For M1 models, please ensure that Ubuntu can be installed in a virtual environment before selecting a course.

Confluent Kafka
Community Edition

Kafka uses Confluent Kafka Community Edition version 7.1.2, not Apache Kafka.

Confluent, founded by the core team behind Kafka, provides enterprise-grade Kafka with enhanced performance and convenience for enterprise customers. It offers 100% compatibility with Apache Kafka while also offering access to a wider range of Kafka modules and integrated binaries. With Confluent, you can leverage the powerful distributed Kafka system in a more elastic and scalable form. This reduces infrastructure deployment and maintenance burden and accelerates development.

Recommended PC Specifications

A full lab environment configuration may require a PC environment with 20-30GB of storage capacity and 4GB or more of RAM .


Check out the Q&A 💬

Q. Should I take the previous lecture, "The Complete Guide to Kafka - Core" or "Connect"?

Two sections of this course cover integration between ksqlDB and Connect. Even if you haven't taken "The Complete Kafka Guide - Connect," you should have a basic understanding of Connect and practical experience to fully understand the exercises in this section.

An understanding of Kafka Core is essential. It's recommended that you take the previous course, "The Complete Guide to Kafka - Core." However, even if you haven't taken it, if you have experience using Kafka's fundamentals—Broker, Producer, and Consumer—and have a solid grasp of the core concepts, you'll be able to follow this course.

Q. Do I need to have experience with RDBMS SQL to take this course?

Many of the exercises in this course are query-based. Therefore, you should have experience with basic RDBMS SQL syntax and GROUP BY and JOIN operations.

Recommended for
these people

Who is this course right for?

  • Anyone who wants to understand the main components of KSQLDB easily and deeply

  • Data engineer who wants to quickly and effectively build a large-scale real-time streaming data processing/transformation analysis system based on Kafka

  • Analysts and data scientists who need to leverage real-time streaming data analytics

  • Developers who want to migrate applications from existing Producer/Consumer-based or Kafka Streams-based to KSQLDB-based

Need to know before starting?

  • (It would be best if you took the Kafka Complete Guide - Core, but if not) you need a solid basic knowledge of topics/producers/consumers.

  • Basic knowledge of Kafka Connect

  • Many of the exercises are query-based. Basic SQL knowledge is required to understand joins and Group By.

Hello
This is

26,218

Learners

1,305

Reviews

3,973

Answers

4.9

Rating

13

Courses

(전) 엔코아 컨설팅

(전) 한국 오라클

AI 프리랜서 컨설턴트

파이썬 머신러닝 완벽 가이드 저자

Curriculum

All

139 lectures ∙ (21hr 31min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

17 reviews

4.8

17 reviews

  • jongho97han8855님의 프로필 이미지
    jongho97han8855

    Reviews 15

    Average Rating 5.0

    5

    30% enrolled

    • jink0608님의 프로필 이미지
      jink0608

      Reviews 2

      Average Rating 4.5

      4

      30% enrolled

      • chillguy님의 프로필 이미지
        chillguy

        Reviews 10

        Average Rating 5.0

        Edited

        5

        100% enrolled

        The teacher's bonus of teaching us how to create graphs using ElasticSearch and Kibana at the end felt like it crowned the finale of the Kafka roadmap. It was rewarding to see the graphs with my own eyes. Thank you for providing such a great lecture.

        • dooleyz3525
          Instructor

          Oh, congratulations on completing the Kafka roadmap finale course. Thank you for the kind review as well. ^^

      • sjhappy9769님의 프로필 이미지
        sjhappy9769

        Reviews 2

        Average Rating 5.0

        5

        6% enrolled

      • jsggo2001님의 프로필 이미지
        jsggo2001

        Reviews 8

        Average Rating 5.0

        5

        100% enrolled

        It was really helpful. I didn't know anything about Kafka, but from Kafka Core to Connect ksqldb, I think I'll be able to participate without hesitation when I have to work on Kafka or when a topic comes up. Of course, I'll have to make it truly mine through repetition, but I think it was really helpful.

        • dooleyz3525
          Instructor

          I felt rewarded for creating the Kafka lecture series. Thank you for the good review ^^

      Limited time deal

      $58.30

      24%

      $77.00

      dooleyz3525's other courses

      Check out other courses by the instructor!

      Similar courses

      Explore other courses in the same field!