강의

멘토링

로드맵

BEST
Data Science

/

Data Engineering

Kafka Complete Guide - ksqlDB

This course is designed to help you learn the use of ksqlDB and its core mechanisms through various hands-on exercises. After completing the course, you will be able to easily and quickly build a real-time streaming data analysis system based on Kafka.

(4.8) 17 reviews

354 learners

  • dooleyz3525
카프카
Kafka
ksqldb
데이터 엔지니어링
streaming
sql
ksqlDB
Data Engineering

Reviews from Early Learners

What you will learn!

  • From KSQLDB's basic concepts to advanced architecture

  • Difference between Stream and Table, and Stateful Streaming Processing Mechanism

  • Information on the creation and management of KSQLDB's main objects, understanding of various data types

  • RocksDB operation mechanism in KSQLDB

  • Understanding various query syntax and functions of KSQLDB

  • Understanding and using Group by and Mview, and its specificity and limitations

  • Understanding and utilizing various types of KSQLDB joins, as well as their specificities and limitations

  • Understanding of various types of Windows and the operation mechanism of Time-based Window Aggregation and Window Join

  • Using Connect in KSQLDB

  • Integration of KSQLDB and Elasticsearch and visualization of analysis results through Kibana

Large-scale real-time streaming analysis system,
Easy and powerful with Kafka + ksqlDB!

Real-time streaming data analysis
Want to achieve efficiency and scalability?

If you are using Kafka , the easiest and fastest way to implement a large-scale real-time streaming analytics system is to use ksqlDB .

ksqlDB, which is installed and operated integrated with Kafka, can process/transform/analyze real-time streaming data simply with a few lines of SQL code , without using complex streaming APIs.

Excerpt from the official Confluent page (link)

Leading domestic and international companies are already facing the need to analyze streaming data in real time with incomparable large volumes and fast latency compared to the past and to immediately reflect the results, and are actively introducing ksqlDB to achieve this.

In the past, the complex Kafka Streams API was used to process/transform/analyze real-time streaming data based on Kafka, but now ksqlDB, which allows for easy and fast construction of streaming analysis systems with just simple queries, is becoming the trend for building real-time streaming data analysis systems.

ksqlDB is growing rapidly, but experts are scarce.

ksqlDB is rapidly replacing the existing Kafka Streams API due to its many advantages, including its easy and convenient SQL-based implementation.

However, finding professionals skilled enough to utilize ksqlDB in practical applications is extremely difficult. This is because ksqlDB is a relatively new solution, and most materials and lectures covering it are superficial and conceptually insufficient to build the skills necessary for practical use.


Real-time, high-volume streaming data processing applications made easy and fast.

'Kafka Complete Guide - ksqlDB'

This course is designed to help you grow into a ksqlDB expert with practical, hands-on content. Our goal is to help you take a leap forward as the ksqlDB expert your company desires .

Those who have repeatedly been blocked by the high wall of ksqlDB

Those who want to understand the core mechanism of ksqlDB

✅ Anyone who wants to immediately use ksqlDB for work

Accordingly, this lecture is filled with the contents that students must acquire in order to utilize ksqlDB in the field.

Unique features of this course
Check it out.

Learn the core operating mechanisms of key ksqlDB components through detailed visual aids and hands-on exercises.

While ksqlDB shares some similarities with a typical RDBMS, it also differs significantly in many aspects. Therefore, to effectively use ksqlDB, a detailed understanding of its operating mechanisms, including its key components—Stream, Table, Query, MView, and RocksDB—is essential. This course will help you master the core mechanisms of ksqlDB through detailed visualizations and hands-on exercises.

Various functions unique to ksqlDB, including joins, group by, window utilization, and practical training for practical use.

To effectively utilize ksqlDB in practice, you must be familiar with the various functions it provides, including JOIN, GROUP BY, and WINDOW. In particular, ksqlDB's JOIN, GROUP BY, and WINDOW operations have limitations that differ from those of SQL, and without understanding them, you cannot fully utilize ksqlDB. This course will provide hands-on practice in implementing these elements, allowing you to clearly understand these differences.

Additionally, through a separate "Online Shoe Shop" practical training section, we will guide you to a level where you can efficiently apply various real-time analytics in ksqlDB in your field.

From leveraging Connect in ksqlDB to Elasticsearch integration and Kibana visualization.

We'll explain how to integrate and leverage Connect in ksqlDB. You'll also learn how to store ksqlDB analysis results in Elasticsearch via Connect and visualize them using Kibana.

We will guide you from beginner to expert level with ksqlDB.

⚙️

Core mechanisms of ksqlDB's main components

🔎

Differences between Streams and Tables, how to use them, and how to create and manage key objects.

🧰

Understanding various query syntax and key functions of ksqlDB through practice

📊

Understanding and utilizing ksqlDB's unique Group by, MView, and Join, as well as differences and limitations compared to RDBMS.

🔐

Understanding of various types of windows and practicing the operation mechanism of time-based window aggregation and window join.

💾

Integration with Connect and collection and visualization of analysis results through Elasticsearch and Kibana

We've gone beyond a superficial overview of ksqlDB, devoting significant effort to covering core content and practical applications not found in any previous lectures or online resources. Furthermore, we've filled the curriculum with a variety of hands-on exercises to ensure a more natural understanding of the theory through hands-on practice.

After completing this course, you will find yourself becoming a ksqlDB expert who can compete with anyone else.

We have structured the content so that you can fully understand it through detailed explanations and various practical exercises.
(We provide students with a PDF of the lecture material, which is over 100 pages long.)

💡 Please note before taking the class!

  • This lecture only covers ksqlDB and not Kafka Streams.
  • Virtualbox VM compatibility may be difficult to practice on Macbook M1/M2 model PCs.

Practice environment

Server OS

The Kafka server OS is Ubuntu Linux 20.04, running on an Oracle VirtualBox VM. Although it uses Linux, it runs on a virtual machine, making it suitable for both Windows and macOS environments.

VirtualBox can be installed on most Windows and macOS platforms. However, for Macs, VirtualBox is not installed on the latest M1 models. Therefore, you must install Ubuntu using a virtual environment such as UTM. For M1 models, please ensure that Ubuntu can be installed in a virtual environment before selecting a course.

Confluent Kafka
Community Edition

Kafka uses Confluent Kafka Community Edition version 7.1.2, not Apache Kafka.

Confluent, founded by the core team behind Kafka, provides enterprise-grade Kafka with enhanced performance and convenience for enterprise customers. It offers 100% compatibility with Apache Kafka while also offering access to a wider range of Kafka modules and integrated binaries. With Confluent, you can leverage the powerful distributed Kafka system in a more elastic and scalable form. This reduces infrastructure deployment and maintenance burden and accelerates development.

Recommended PC Specifications

A full lab environment configuration may require a PC environment with 20-30GB of storage capacity and 4GB or more of RAM .


Check out the Q&A 💬

Q. Should I take the previous lecture, "The Complete Guide to Kafka - Core" or "Connect"?

Two sections of this course cover integration between ksqlDB and Connect. Even if you haven't taken "The Complete Kafka Guide - Connect," you should have a basic understanding of Connect and practical experience to fully understand the exercises in this section.

An understanding of Kafka Core is essential. It's recommended that you take the previous course, "The Complete Guide to Kafka - Core." However, even if you haven't taken it, if you have experience using Kafka's fundamentals—Broker, Producer, and Consumer—and have a solid grasp of the core concepts, you'll be able to follow this course.

Q. Do I need to have experience with RDBMS SQL to take this course?

Many of the exercises in this course are query-based. Therefore, you should have experience with basic RDBMS SQL syntax and GROUP BY and JOIN operations.

Recommended for
these people

Who is this course right for?

  • Anyone who wants to understand the main components of KSQLDB easily and deeply

  • Data engineer who wants to quickly and effectively build a large-scale real-time streaming data processing/transformation analysis system based on Kafka

  • Analysts and data scientists who need to leverage real-time streaming data analytics

  • Developers who want to migrate applications from existing Producer/Consumer-based or Kafka Streams-based to KSQLDB-based

Need to know before starting?

  • (It would be best if you took the Kafka Complete Guide - Core, but if not) you need a solid basic knowledge of topics/producers/consumers.

  • Basic knowledge of Kafka Connect

  • Many of the exercises are query-based. Basic SQL knowledge is required to understand joins and Group By.

Hello
This is

26,326

Learners

1,323

Reviews

3,983

Answers

4.9

Rating

13

Courses

(전) 엔코아 컨설팅

(전) 한국 오라클

AI 프리랜서 컨설턴트

파이썬 머신러닝 완벽 가이드 저자

Curriculum

All

139 lectures ∙ (21hr 31min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

17 reviews

4.8

17 reviews

  • jongho97.han님의 프로필 이미지
    jongho97.han

    Reviews 15

    Average Rating 5.0

    5

    30% enrolled

    • 정진묵님의 프로필 이미지
      정진묵

      Reviews 2

      Average Rating 4.5

      4

      30% enrolled

      • logt님의 프로필 이미지
        logt

        Reviews 10

        Average Rating 5.0

        Edited

        5

        100% enrolled

        선생님이 마지막에 보너스로 ElasticSearch와 Kibana를 이용해서 그래프 만드는 것까지 모든 실습을 가르쳐 주셔서 Kafka 로드맵의 피날레를 장식한 것 같습니다. 직접 그래프를 눈으로 확인하니 뿌듯하네요. 좋은 강의 제공해주셔서 감사합니다.

        • 권 철민
          Instructor

          오, Kafka 로드맵 피날레 강의까지 다 들으셨다니, 축하드립니다. 좋은 수강평도 감사 드려요 ^^

      • sjhappy님의 프로필 이미지
        sjhappy

        Reviews 2

        Average Rating 5.0

        5

        6% enrolled

      • jsggo2001님의 프로필 이미지
        jsggo2001

        Reviews 8

        Average Rating 5.0

        5

        100% enrolled

        정말 많은 도움이 되었습니다 카프카에 대해서 1도 몰랐는데 카프카 코어편부터 커넥트 ksqldb 까지 어디가서 카프카에 대한 업무를 해야할때나 주제가 나오면 주저 없이 참여할 수 있을것 같습니다 물론 반복을 통해서 진정 내것이 되도록 해야하겠지만 그래도 정말 많은 도움이 된것 같습니다.

        • 권 철민
          Instructor

          Kafka 강의 시리즈를 만든 보람을 느꼈습니다. 좋은 수강평 감사합니다 ^^

      $77.00

      dooleyz3525's other courses

      Check out other courses by the instructor!

      Similar courses

      Explore other courses in the same field!