![[개정판] 파이썬 머신러닝 완벽 가이드강의 썸네일](https://cdn.inflearn.com/public/courses/324238/cover/7e380aa0-48ba-4ee7-a6b2-8da7900568d6/324238-eng.png?w=420)
[개정판] 파이썬 머신러닝 완벽 가이드
권 철민
이론 위주의 머신러닝 강좌에서 탈피하여 머신러닝의 핵심 개념을 쉽게 이해함과 동시에 실전 머신러닝 애플리케이션 구현 능력을 갖출 수 있도록 만들어 드립니다.
초급
Python, 머신러닝, 통계
Through in-depth theoretical explanations of Kafka Connect and detailed practical training that can be used immediately in the field, we will help you grow into an expert in building data linkage and data pipelines based on Kafka Connect that are needed in the field.
Core Mechanisms of Kafka Connect's Key Components
Understanding CDC(Change Data Capture) and practical application techniques
Understanding MySQL Data Replication and CDC (Change Data Capture) and Practical Application Methods
Core Mechanisms and Features of the Debezium CDC Source Connector
Inter-RDBMS Data Integration Using Debezium CDC Source Connector
Know-how on building a Debizium Connect-based linkage system
Setting up and running JDBC-based Source Connector and Sink Connector environment
Application of various SMT classes for message conversion
Managing connections using REST API
Utilizing the Schema Registry and integrating with Connect
Managing schema registry using REST API
Connect for Apache Kafka Practices,
From principles to practical applications, everything is clear!
Kafka Connect allows you to easily, quickly, and reliably build real-time data connections between various systems through pre-built connectors without any separate coding implementation.
Many companies overseas have already adopted Kafka Connect, and in Korea, as Kafka Connect is utilized for integration between heterogeneous data systems and the construction of enterprise data pipelines, demand for talent with practical Kafka and Kafka Connect skills is growing. Unfortunately, however, learning materials for Kafka Connect remain scarce. Books, materials, and lectures offer only basic and superficial information, making it difficult to cultivate individuals with the practical skills required for practical work.
detailed
Mechanism Description
Practical level
Various examples
Issue Resolution
OK to the room
This course covers Kafka Connect at a level of detail and practicality unmatched in any other lecture or book. Through a detailed explanation of the mechanisms of Kafka Connect's core components and numerous hands-on examples that demonstrate various data integration and operational management using Connect , we aim to help you grow into a Kafka Connect expert in demand in the field .
Most enterprises' core data systems are RDBMSs. Real-time integration of physically separate databases is now dominated by Change Data Capture (CDC) . CDC is an excellent data integration technique that enables real-time integration of large amounts of data without delay while minimizing system load. Debezium Connector is a leading CDC solution that utilizes Kafka Connect to enable data integration between different RDBMSs.
Many companies are demanding talent capable of handling CDC-based connections. Therefore, this course will provide a detailed theoretical and practical explanation of the mechanisms of CDC and Debezium Connectors, their configuration and implementation, as well as various issues and solutions that may arise when applying Debezium in production.
We will provide you with detailed explanations and hands-on practice to acquire core fundamental knowledge on Connect Clusters, Connectors, SMT (Single Message Transform), Converters, and more, enabling you to utilize them freely.
We help you build a Kafka-based, practical data linkage system through various connector configuration parameters, internal mechanisms, and various application practices that can be applied to the RDBMS operating environment, such as SpoolDir Source, JDBC Source/Sink, and Debezium Source Connector.
We've covered a lot about the Debezium CDC source connector. We'll provide a detailed guide on how to build real-time connectivity between disparate RDBMSs in an RDBMS production environment using Debezium CDC and the JDBC Sink Connector.
We'll cover in detail the transfer and central management of schema data through Connect, Avro, and the Schema Registry, as well as schema compatibility, which is crucial in practice. Through this, you'll learn how to integrate Connect and the Schema Registry to build efficient enterprise data integration and data pipelines, which are essential in practice.
You will learn how to create/modify/delete/manage key elements of Connect and Schema Registry through various REST APIs.
We provide a 200-page course textbook to all students. We hope it will help you learn Kafka Connect.
The Kafka server OS is Ubuntu Linux 20.04, running on an Oracle VirtualBox VM. Although it uses Linux, it runs on a virtual machine, making it suitable for both Windows and macOS environments.
VirtualBox can be installed on most Windows and macOS platforms. However, for Macs, VirtualBox is not installed on the latest M1 models. Therefore, you must install Ubuntu using a virtual environment such as UTM. For M1 models, please ensure that Ubuntu can be installed in a virtual environment before selecting a course.
Kafka uses Confluent Kafka Community Edition version 7.1.2, not Apache Kafka.
Confluent, founded by the core team behind Kafka, provides enterprise-grade Kafka with enhanced performance and convenience for enterprise customers. It offers 100% compatibility with Apache Kafka while also offering access to a wider range of Kafka modules and integrated binaries. With Confluent, you can leverage the powerful distributed Kafka system in a more elastic and scalable form. This reduces infrastructure deployment and maintenance burden and accelerates development.
Although file data linking is also provided as a practice, such as with the Spooldir Source Connector, most of the Connect practice linking is centered around data linking between RDBMSs.
In particular, many exercises use the same MySQL database for both Source and Sink. The Source uses MySQL, while the Sink uses PostgreSQL. The versions used in these exercises are MySQL 8.0.31 and PostgreSQL 12 .
A full lab environment configuration may require a PC environment with 20-30GB of storage capacity and 4GB or more of RAM .
Q. Why should I learn Kafka Connect?
Kafka Connect is a core component for Kafka-based data integration . Many companies that have already adopted Kafka are effectively leveraging Kafka Connect to easily build large-scale data pipelines.
Kafka Connect is used to interconnect heterogeneous data systems, including major RDBMSs such as Oracle, MySQL, and PostgreSQL, as well as NoSQL systems such as MongoDB and ElasticSearch, and DW systems such as RedShift, SnowFlake, Vertica, and Teradata, through over 120 different connectors.
Kafka Connect allows for easy interconnection/integration of heterogeneous data systems without the need for separate coding implementation. In particular, its use and utilization are increasing in many companies due to its advantages such as reduced interconnection S/W costs through Community license and real-time interconnection of large amounts of data without delay based on CDC.
If you master Kafka Connect through this course, you will be able to take a step forward as a Kafka expert that companies want.
Q. Should I take the previous lecture, "Kafka Complete Guide - Core"?
It would be better if you took the previous lecture, Kafka Complete Guide - Core Edition, but even if you didn't take the lecture, if you have a good understanding of the basic concepts of Kafka, such as Broker, Producer, and Consumer, and have experience applying Kafka message sending and reading, you can sufficiently take this lecture.
Q. Do I need to have RDBMS experience to take this course?
Unfortunately, this course requires at least 3 months of RDBMS experience .
You can do most of the lecture exercises if you just understand the basics of RDBMS table and column alteration creation. However, if you do not have some experience with RDBMS, you may find the exercises difficult, even though the lecture explains CDC and RDBMS replication in detail.
Who is this course right for?
Anyone who wants to understand the internal mechanisms of Kafka Connect and apply them in practice
Data engineers or architects who want to build a data pipeline and understand CDC-based data architecture
DBAs or system administrators who need to operate JDBC or Debezium CDC Connector
DW developer considering ETL and DB linkage through real-time synchronization of operational DB
Developers and architects considering CDC-based data linkage when configuring microservice-based architecture
Need to know before starting?
Basic knowledge of Kafka Broker, Producer, and Consumer
More than 3 months of RDBMS development or operation experience
26,182
Learners
1,304
Reviews
3,973
Answers
4.9
Rating
13
Courses
(전) 엔코아 컨설팅
(전) 한국 오라클
AI 프리랜서 컨설턴트
파이썬 머신러닝 완벽 가이드 저자
All
147 lectures ∙ (24hr 35min)
Course Materials:
Limited time deal
$74,250.00
25%
$77.00
Check out other courses by the instructor!
Explore other courses in the same field!