Apache Flink, the real-time ultra-low latency platform used by Naver interviewers
Most developers are still stuck on Batch and CronJobs when it comes to data processing. However, in real-world production environments, data is generated continuously, and failing to process that flow immediately leads to latency, bottlenecks, and consistency issues. Having personally dealt with real-time recommendations, state synchronization, and event delay problems in high-traffic environments, I’ve often asked myself, "Is batch processing really the right approach for this?" This course begins with that very question. It explores how to use Apache Flink to perform computations the moment data flows, manage state securely, and produce accurate results based on Event Time—all from a practical perspective. Beyond simple theoretical explanations, you will experience how real-time stream processing systems are designed and operated through actual source code and architectures. For those who found real-time processing vague or were curious about the world beyond messaging, this course will provide a clear sense of direction.
I'm already opening my second course!! Personally, I have a strong desire to share knowledge, so I wanted to freely share everything I know with you.
The topic I want to cover this time is Apache Flink, which I've selected as a subject that you should definitely know about.
Let me be honest!! If you don't have knowledge about real-time communication, this content might be difficult for you to understand. (Of course, I'll teach you everything, and honestly, I think the course is structured really well. haha)
But you absolutely must learn this. Even though it's difficult, I want to guarantee and tell you that you must learn it.
Real-time communication seems to be becoming extremely important in today's era. I think we need to provide more content to generate more traffic and collect more data.
In fact, this aspect is reflected in numerous large corporations and unicorn startups. Netflix's recommendation system that you know, Uber's real-time fare calculation, and recommendation systems like Musinsa
I haven't seen any platform or service that you use that doesn't include this real-time communication.
So please, I really hope many people learn through this video. You really must learn this. Please, I'm begging you for the sake of your career hahaha
5.0
에이미
89% enrolled
I think this is one of the best courses I've seen on Inflearn..!!
The theoretical parts are not lacking at all, and I think you can learn a great deal across the board, from example code to hands-on practice.
When explaining the concept of Source, I found it even more impressive that you implemented it without using external storage or MSQ. If you had covered that part, it probably would have deviated somewhat from the course objectives.
I really enjoyed the course. I'm learning so much from it.
5.0
warna
85% enrolled
I recently got a job at Naver Cloud while watching your lectures!! I'm truly so grateful.
You're my favorite instructor on Inflearn, and although you may not be famous, I think these are excellent lectures because every course shows how you teach so many concepts in a short time and make efforts to use time efficiently.
This time too, I enrolled right as soon as you opened the new course... and as expected, the content is really beneficial. It was content I could learn a lot from.
What you will gain after the course
Design for processing data based on real-time streams rather than batches
Accurately process late data using Event Time and Watermarks.
Implementing 'stateful' real-time applications through stateful processing
Production-ready stream processing pipelines with Apache Flink
Architecture for designing real-time recommendation, billing, and aggregation systems
Development · Programming · Data Engineering
If you are still stuck with batches and cron jobs
Data flows in constantly, but processing only runs at scheduled times. As delays build up in between, they become bottlenecks and lead to consistency issues. Apache Flink is an engine that processes that flow as it arrives. This course builds that architecture from the ground up.
Difficulty: Beginner
Example code provided
Event Time · Watermark
Direct implementation
Unlimited access period
From Architecture to ImplementationCluster Structure · State Management · Writing Examples
Event Time · WatermarkTackling the most challenging concepts in stream processing head-on
Two current developers10 years at NAVER headquarters · Pangyo platform server development
Flink is a distributed stream processing engine that processes continuous data in real time. While batch processing collects and processes data at set times, streaming processes data as soon as it arrives.
Where this lecture begins
You've made it to Kafka, but there's nothing after that.
Batch · CronJob
Collect and process later
It runs all at once at a scheduled time. Data that comes in during the interval must wait. Latency builds up as a result.
The paradigm is different
Stream
as they arrive
When a single event comes in, it is processed right then and there. This is a way of handling data that has no end.
Many people make it as far as streaming events into Kafka. However, they get stuck at the stage of aggregating and managing state on top of that flow. This course addresses that very point.
Where this lecture began
This is an actual conversation that took place.
Not long after the NATS lecture ended, I was contacted again. This lecture began from that conversation.
After finishing the NATS lecture, I was contacted again during the holiday.
NNaver DeveloperI've decided I want to cover one more topic; are you busy, Hong?
HHongWhy are you looking for me during the holidays... What kind of flash of inspiration did you have?
NNaver DeveloperAfter dealing with NATS, I've wanted to explore real-time communication further. What do you think about trying out Apache Flink?
HHongI'm all for trying anything new, but I haven't used that before.
NNaver DeveloperThat's fine. I'll prepare all the materials. Apache Flink is really great, but people don't seem to know much about it. From a career perspective, real-time data collection is truly important, so I don't know why it's not being covered more.
NNaver DeveloperI wonder if people aren't curious about things like real-time recommendations or real-time billing...
HHongI'm always down for it. Let's prepare it together. I'll dedicate my years of experience to making this happen.
What you will learn
In the order of Concept · Structure · State · Implementation
Why Stream?We will look at the limitations of traditional systems first. The difference in paradigms between batch and stream will be shown through example code.
Cluster StructureHow Flink is distributed and how the execution layers are structured
Task Slots and ParallelismHow to secure throughput. From data exchange to performance optimization.
Event Time and WatermarksHow to handle late-arriving events. The most difficult concept in streaming.
DataStream API and WindowThe API actually used in practice. This is the most significant section.
State ManagementHow stream processing carries state
Direct ImplementationIn the final section, we will write and run an example together.
Where is it placed?
Flink is on the path where data passes through.
Flink is not storage. It is positioned to receive data from somewhere (Source), process it, and send it out somewhere else (Sink). These are the systems commonly attached to the front and back.
Apache Flink
Apache Kafka
Apache Pulsar
RabbitMQ
Amazon Kinesis Data Streams
Elasticsearch
Amazon S3
Apache Cassandra
Amazon DynamoDB
Apache Hadoop
Apache NiFi
What this lecture covers is not the usage of individual connectors, but the concepts of Source and Sink themselves. That is addressed in the first lecture of Section 4, and once you grasp the concepts, anything attached to either side will be understood in the same way.
It doesn't end with just listening to the concepts.
The entire final section is dedicated to writing and executing examples.
We start by establishing concepts and architecture in the first half, then move through APIs and state management to hands-on implementation in the second half.
01
Course Introduction and Materials
Course Introduction
Source Code and Skeleton
Apache Flink Site
02
Introduction to Apache Flink and Architecture
Overview and definition of Apache Flink and the limitations of traditional systems
Apache Flink's role as a distributed stream processing engine
[Example Code] Stream Processing vs. Batch Processing Paradigms
[Example Code] Key factors for real-time communication in Apache Flink
03
From the basics to core concepts for stream processing
Apache Flink's Cluster Components and Execution Hierarchy
Strategies for Securing Throughput via Apache Flink's Task Slots and Parallel Processing
Data exchange and performance optimization through Task Slots
[Example Code] Time Semantics and Watermark Techniques in Flink
04
Data Stream API and Window for Practical Application
[Example Code] The most frequently used and fundamental Stream operators [ Sink and Source ]
[Example Code] Most frequently used and fundamental Stream operators [ Transformations, Aggregation, Combination ]
[Example Code] Advanced operators to prepare for all situations [ Process, KeyedProcess, CoProcess, ProcessWindow ]
[Example Code] Advanced operators to prepare for all situations [ Side Output, CEP ]
[Example Code] Window Types for state storage most commonly used in practice [ Tumbling, Sliding ]
[Example Code] The most commonly used Window Types for state storage in practice [ Session, Global ]
[Example Code] Window Function [ Process With Aggregate ]
05
State Management
[Example Code] Complex Flink state objects that anyone can definitely understand
Storage and access performance tradeoffs through State Backends
Queryable State supporting external state lookups
06
Examples to write and run together
Deep Dive into Building Lightweight Environments Using Docker Compose
[Hands-on] WordCount Example Guide
[Hands-on] Windowing Example Guide
[Practice] State Management Example Guide
[Hands-on] Checkpoint Example Guide
[Hands-on] Event Time vs Processing Time Example Guide
For people like this
Who is this course for?
Backend developers who feel the limitations of batch processing and cron jobs
Developers who have used Kafka but don't know the next step
Those who want to understand the architecture of real-time recommendation, billing, and aggregation systems
Stream processing beginners who find the concepts of Event Time and State difficult
Mid- to senior-level developers who want to differentiate their careers through real-time data processing
The current market
Talk that AI will replace developers
New hiring is decreasing, and companies are looking to hire only proven professionals. These are articles that have come out in the past few months.
Krafton, which achieved record-breaking performance, has begun downsizing its workforce. The reason given was its transition into an 'AI-first' company.
SW specialized companies are halting the recruitment of new developers. There are also projections that the hiring of entry-level developers will plummet by 77%.
53% of game designers answered, "AI will replace my job." Cases of recommended resignation have also been reported.
As much as companies are feeling uneasy, those being hired must demonstrate a clearer distinction. Ultimately, studying is something you do on your own, but depending on what you look at and from which perspective, the depth of what you can explain changes even when spending the same amount of time. I created this lecture to pass on that perspective.
Course Reviews
Stories from those who listened first
I copied this directly from the Inflearn course reviews.
"I consider this one of the best courses I've taken on Inflearn. The theoretical parts are not lacking at all, and I think you can learn a lot overall from the example code and subsequent practice sessions. It was particularly impressive that when teaching the concept of 'Source,' it was implemented without necessarily using external storage or message queues."
Amy · Written after 89% completion
"I recently changed jobs to Naver Cloud while watching your lectures. Thank you so much. Even though it might not be famous, it's a great course because I can see in every lecture how you try to teach so many concepts in a short amount of time and strive to use time efficiently."
warna · Written after 85% completion
Instructor
Created together by two current developers
It was co-created by Ande, a 10-year backend developer at NAVER headquarters, and Hong, a platform server developer in Pangyo.
NAVER · BACKEND ENGINEER · 10th Year
Ande
I am a 10-year veteran backend developer building servers at NAVER headquarters. Most of the architectures and decision-making criteria covered in this course are things I've personally experienced and organized while working in the field. Please feel free to leave any questions.
"There comes a point where you can no longer hold out with batch processing. That is when you must change the architecture."
Current NAVER Server (HQ) Developer · Former Shinsegae Group Backend Developer · Former Healthcare Startup Server Developer · Computer Science Major
Real-time Data ProcessingHigh TrafficBackend ArchitectureStream Pipeline
Knowledge Sharer · Pangyo Platform Server Development
Hong
I am in charge of platform server development in Pangyo. I continue my activities as a knowledge sharer to share the methods I studied myself and the problems and solutions encountered in practice. The lectures are not created alone but together with several developers currently working in the industry.
"The word 'real-time' sounds cool, but when you actually try to code it, the concept of time itself is difficult. That is where we begin."
Platform Server DevelopmentData PipelineMultiple Backend Courses
Frequently Asked Questions
Apache Flink Course Frequently Asked Questions
Q.Is it okay if I am new to stream processing?
⌄
The difficulty level is set to introductory. Starting from Section 2, I demonstrate the limitations of traditional systems and show the differences between batch and stream processing through example code, before moving into the architecture of Flink. I designed the course assuming you are hearing the concept of "streams" for the very first time.
Q.Do I need to know Kafka?
⌄
It is not mandatory. However, if you have experience streaming events through Kafka, you will grasp what comes "next" much faster. That is why "developers who do not know the stages after Kafka" are included in the recommended audience.
Q.Why are Event Time and Watermarks so difficult?
⌄
This is because the time an event occurs and the time it arrives at the system are different. If the network is delayed, events that happened earlier may arrive later. The mechanism to handle this discrepancy is the Watermark; without understanding this, you won't be able to explain why aggregation results are off. We cover this in Section 3 along with example code.
Q.Do we write the code ourselves?
⌄
The entire final section is dedicated to that. You will set up a lightweight environment using Docker Compose and write and execute examples one by one, covering WordCount, Windowing, State Management, Checkpoints, and the comparison between Event Time and Processing Time. The source code and skeleton projects are provided as resources in Section 1, so you can download them and get started.
Q.What is this technology used for?
⌄
It is for areas where data must be reflected as soon as it arrives, such as real-time recommendations, billing, and aggregation. The kind of tasks where, if processed in batches at set times, the resulting delay would be directly visible to the user.
Things that cannot be caught up with by batch processing
We will take a look at the structure of processing data at the speed it arrives, all the way to the end.
It takes a long time when you're stuck alone. There is a separate space where you can comfortably share everything from parts you get stuck on while taking the lecture, questions that arise while applying it to your own service, to stories about careers and the industry.
OPEN CHAT · Developer Community
What do we share?
We discuss real-world problems encountered in practice, the architectures others have chosen, and stories about developer careers. Questions related to the lectures are also welcome.
"The places where you get stuck are usually where others get stuck too. Asking someone who has already been through it is the fastest way."
I started studying development after becoming interested in it while idling at home, and I am currently responsible for platform server development in Pangyo. I am continuing my activities as a knowledge sharer because I want to provide you with the methods I used to study, as well as the various problems and solutions you may encounter in practice.
These lectures are not created solely through my own knowledge. There are others who collaborate on every lecture.
I'm already opening my second course!! Personally, I have a strong desire to share knowledge, so I wanted to freely share everything I know with you.
The topic I want to cover this time is Apache Flink, which I've selected as a subject that you should definitely know about.
Let me be honest!! If you don't have knowledge about real-time communication, this content might be difficult for you to understand. (Of course, I'll teach you everything, and honestly, I think the course is structured really well. haha)
But you absolutely must learn this. Even though it's difficult, I want to guarantee and tell you that you must learn it.
Real-time communication seems to be becoming extremely important in today's era. I think we need to provide more content to generate more traffic and collect more data.
In fact, this aspect is reflected in numerous large corporations and unicorn startups. Netflix's recommendation system that you know, Uber's real-time fare calculation, and recommendation systems like Musinsa
I haven't seen any platform or service that you use that doesn't include this real-time communication.
So please, I really hope many people learn through this video. You really must learn this. Please, I'm begging you for the sake of your career hahaha
LOL I've never seen someone so passionate before. I also think that real-time communication as a career is truly a huge help from a developer's perspective. I really resonate with what this person said, and I hope many people recognize and take note of the parts we're relating to.
It's always an honor to create great courses together~~! Looking forward to continuing to work with you!!
At first I was like, "Do I really need to know this?? Is it right for my career to study this??" But wow, this is really so beneficial. It's not just a simple lecture - you learn about the service called Flink, but it seems like a lecture that lets you see beyond that.
I really enjoyed the lecture.
Hello Boosta!! It's a difficult platform, but I tried my best to explain and share things as simply as possible. Thank you for your encouraging review!!
I recently got a job at Naver Cloud while watching your lectures!! I'm truly so grateful.
You're my favorite instructor on Inflearn, and although you may not be famous, I think these are excellent lectures because every course shows how you teach so many concepts in a short time and make efforts to use time efficiently.
This time too, I enrolled right as soon as you opened the new course... and as expected, the content is really beneficial. It was content I could learn a lot from.
Hello warna! Naver Cloud, wow!!! You really made it there haha I'm so jealous!! If I get the chance, maybe I could too...? lol Just kidding. I'll continue to cover more useful and interesting content and become an instructor who connects with all of you.
Have a great day and I hope only good things come your way!!
I think this is one of the best courses I've seen on Inflearn..!!
The theoretical parts are not lacking at all, and I think you can learn a great deal across the board, from example code to hands-on practice.
When explaining the concept of Source, I found it even more impressive that you implemented it without using external storage or MSQ. If you had covered that part, it probably would have deviated somewhat from the course objectives.
I really enjoyed the course. I'm learning so much from it.
Hello Amy!! Thank you so much for leaving this review. To say it's one of the best lectures ㅠㅠ that's such an encouraging comment.
I'll prepare the next lecture with even more useful content that can help you, Amy. Have a great day!
I really enjoyed the lecture. The structure is excellent. I think I've clearly understood what real-time communication is through just this one lecture.
It wasn't simply about covering the Flink platform, but also content where I could get a glimpse of that senior developer's perspective.
Hello, developer born in '02!! Thank you for leaving such a positive review!! As you mentioned, I wanted to provide a perspective that allows you to see things from a broader viewpoint rather than just simple usage methods.
I'll continue to provide more beneficial lectures in the future. Thank you for your review, and have a great day!!