inflearn logo

Kakao Interviewer's Hands-on Spring Batch: Everything About Large-Scale Data Processing

The biggest challenges in operating batch systems that process millions of records are performance degradation and the difficulty of recovery in the event of a failure. These issues cannot be solved with simple loops or poor design. Based on my experience of reliably processing large-scale data in real-world service environments, this course covers the structure and operating principles of Spring Batch and provides design methods to achieve both performance and stability. Moving beyond simple functional descriptions, this course will help you develop fundamental data processing problem-solving skills through batch architectures that can be applied immediately in practice.

(5.0) 2 reviews

65 learners

Level Beginner

Course period Unlimited

Java
Java
Spring
Spring
Spring Boot
Spring Boot
Spring Batch
Spring Batch
spring-framework
spring-framework
Java
Java
Spring
Spring
Spring Boot
Spring Boot
Spring Batch
Spring Batch
spring-framework
spring-framework

What you will gain after the course

  • Spring Batch design capability to reliably process large-scale data (millions to tens of millions of records)

  • Practical skills to implement chunk-based processing, transaction management, and retry/skip strategies in actual code

  • Ability to design scalable and maintainable batch architectures utilizing Job / Step structures

  • Problem-solving skills to independently design and apply reprocessing and recovery strategies in the event of a failure

  • Experience in implementing parallel processing and optimizing batch performance using multi-threading, partitioning, and other techniques.

⚡ How is your Batch processing application doing?

😁 Toss Developer: These days, I'm scared it'll crash every time I run a batch... There's just too much data now.

😄Kakao Interviewer: About how many records do you process?? Since it's the finance sector, I imagine it must be quite a lot.

😁 Toss Developer: Processing tens of millions of records is standard for a single run... but the processing speed is an issue, and recovery is so difficult if a failure occurs. It feels like it might be because Spring is a bit heavy..

😄 Naver Interviewer: Hmm... Is that so?? Spring itself is heavy, but if you apply something like Multi Threaded, it doesn't run that heavily unless it's a DB issue.

😄Kakao Interviewer: I agree. Even things that are hard to recover from don't seem that difficult if you introduce something like a Listener. Since adding things like Skip and Retry provides individual Transaction support.

😁 Toss Developer: True.. but why is it so slow?? I guess I didn't study Spring Batch well enough lol.

😄Kakao Interviewer: That's just how you are lolololol. I'll try to turn this into a lecture. There are more people than I thought who don't know Spring Batch that well.

😄Hong : At this point... you don't even ask about my schedule anymore, you just throw it at me;;

Java, Spring, Spring Boot, Spring Batch, spring-framework

⎈ Are you handling large-scale data batch processing with just simple loops?

In practical work, there inevitably comes a moment when data processing goes beyond simple CRUD.

  • Processing millions to tens of millions of data records

  • The issue of reprocessing upon failure

  • Transaction management and performance issues

  • Batch Scheduling and Operations

Can all these problems be solved with just a few lines of code? Or would you need to write tens of thousands of lines?? No, if you take that approach, your Spring Application will not be safe.

Therefore, this course covers various perspectives and features for designing large-scale data processing using Spring Batch. Please keep in mind that this is not simply about "how to run a batch."

👉 How to design a structure that can be used in actual services

👉 What the methods are for processing data reliably

👉 What are the ways to maximize the performance of Spring Batch?

👉 How to go beyond simple data processing and implement monitoring

🚀 This is not a lecture that only covers theory. I hope you take this time to learn how to directly use and implement the features of Spring Batch through various actual source codes.

🌳 Why Spring Batch!!

Spring Batch is a framework for processing large volumes of data, enabling processing that considers stability and scalability beyond simple repetitive tasks.

Spring Batch has the following features.

  • Chunk-based Processing Model
    → Optimizes memory usage and performance by dividing large amounts of data into specific units for processing

  • Transaction Management
    → Manages transactions in chunk units, allowing for stable rollback and reprocessing in case of failure

  • Retry / Skip / Fault Tolerance
    → Provides flexible recovery strategies instead of total shutdown when specific data processing fails

  • Job / Step-based Structure
    → Allows for clear design by dividing complex batch logic into sequential steps

  • ExecutionContext-based State Management
    → Provides a restart function to resume interrupted tasks

  • Optimization for large-scale data processing
    → Performance can be maximized through multi-threading, partitioning, and parallel processing.

The official documentation describes Spring Batch as follows.

A lightweight, comprehensive batch framework designed to enable the development of robust batch applications vital for the daily operations of enterprise systems.

Spring Batch provides reusable functions that are essential in processing large volumes of records, including logging/tracing, transaction management, job processing statistics, job restart, skip, and resource management. It also provides more advanced technical services and features that will enable extremely high-volume and high performance batch jobs through optimization and partitioning techniques. Simple as well as complex, high-volume batch jobs can leverage the framework in a highly scalable manner to process significant volumes of information.

( Spring Batch is a lightweight and comprehensive batch framework that enables the development of reliable batch applications essential for the daily operations of enterprise systems. It provides core functions required for large-scale data processing—such as logging and tracing, transaction management, job processing statistics, job restart, skip processing, and resource management—in a reusable form, allowing for more stable implementation of complex batch logic. Furthermore, it supports ultra-large data processing and high-performance batch operations by utilizing techniques such as optimization and partitioning. Designed with a scalable structure, it is a framework that helps process massive amounts of data reliably, ranging from simple batches to complex, large-scale batch operations. )

In modern backend systems, simply building a good API server is not the end of the story.

👉 Data keeps accumulating
👉 The ability to process that data
👉 And handle it reliably

These are exactly the things that become a developer's competitive edge in the field. Through this course, I hope you will understand Spring Batch not just as a simple "technology," but as a core tool for handling large-scale data, and gain the batch design skills that can be applied immediately in practice. 🚀

🍡 A sneak peek at the actual course content

Infrastructure objects working quietly in the background: JobRepository, JobLauncher, and ExecutionContext

DDL for WorkFlow managed by Spring Batch

Multi-Threaded Step that provides parallel processing while sharing a single Reader

Features of this course

📌 Concepts you must know to use Spring Batch, from the perspective of a Kakao interviewer

  • Instead of simply explaining the features, I will teach you how to design Spring Batch by focusing on the key points that are considered important in actual interviews and practical work.


📌 Curriculum composed of over 90% hands-on practice

  • Batch processing is difficult to understand through theory alone. This curriculum is designed so that you can experience firsthand how data is processed and where bottlenecks occur by implementing and executing it yourself.


📌 The core competency that separates juniors from seniors: "Data Processing"

  • This course provides an opportunity to grow from a developer who just makes APIs well, or has only ever made APIs, into a developer who can design data flows.


🤭 The career background of the Kakao interviewer who helped prepare this course

Choi (Kakao)

Click on the instructors' names to check their detailed profiles!

I am Choi (abbreviated name), a 12th-year backend server developer who works at Kakao as a server developer and also serves as an interviewer.

I first met Hong at a Conference, and since the mid-point of his teaching career, I have been actively participating and creating lectures on various topics together. I believe that communicating and interacting with many people while creating these lectures has been a great help in my life as a developer and a time to learn diverse perspectives, so I am striving to cover an even wider range of topics.

I don't believe that a single career milestone at a so-called "big tech" company proves that someone is a great developer. However, I do believe it allows for exposure to significantly more traffic and experiences compared to typical platforms. I will always strive to incorporate these insights into my lectures to share them with you.

[Current] Kakao Headquarters Server Developer

[Former] Computer Science major at a 4-year university in Seoul

Notes

Practice Environment

  • OS :

    Apple M3 Air

  • Java : java 17.0.12 2024-07-16 LTS

  • IDE : VsCode

I am running an open chat room to help you prepare for your career. We look forward to your interest!

Recommended for
these people

Who is this course right for?

  • A backend developer who knows that batch jobs are slow or frequently failing but doesn't know the exact cause.

  • Developers who have experienced service failures due to memory and performance issues when processing large-scale data

  • A developer who is using Spring Batch but is implementing it inefficiently because they do not properly understand its structure.

  • Developers who feel overwhelmed by how to design reprocessing or recovery logic when a failure occurs

  • A developer who wants to grow beyond simple CRUD and develop backend capabilities focused on data processing

Hello
This is Hong

8,067

Learners

510

Reviews

145

Answers

4.7

Rating

27

Courses

Introduction

I started studying development after finding an interest in it while idling at home, and I am currently in charge of platform server development in Pangyo. I am continuing my activities as a knowledge sharer because I want to provide you with the methods I used to study, as well as the various problems and solutions you may encounter in practice.

 

These lectures are not created solely through my own knowledge. Every lecture is made together with others.

 

Instructor Experience

[Former] Blockchain developer related to Sandbox IP

[Former] Metaverse Backend Developer

[Current] A veteran server developer in Pangyo

 

Interview History

Other Inquiries

  • unduck2022@gmail.com

More

Curriculum

All

24 lectures ∙ (5hr 24min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

2 reviews

5.0

2 reviews

  • kyletes6021420님의 프로필 이미지
    kyletes6021420

    Reviews 2

    Average Rating 5.0

    5

    92% enrolled

    It was a great lecture that provided a glimpse into the interviewer's insightful perspectives. Even though I feel like I've been working in development for quite some time, this lecture made me realize there is still so much more to learn. Thank you for the great content.

    • jhong
      Instructor

      Hello letes ky, thank you for leaving such a great review!!

  • youngba8935643님의 프로필 이미지
    youngba8935643

    Reviews 10

    Average Rating 5.0

    5

    88% enrolled

    I really enjoyed the lecture. This was my first time learning about Spring Batch, and your explanation was so easy to follow that I could understand everything even as a beginner. It was very helpful!

    • jhong
      Instructor

      Hello Tenburger, thank you for leaving such a great review!!

Hong's other courses

Check out other courses by the instructor!

Similar courses

Explore other courses in the same field!

Limited time deal

$38.50

50%

$77.00