inflearn logo

Spring AI Multi-LLM Architecture and Orchestration-focused Agent System

This course is an advanced program focused on designing Multi-LLM architectures and orchestration-centered Agent systems by strategically combining GPT, Gemini, and LLaMA based on Spring AI. Moving beyond simple LLM calls, you will implement scalable, stable, and continuously improving AI systems by applying Workflow Patterns and Multi-Agent structures. Furthermore, the course covers Circuit Breakers, Reactive Streams, Redis monitoring, parallel processing, and iterative evaluation loops to complete your capability in designing production-grade AI architectures. The ultimate goal is to grow into a developer who can design AI systems, rather than just being an AI user. 🚀

18 learners are taking this course

Level Basic

Course period Unlimited

Spring Boot
Spring Boot
gemini
gemini
ChatGPT
ChatGPT
Llama
Llama
Spring AI
Spring AI
Spring Boot
Spring Boot
gemini
gemini
ChatGPT
ChatGPT
Llama
Llama
Spring AI
Spring AI

What you will gain after the course

  • Ability to design Multi-LLM based AI architectures

  • How to apply Agentic Workflow patterns in practice

  • Experience in implementing Orchestrated Multi-Agent systems

  • Parallel processing + Exception handling + Disaster recovery design skills

  • AI Advancement Strategy Based on Quality Evaluation Loops

  • Understanding the backend architecture of AI services at a professional level

Spring AI Multi-LLM Architecture and Orchestration-focused Agent System

This course is not simply about learning how to use LLMs.

This is a course for building AI system design capabilities that can be applied to actual services.

This course consists of the following three-step structure.

Multi-LLM Strategy Design → Workflow Pattern Application → Multi-Agent Orchestration Completion


1️⃣ Multi-LLM Architecture Design

Instead of relying on a single LLM, we design a structure that strategically combines GPT, Gemini, and LLaMA.

In this stage:

  • Analysis of model characteristics and separation of roles

  • Designing a Router based on request types

  • CircuitBreaker + Fallback fault tolerance architecture

  • Real-time response processing based on Reactive Streams

  • Sensitive Information Branching (Security Routing)


👉 The goal is “not just to use the model well, but to design an architecture that strategically selects the model.”


2️⃣ Applying Agentic Workflow Patterns

Expand the LLM from a simple API into a collaborative Agent structure.

The key here is the "pattern."

  • Chain Workflow → Step-by-step processing structure

  • Parallelization Workflow → Parallel analysis structure

  • Routing Workflow → Dynamic model selection

  • Orchestrator–Workers → Role-separated collaborative structure

  • Evaluator–Optimizer → Iterative Improvement Loop

At this stage:

✔ Parallel Processing (CompletableFuture)
✔ Caching Strategy
✔ Exception Handling
✔ Logging Structure
✔ Automated Quality Improvement Loop

👉 The goal is to acquire the "ability to design AI collaboration structures" rather than just "calling an AI."


3️⃣ Implementing Orchestrated Multi-Agent Systems

By integrating all the patterns learned previously, we will implement a complete Multi-Agent orchestration system.

It is not just about learning simple patterns, but about completing a complex, interconnected structure at a production-ready service level.

Example systems to be implemented:

  • Automated Strategy Analysis System

  • Idea → Automated Execution Output Generation System

  • Majority Vote-Based AI Decision System

At this stage:

✔ Redis-based real-time status monitoring
✔ Parallel processing + ThreadPool design
✔ Step-by-step exception safety handling
✔ Iterative quality improvement loop
✔ Orchestration-based flow control

👉 The goal is not just “implementing AI features,” but completing the “ability to design and control AI systems.”


💡Core Content of This Course

  • Designing Multi-LLM Routing Strategies

  • Role separation based on model characteristics

  • Implementation of Chain / Parallel / Routing Patterns

  • Orchestrator-Workers structure design

  • Evaluator-Optimizer iterative improvement loop

  • Circuit Breaker + Fallback Structure

  • Real-time Status Monitoring Architecture

  • Implementation of strategy analysis, execution automation, and majority vote decision systems

💡Key Features of This Course

Implementation-focused rather than theoretical
Multi-Model strategy design instead of a single model
Production-level architecture considering fault tolerance
AI quality improvement design based on iterative evaluation
✔ Redis-based real-time status monitoring

💡Learning Content

1⃣ Setting up Spring AI development environment and Multi LLM environment in SpringBoot

2⃣ Chapter 1. Multi LLM Architecture 

3⃣ Chapter 2. Agentic Workflow Patterns

4⃣ Chapter 3. Orchestrated Multi-Agent Systems

What you will learn


1⃣ Building Spring AI Development and Multi LLM Environments in SpringBoot


Before designing the AI architecture, we will complete a professional-level Spring AI development environment.

The class is conducted based on practice-oriented projects, rather than simply adding libraries.

✔ Teaching materials included within the practice project
✔ Check code and theory simultaneously
✔ Provides a ready-to-run structure


📌 Learning Content

  1. Explanation of the development environment and overall course structure

  2. Creating the practice project and setting up the teaching materials

  3. OpenAI API Key issuance and integration

  4. Gemini API Key issuance and integration

  5. Ollama environment setup and Llama 3.2 setup

  6. Configuring the Spring AI environment and understanding the basic structure

  7. Redis integration settings




2⃣Chapter 1. Multi LLM Architecture 

Design a strategic Multi-LLM architecture, rather than relying on a single model.


1. Multi LLM

This is a structure that simultaneously uses three LLMs: GPT, Gemini, and LLaMA.

We implement a Multi-LLM-based service architecture that selects the appropriate model based on the request type and strategy.

✔ Strategy design based on model characteristics
✔ Implementation of model separation structure at the service level


2. Multi LLM Stream

Streaming processing based on Reactive Streams (Project Reactor)

✔ Real-time token-based response
✔ Asynchronous-based structure
✔ Improved user experience


3. Fallback (Fault Tolerance Architecture)

LLaMA(Ollama) + GPT Dual Architecture Design

✔ Resilience4j CircuitBreaker applied
✔ Automatic GPT Fallback upon failure
✔ Ensuring Production Level stability


4. Router (Model Strategy Classification)

Classifies user requests to select a model.

CREATIVE(Creative Writing / Reasoning / Coding) Use GPT

TECHNICAL (Technical Analysis / Data Processing) Use Gemini

GENERAL (Simple Q&A) Use LLaMA

UNKNOWN (Classification failure) Use GPT

✔ Design of a strategy-based model selection structure


5. Security (Sensitive Information Protection Architecture)

It detects sensitive information within the input data and branches it to either a Local LLM or an external LLM.

✔ Phone number
✔ Email
✔ Resident registration number
✔ Confidential keywords



3⃣Chapter 2. Agentic Workflow Patterns

Expand LLM from a "tool" into a collaborative Agent structure.


1. Chain Workflow

Implement a structure that connects tasks sequentially, passing the results of the previous step as input for the next step.

  • Step1 (Summary) → Gemini (Fast Processing)

  • Step2 (Deep Analysis) → GPT (Accurate Reasoning)

  • Step3 (Security Review) → Llama (Local Execution)

✔ Step-by-step processing structure
✔ Role-based model separation


2. Parallelization Workflow

Implement a structure that executes the same input across multiple LLMs in parallel and merges the results.

Technical Analysis → LLaMA
Market Analysis → GPT
Global Analysis → Gemini
Final Report → GPT

✔ CompletableFuture parallel processing
✔ Timeout + exception handling
✔ Practical structure including logging


3. Routing Workflow

Implement a structure that branches to the optimal LLM based on the input conditions.

  1. Check cache

  2. LLaMA Classification (SIMPLE / COMPLEX / SEARCH)

  3. Routing table-based model selection

  4. Model Call

  5. Result Caching

✔ Cost Optimization
✔ Performance Optimization


4. Orchestrator–Workers Structure

Implement a structure where a central Orchestrator controls GPT, Gemini, and Llama as Workers.

GPT → Orchestrator
LLaMA / Gemini → Worker

  1. Planning (GPT)

  2. Parallel Execution

  3. Aggregation

✔ Collaborative Agent Structure
✔ Parallel Execution
✔ Step-by-step Log Output


5. Evaluator–Optimizer (Quality Improvement Loop)

Implement an iterative structure that evaluates and improves the generation results.

GPT → Draft generation
Gemini → Quality evaluation
Iterative improvement if the score is below the standard

✔ Iterative improvement structure
✔ Automated quality optimization system



4⃣Chapter 3. Orchestrated Multi-Agent Systems

By integrating all previous patterns, we will implement a complete Multi-Agent system.


1. Strategic Analysis

We implement an iterative improvement-based analysis service to maximize strategic quality.

Multi-model based strategic analysis

Architecture Flow
Routing → Parallel Analysis → Evaluator-Optimizer Loop → Chain Orchestrator-Workers → Final Output

✔ CompletableFuture parallel processing
✔ Redis-based real-time status monitoring
✔ Iterative quality improvement loop

2. Precision Execution

We implement execution-oriented services that transform ideas into actual deliverables (code/documents).

Architecture Flow
Refinement → Orchestrator → Parallel Workers → Aggregation

✔ Redis-based real-time status monitoring
✔ Dedicated ThreadPool
✔ Secure exception handling

3. Voting Analysis

We implement a consensus-based analysis service to eliminate model bias and improve reliability.

Parallel invocation of GPT, Gemini, and LLaMA → GPT performs the role of the final judge

Architecture Flow
Parallel Invocation → Response Collection → Judge Decision

✔ Parallel processing structure
✔ Redis Pub/Sub status publishing
✔ Executor graceful shutdown



💡Key concepts you will learn in this course

✔ Multi-LLM Strategy Design
✔ Agentic Workflow Implementation
✔ Orchestration-based Collaborative Structure
✔ Fault Tolerance + Reliability Design
✔ Quality Improvement System based on Iterative Evaluation

💡Final Goal

  • This course is not simply about learning how to use LLMs.


    Multi-LLM Strategy Design → Workflow Pattern Application → Multi-Agent Orchestration Completion

    This is a course designed to build AI system design capabilities that can be applied to actual services.

Notes before taking the course

Practice Environment

  • Operating System and Version (OS): All OS types including Windows, macOS, and Linux are supported.

  • Tools used: JDK, Intellij(Ultimate or Community), Redis& Docker, Node.js, VSCode

  • PC Specifications: A basic specification PC with internet access

Learning Materials

  • Format of provided learning materials: Various types of materials provided, including educational environment projects and teaching plans.

  • Quantity and Capacity: Learning materials provided for each section

Prerequisite Knowledge and Precautions

  • Those with basic knowledge of HTML, CSS, and JavaScript.

  • Those with experience in Java and SpringBoot development

  • Please post your questions on the bulletin board, and I will make sure to respond as quickly as possible.


  • The copyright of this lecture belongs to Tonesol Co., Ltd., and unauthorized distribution or reproduction is prohibited. The learning materials are also copyrighted and may not be used for any purpose other than personal study.

This template is provided with reference to the lecture 'Artificial Intelligence and Autonomous Driving with AWS DeepRacer'.

Recommended for
these people

Who is this course right for?

  • ✔ Multi-LLM Strategy Planner

  • ✔ Agentic Workflow Architect

  • ✔ Orchestration-based Multi-Agent System Implementation Developer

  • ✔ A Production-Level architect who designs with incident response and quality improvement in mind

Need to know before starting?

  • Basic knowledge of HTML, CSS, and JavaScript

  • Experienced in Java and SpringBoot system development

Hello
This is tootoo

107

Learners

9

Reviews

3

Answers

5.0

Rating

2

Courses

Hello

I am Jinman Lee (nickname tootoo), a knowledge sharer.

I have communicated with you in the classroom for a long time.

I will now create great content so that I can communicate with you online.

Thank you.

More

Curriculum

All

22 lectures ∙ (6hr 56min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

Not enough reviews.
Please write a valuable review that helps everyone!

Similar courses

Explore other courses in the same field!

Limited time deal

$357,493.00

40%

$22.00