inflearn logo

First Steps in System Design with LLMs: Designing Architectures for Operable AI Services (RAG·AI Agents)

How does the architecture change when an LLM is integrated into an existing backend? Going beyond simple LLM API calls, this introductory system design course helps you understand the core components of AI services—such as Model Routers, LLM Gateways, and Vector DBs—and progressively design operational AI service architectures, including RAG and AI Agents. It helps those who want to build their own LLM-based AI services quickly learn the design process.

(4.3) 6 reviews

86 learners

Level Basic

Course period Unlimited

Architecture
Architecture
software-design
software-design
system-design
system-design
LLM
LLM
RAG
RAG
Architecture
Architecture
software-design
software-design
system-design
system-design
LLM
LLM
RAG
RAG

What you will gain after the course

  • System architecture design skills based on LLMs, RAG, and AI Agents

  • Understanding the core components of LLM services (Model Router, LLM Gateway, vector DB, Retriever, etc.)

  • Understanding design trade-offs considering the performance, scalability, and reliability of AI services

A starting point for growing into an AI engineer who can design AI services ready for production 🚀


"I want to build a service using an LLM—how should I design it? 🤔"


With the emergence of LLMs, we have entered an era where anyone can apply AI functionality to their services.

Now, you can quickly build a variety of AI services simply by connecting LLM APIs such as ChatGPT and Claude.

However, once you move beyond simply calling an LLM API and start building AI services that can actually be deployed and operated, new challenges arise.

  • Which model should I choose? How should I route between multiple models?


  • How should conversation history and context be managed?

  • When is RAG needed, and how should it be designed?

  • What components make up an AI agent, and what should you consider when designing one?

  • How can we improve cost, response speed, and quality?


As LLMs become a new system component, the system architecture of AI services is also evolving into a form different from that of traditional backend systems.

This course is a system design course where you learn how to design services using LLMs as stable, scalable, and operable systems.

You will understand the roles of key components such as the Model Router, LLM Gateway, Conversation Manager, and Vector DB, explore RAG and AI Agent architectures, and learn the overall design process for operational AI systems.

This course is not about developing AI models yourself or learning how to use a specific AI framework. Instead, it focuses on learning how to design AI systems from the perspective of an engineer who applies AI to real-world services and operates them.

AI models and frameworks are evolving rapidly and constantly changing, but the principles of designing good systems do not change easily.

Through this course, let’s learn the fundamental principles of AI service system design and develop the judgment to independently design and evaluate AI system architectures suited to each situation, even as new technologies emerge.

#LLM #AISystem #System Design #RAG #AI Agent


🌟 What Sets This Course Apart


📌 1. Design systems from an AI engineer’s perspective.

This course is not about learning how to write effective prompts or use a specific AI framework.

Learn how to design production-ready AI system architectures from the perspective of a backend or AI engineer building real-world services.


📌 2. Connect all the key components required in a production environment.

Rather than learning the core components used in real-world AI services—such as a Model Router, LLM Gateway, Conversation Manager, Vector DB, RAG, and AI Agent—separately, you will understand from an architectural perspective how they connect and work together within a single system.


📌 3. Learn design principles that remain relevant even as technology changes.

AI models and frameworks continue to change, but the principles of designing good systems do not change easily.

In this course, rather than focusing on how to use specific technologies, we focus on developing the mindset to independently design and evaluate AI system architectures suited to each situation, even as new technologies emerge.

✏ Learn the flow of designing LLM service systems

Section 1. Why should you learn system design using LLMs?

  • Learn about the changing AI service development landscape brought about by the emergence of LLMs, and why you need to understand them from a system design perspective rather than simply use them.

  • We will examine the changing role of developers following advances in AI and learn how design approaches and the factors to consider are changing as LLMs are added as a new component to existing system architectures.



Section 2. Understanding LLM System Design

  • Learn through various use cases what services you can build using LLMs as a new system component.

  • Understand how the core goals of traditional backend system design—reliability, availability, performance, and scalability—differ in LLM-based systems, and explore additional considerations unique to LLM services.

  • You will also understand how LLMs have evolved and the key technological developments centered around the Transformer architecture, building the foundational knowledge needed to design LLM-based systems.

Section 3. Basic Architecture for Using LLMs

  • Explore the most basic service architecture that leverages LLMs and understand the components needed to build an operable AI service beyond simple LLM API calls.

  • Learn why the key components of LLM-based services—prompt engineering, model selection and serving strategies, LLM Router & Gateway, and Conversation Manager for managing conversation history—are necessary and when to use them.



Section 4. RAG Architecture

  • Going one step beyond the basic architecture for using LLMs, you will understand the RAG architecture for reducing LLM hallucinations and leveraging a service’s own data.

  • Learn how embeddings and vector databases work, and explore the entire data pipeline of a RAG system, from document collection through preprocessing, chunking, indexing, and retrieval.

  • We also learn RAG design approaches for production environments, including incremental indexing to efficiently process large documents, and Hybrid Retriever and Reranker to improve search quality.

Section 5. AI Agent Architecture (Coming in November)

  • Understand the structure of AI Agent systems that go beyond RAG, where LLMs make decisions on their own, choose the tools they need, and perform multiple tasks.

  • Explore the roles of Planning, Tool Calling, Memory, and Agent Runtime, the core components of an Agent.

  • Explore application architectures, including Agentic RAG architectures that combine RAG and Agents.



Section 6. Operating and Optimizing LLM-Based AI Services (Available in December)

  • Learn about monitoring, security, evaluation, and performance improvement methods needed to provide stable LLM services in real-world production environments.

  • Learn how to measure and improve the quality and cost of LLM services, and understand the key elements needed during the operational phase, including Prompt/Response Logging, Evaluation, and Guardrails.

  • Learn how to design systems to optimize response speed and costs, and explore the process of completing an operational AI service.

🙋 Recommended for

I want to build a service using an LLM, but I’m not sure where to start with the design

Anyone who wants to learn the overall architecture and design process for operational AI services beyond simply calling an LLM API.

I want to apply LLMs to company projects or work.


Developers who want to design AI system architectures that can be applied in real-world settings, such as automating internal tasks, adding AI features, and developing AI services

I want to grow into an AI engineer or AI service developer.


Those who want to develop the mindset to design AI services independently while considering performance, cost, quality, and scalability

Notes Before Taking the Course

📚 Learning Materials

  • A PDF of the lecture slides is provided. (Approximately 220 pages (+more to be added))

📖 Prerequisite Knowledge and Important Notes

  • This is a theory-focused course on designing system architectures for LLM-based AI services.

    Therefore, we do not cover how to use specific frameworks such as LangChain, LlamaIndex, and Spring AI, or hands-on coding practice.

  • If you have basic backend development experience, you will be able to follow along more easily, and knowing the following will help with your learning.

    • The basic operating principles of HTTP and REST APIs

    • Development experience using databases (RDB or NoSQL)

    • Experience with web services or backend development

    • Experience using LLMs such as ChatGPT and Claude or making simple calls to LLM APIs

  • No specialized knowledge of LLMs or experience developing AI models is required. Since we explain everything step by step, from the basic concepts of LLMs to how AI services are structured and operate, even developers with limited AI experience can understand and follow the overall flow of AI system design.


Recommended for
these people

Who is this course right for?

  • A developer who wants to design a stable architecture for LLM-based AI services firsthand

  • A job seeker who wants to work on an LLM-based project

  • Developers who want to apply LLMs to their work, such as automating tasks or improving internal company platforms

  • A backend developer who needs to add an LLM to an existing backend system

Need to know before starting?

  • Basic understanding of backend architecture (fundamental operating principles of the backend, including HTTP, RESTful APIs, web servers, databases, etc.)

  • Basic interest in and experience with LLMs (It would be preferable if you have called an LLM API such as ChatGPT or Claude at least once, or have experience writing prompts.)

Hello
This is mindlantern

Career Verified

849

Learners

63

Reviews

9

Answers

4.9

Rating

2

Courses

Hello, I’m developer Growth Lantern.
I currently work at a major IT company in Korea and dream of a world where anyone can become a developer and build whatever they imagine.

I want to share the challenges faced in the field and the technologies actually used in practice so that even those learning them for the first time can easily understand them.
As someone who enjoys learning new things and growing, I hope that those taking my courses can also experience the joy of growing together.

 

Email: mindlantern.studio@gmail.com

Linktree: https://linktr.ee/mindlantern

More

Curriculum

All

25 lectures ∙ (5hr 24min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

6 reviews

4.3

6 reviews

  • zndcvkln1372님의 프로필 이미지
    zndcvkln1372

    Reviews 1

    ∙

    Average Rating 5.0

    5

    32% enrolled

    • seokjunwon7426님의 프로필 이미지
      seokjunwon7426

      Reviews 3

      ∙

      Average Rating 4.0

      4

      60% enrolled

      I think this will be very helpful for using AI.

      • mindlantern
        Instructor

        Thank you for leaving a course review, Seokjun! We’ll do our best to make the upcoming Sections 5 and 6 even more helpful for AI-assisted design~

    • coen님의 프로필 이미지
      coen

      Reviews 84

      ∙

      Average Rating 5.0

      5

      32% enrolled

      • suylee4596님의 프로필 이미지
        suylee4596

        Reviews 1

        ∙

        Average Rating 5.0

        5

        32% enrolled

        • mindlantern
          Instructor

          Thank you for leaving such a great course review, Suyeon 😊 We’re working hard to fill Sections 5 and 6 with helpful content, so please wait just a little longer! If you have any questions or anything you’re curious about while taking the course, please feel free to leave a question anytime~!

      • twlee4117님의 프로필 이미지
        twlee4117

        Reviews 5

        ∙

        Average Rating 3.8

        4

        32% enrolled

        Similar courses

        Explore other courses in the same field!

        25% off for new members

        $46.00

        25%

        $66.00