inflearn logo

Essential Theory for AI Semiconductor Design — From Neurons to LLMs

To design AI semiconductors such as NPUs, transformer accelerators, and mini LLMs yourself, you need to understand theoretically why the computations inside them occur. This course distills and condenses into three parts only the essential theory required for hardware design, covering everything from a single neuron to CNNs, transformers, and LLMs. Rather than memorizing formulas, a professor with 30 years of experience in the semiconductor industry connects software theory with hardware implementation through an intuition for “why it works that way.”

(4.0) 1 reviews

29 learners

Level Beginner

Course period Unlimited

Deep Learning(DL)
Deep Learning(DL)
Machine Learning(ML)
Machine Learning(ML)
CNN
CNN
transformer
transformer
LLM
LLM
Deep Learning(DL)
Deep Learning(DL)
Machine Learning(ML)
Machine Learning(ML)
CNN
CNN
transformer
transformer
LLM
LLM

What you will gain after the course

  • The ability to intuitively understand how neurons, neural networks, CNNs, transformers, and LLMs work

  • Core theoretical foundations needed for AI hardware design, including learning, gradient descent, and quantization

  • An understanding at the conceptual level of how ChatGPT, GPT, and BERT are created

  • The big picture of how AI theory translates into actual semiconductor circuits (NPUs and accelerators)

  • A theoretical foundation for understanding and following hands-on AI accelerator and NPU design exercises thereafter

■ Why is this course necessary?

NPU, Transformer accelerators, mini LLMs—to design AI semiconductors like these yourself, you need to understand what computations take place inside them, why they take place, and what those computations cost on the chip. Simply following circuit diagrams will only get you so far.

This course distills the essential theoretical foundations into three concise parts. Starting with the multiplication of a single neuron and extending to CNNs, attention, transformers, and LLMs—we have selected only 'what is actually needed for hardware design' from the vast body of AI theory.

Therefore, each part explains the connection to semiconductor design.

Part 1 focuses on the fact that neural network multiplications and additions directly translate into power consumption on the chip,

Part 2 focuses on the fact that the matrix operations of attention call for a dedicated accelerator called a systolic array,

Part 3 concludes with the fact that a significant portion of power is spent not on computation, but on moving data.

This is not theory for academic depth, but theory for building AI semiconductors yourself.


■ Why This Course Is Different

Commercial AI theory courses stop at the perspectives of software and mathematics. This course is designed from the outset as 'theory for building in hardware.' It explains AI theory through the eyes of a semiconductor designer: how the multiplication and addition of neurons become power on a chip, why the matrix operations of transformers call for dedicated accelerators (NPUs), and why low-power AI semiconductors are the future. A current professor with 30 years of experience in the semiconductor industry connects software theory and hardware implementation as one.


■ This lecture is the starting point.

This lecture is the theoretical starting point for the hardware design lectures that follow.

· This lecture (theory) → NPU design → Transformer accelerator → mini LLM SoC

With the theory laid down here, you’ll be able to follow the later lectures while understanding why each circuit is designed this way. Conversely, if you look only at the circuits without the theory, you’ll end up memorizing them and won’t be able to apply what you learn.


■ Three-Part Structure

· Part 1: From Neurons to CNNs — Principles of learning (gradient descent), the perceptron and the XOR barrier, deep neural networks (DNNs), and convolutional neural networks (CNNs)

· Part 2: From RNNs to Transformers — sequential data and RNNs, word embeddings, attention mechanisms, and Transformers

· Part 3: From Transformer to LLM — pretraining and transfer learning, BERT and GPT, ChatGPT and RLHF, and low-power semiconductors

The three parts form one continuous story. Studying them in order will give you a complete picture, from neurons to LLMs.


■ Recommended for those who:

· Those who plan to design NPUs, AI accelerators, and AI SoCs themselves in the future (based on that theory)

· Those who want to learn the essential background theory before taking an AI hardware design course

· Those who want to understand AI from its principles, but are looking for the “essentials for building” rather than academic depth

· Those who have heard of terms like CNN, attention, transformers, and LLMs but do not understand how they work


■ This is the kind of course it is

· Carefully selected only the theory needed for hardware design—the essence of extensive AI theory

· Intuition over memorizing formulas — illustrating “why it works that way” through diagrams and analogies

· One seamless story that connects everything from a single neuron to an LLM

· A rare perspective that connects AI theory and semiconductor hardware

· Complete the course using only the videos, without separate hands-on exercises, boards, or installations.


■ Note

· This course focuses on theory. Hands-on practice for building an AI accelerator directly with an FPGA is covered in separate courses. The theory in this course provides the foundation for those hands-on courses.

· All content in this course was created by the instructor, and the narration (voice-over) was produced using TTS.

Recommended for
these people

Who is this course right for?

  • Those who plan to design NPUs, AI accelerators, and AI SoCs themselves but lack a theoretical foundation.

  • I want to solidify the essential background theory before taking an AI hardware design course.

  • Those who know the terms CNN, attention, transformer, and LLM but don’t understand how they work.

  • Those who want to understand AI from its principles, but are looking for the “essentials for building” rather than academic depth

Need to know before starting?

  • Explained intuitively so that it can be understood even without specialized knowledge of the players.

  • Knowing high school-level mathematics (functions and graphs) will make it easier.

  • No coding or hands-on practice is required (theory section)

Hello
This is EdgeChipLab

Career Verified

Design and implement AI semiconductors directly on FPGAs

As a current university professor, I bring 30 years of experience in the system semiconductor industry, along with expertise in AI semiconductor research and education.

Starting as a system semiconductor researcher in Samsung Electronics’ DS Division, I received the Silver Award (Group Chairman’s Award) in the field of design automation. I later served as Director of Samsung’s UK and German subsidiaries and as Head of the System LSI Marketing and Sales Group, and I currently teach semiconductor engineering. Based on that experience, this curriculum is a hands-on course in which you design systems yourself and verify them on an FPGA.

This is not simply a course on how to use FPGAs.
It is designed to help you understand AI algorithms → directly design NPUs and CPUs in RTL → implement them on an FPGA → verify them on actual hardware → and ultimately extend them to an AI SoC.

AI Algorithm → RTL Design → FPGA Implementation → Bit-True Verification → AI SoC

 

Beyond theory to real-world AI hardware implementation

With only scattered, fragmented knowledge, it is difficult to develop system-level AI semiconductor design capabilities.

Learners do not stop at using completed black-box IP. They progressively develop the system design skills needed to design an AI NPU and RISC-V CPU themselves, integrate memory and peripherals, port an RTOS on top of them, and ultimately build their own AI SoC.

Starting with the fundamentals of transistors and digital circuits, this course connects FPGA, CPU, and NPU technologies to hardware for accelerating Transformers and LLMs.

 

We validate the research results on actual FPGA hardware.

This is not a lecture you only watch. Rather than stopping at theory or simple simulations, you will run AI semiconductor circuits you designed yourself on an actual FPGA and perform Bit-True verification, confirming that the Python model and RTL results match down to the last bit.

We also connect the instructor’s research results, including four KCI-indexed papers on AI semiconductors, FPGAs, and RISC-V, with the educational content, applying technologies validated through research to actual design and FPGA implementation.

We provide source code essential for hands-on practice, including our self-developed RISC-V CPU, Python code for image training, and Verilog RTL.

Centered around the affordable Arty S7-25 FPGA board and the free Vivado design tool, this course is designed to allow learners to reproduce the entire process of designing, implementing, and verifying their own work.

It can also be used with Digilent FPGA boards, including the Zybo Z7. You just need to modify the XDC pin mapping (board) and clock division parameters (RTL) to match the board.

The “MNIST NPU with FPGA” course only requires modifying the XDC pin mapping, while the courses using the OLED and Camera require modifying the clock division parameters (RTL).

→The course exercises are configured with clock division parameters based on Arty S7’s 100 MHz input clock. Since other boards have different input clocks (for example, Zybo Z7 has 125 MHz), you will need to recalculate the division values so that the target frequency remains the same. If you miss this point, the display or video may not output properly, which can easily be mistaken for a code error, so please check this.

 

A seamless full-stack roadmap: from the basics to a mini LLM accelerator

Every lecture is not an isolated fragment, but part of a journey toward completing one vast system.

  • Step 1: AI Theory and Image Processing Fundamentals (including Machine Learning)

  • Step 2: AI Accelerator (NPU) Design and Verification (Including Machine Learning)

  • Step 3: RISC-V CPU Design and System Integration

  • Step 4: Advanced AI SoC Implementation and Expansion into a mini LLM Acceleration Platform (Continuous Content Updates)

     

 

Objective verification metrics

All design deliverables from this course have undergone rigorous validation against global standards and by the academic community.

  • RISC-V Architecture Verification: In-house developed RISC-V CPU, passed the International Foundation’s official compliance test (ACT), and source code released (GitHub)

  • Academic Authority: Four Sole-Author Academic Papers (published in the prestigious KCI A-ranked journal IJIBC)

  • Global Recognition: Published four global Amazon technology books (reached #1 bestseller status)


    Published books: Roger Kim: books, biography, latest update

  • Real-World Operation Verification Complete: Core IPs, including the RV32I CPU, NPU, vision system, GPS, Transformer, and AURA-Edge SoC, operate flawlessly in the Arty S7 environment.

     

Take on the challenge. By the time you complete this process of uploading code to the board yourself and checking the results, you will have leveled up into a hardware engineer with a completely different perspective than before.

More

Curriculum

All

3 lectures ∙ (1hr 40min)

Published: 
Last updated: 

Reviews

All

1 reviews

4.0

1 reviews

  • mdrifatnahid님의 프로필 이미지
    mdrifatnahid

    Reviews 1

    ∙

    Average Rating 4.0

    4

    100% enrolled

    As it is titled Semiconductor, there is almost nothing about it. But Good for starting very basic AI fundamentals learning.

    • estlit
      Instructor

      Thank you for taking the time to write, and for the kind words about the fundamentals — I am glad that part was useful. On the title, let me share what the course is trying to do, in case it is helpful. This is the theory volume of a hardware series. The later courses build a RISC-V CPU, a CNN accelerator, an NPU and eventually a mini LLM SoC on a real FPGA board — a Transformer engine has already been implemented and verified on FPGA in this series. This volume exists because those circuits are difficult to design from diagrams alone. You have to know what operation is happening inside them, and why it costs what it costs. So the AI theory here is selected and framed for that purpose. A few examples: The multiply-accumulate as a power problem. Part 1 closes on the point that a neural network is fundamentally a vast repetition of multiplies and adds, and that each one of those operations draws power on a real chip. That single observation is what the whole low-power discussion later rests on. Convolution as a hardware pattern. The convolution lecture describes a small filter sliding across the image, multiplying and summing overlapping regions. That sliding-window structure — the same weights reused across positions — is exactly the access pattern an accelerator is built to exploit. Why the Transformer calls for a dedicated accelerator. Part 2 explains that removing recurrence makes every word computable in parallel, and that the substance of that parallelism is large matrix multiplication. It then names what that becomes in silicon: laying those multiply-adds out in a grid and running them together is a systolic array, which is the core structure of an NPU. Why data movement dominates the energy budget. Part 3 makes the point that a large share of AI power is spent not on arithmetic itself but on moving data. That is the constraint every low-power AI chip is designed around, and it is why the series ends on minimising data movement and lightening the model as the two directions that have to advance together. Each of these sections closes with the same line in the lecture: this is the bridge to hardware. The orientation slide at the start states it directly — every abstract concept in this course becomes a real operation on silicon later in the series, which is why the theory is built in an implementable form rather than a purely mathematical one. Your comment also helps me see the description from a reader side. The note about where this volume sits in the series currently sets at the bottom of the page — I will bring it up front, so it is clear before anyone enrols. Thank you very much. EdgeChipLab

EdgeChipLab's other courses

Check out other courses by the instructor!

Similar courses

Explore other courses in the same field!

25% off for new members

$7.70

25%

$7.70