Calculus for AI — Derivatives, Gradients, and How Neural Networks Learn
This course is for those who want to understand the calculus and backpropagation formulas behind deep learning. Starting with functions and limits, you will learn derivatives, the chain rule, gradients, and gradient descent in sequence, then manually calculate the forward propagation, loss calculation, backpropagation, and weight updates of a small neural network. Explore the concepts through animations without coding or installation, and connect the formulas to the AI learning process.
You can understand functions, limits, derivatives, and the chain rule through graphs and equations, and explain their relationship to neural networks.
You can calculate the gradient of the loss function and determine in which direction and by how much to adjust the weights.
You can manually calculate the forward pass, loss calculation, backpropagation, mini-batch average, and weight update of a small neural network.
You can compare the principles of gradient descent, momentum, and Adam, and explain how the learning rate and activation functions affect training.
Mathematics for AI · Calculus
Understanding How Neural Networks Learn, Through Calculus
Do derivatives and backpropagation formulas in deep learning feel unfamiliar? Starting with functions and limits, we will steadily explore the process by which a small neural network learns by changing its weights.
This course helps you understand concepts through animated visuals, organize them with formulas, and connect them to AI learning by calculating them yourself.
18 sections · 144 lessons3 comprehensive projectswithout installing a coding or development environment
It begins with questions like these.
Why do the weights change in that direction?
Frameworks automatically calculate gradients. But when training is slow or the loss does not decrease, you need to understand what those calculations mean in order to assess the learning rate, activation functions, and loss function.
If we change this value slightly, how much will the error change?
Differentiation and gradients answer this question, and gradient descent uses that answer to adjust the parameters.
How You Learn
From diagrams to formulas, from formulas to real calculations
01
Understanding Through Illustrations
Explore limits, tangent lines, and gradients through animations, then learn the formulas and notation.
02
Connecting to AI
See how the chain rule is used in backpropagation and how gradients are used to update weights.
03
Calculate Directly
From a single neuron to a small neural network, apply the concepts you’ve learned together in three projects.
What You’ll Be Able to Do After Taking the Course
So you can explain the calculations that used to run automatically yourself
Interpret functions, derivatives, and the chain rule using graphs and equations
Find the gradient of the loss function and update the weights
Calculate the training process of a small neural network, from forward propagation to backpropagation.
Comparing how learning changes according to the learning rate, batch size, and activation function
Distinguishing how Momentum, RMSProp, and Adam work
The Full Learning Process
From the basics of functions to the entire process of training neural networks
Learn the basics → differentiate neurons → reduce loss → train neural networks. Follow the 18 sections in order and build the necessary concepts.
SECTION 01
Course Introduction and Learning Guide
We’ll examine the entire flow from functions to neural network training. Across three projects, we’ll explain what to calculate and in what order it’s best to study.
SECTION 02
Starting Over with Functions
We review functions and graphs, the slope of a line, and exponents and logarithms. By expressing neural networks with composite functions, you build the foundation for learning differentiation.
SECTION 03
Understanding Change Through Limits
We explore limits using numerical values and graphs, and learn about continuity, left- and right-hand limits, and asymptotes. We examine how the slope between two points leads to the slope of a tangent line.
SECTION 04
Derivatives and Instantaneous Rates of Change
We move from the average rate of change to the definition of the derivative. We learn to read graphs of tangents and derivatives, examine cases where differentiation is not possible, and explore the direction of change indicated by the derivative.
SECTION 05
Differentiation Rules and the Chain Rule
Learn how to differentiate polynomial functions, products and quotients, and composite functions. You’ll also learn about higher-order derivatives and function approximation, and see how the chain rule is used in backpropagation.
SECTION 06
Exponential/Logarithmic Functions and Activation Functions
Starting with the natural constant e and the derivatives of exponential and logarithmic functions, we examine sigmoid, tanh, ReLU, and softmax. We compare how the slopes of activation functions affect learning.
SECTION 07
Project 1 · Differentiating a Single Neuron
Express a single neuron using a weighted sum, bias, and activation function. Break the calculation down step by step and apply the chain rule to determine how the weights and bias affect the output.
SECTION 08
Loss Functions and Gradients
We compare MSE, MAE, cross-entropy, and Huber loss. We calculate the gradient of each loss and examine how the training process changes depending on the choice of loss function.
SECTION 09
Gradient Descent and Learning Rate
Learn the principle of updating weights in the direction that reduces the loss. Compare convergence, oscillation, and divergence depending on the learning rate, and understand why learning slows down in flat regions.
SECTION 10
Understanding Changes in Multiple Variables
Understand partial derivatives and gradient vectors through surfaces and contour lines. Learn directional derivatives, the Hessian, and the Jacobian, and extend this to differentiation as the number of parameters increases.
SECTION 11
Batch and Mini-Batch Learning
Calculate the process of combining and averaging the loss and gradients for each data point. Compare the differences between full-batch, mini-batch, and SGD, as well as the learning trajectories according to batch size.
SECTION 12
Project 2 · Following the Loss Surface
We calculate the loss surface for a model with two parameters and a small dataset. We compute the gradients and update the parameters repeatedly, then plot the learning path on the contour map.
SECTION 13
Computation Graphs and Backpropagation
After obtaining the predictions through forward propagation, we apply the chain rule to the computational graph. We calculate backward using local gradients and understand the principle of reusing intermediate results.
SECTION 14
Expanding to Deeper Neural Networks
We examine how gradients change as they pass through multiple layers. We cover vanishing and exploding gradients, the choice of activation functions, and the computational cost associated with depth and width.
SECTION 15
Optimization and Optimizers
Learn about local minima and saddle points, and the differences between convex and nonconvex functions. Compare Momentum, RMSProp, and Adam on the same loss surface, and examine the roles of learning-rate schedules and curvature.
SECTION 16
From Integration to Probability
Understand integration through the area under a curve and accumulated quantities. After learning the Fundamental Theorem of Calculus, connect it to probability density, expected values, and Monte Carlo integration.
SECTION 17
Final Project · Complete Neural Network Training Calculations
Define the architecture and initial weights of a small neural network, then calculate the forward pass, loss, backpropagation, mini-batch averaging, and updates in sequence. Repeat the training and observe changes in the loss curve.
SECTION 18
Summary of Key Concepts and Next Steps
We summarize the relationships among functions, derivatives, gradients, backpropagation, and optimizers. We then introduce probability and statistics for AI and outline the next steps for learning.
Pre-Course Information
Recommended for those who:
Those who want to understand the differentiation and backpropagation equations in deep learning lectures or papers
Those who have run models but are curious about how gradient calculations work
Those who want to study the calculus needed for AI after linear algebra
Those who want to understand concepts through diagrams and calculations before proofs
Prerequisites and Materials
Completion of “Linear Algebra for AI” or equivalent foundational knowledge It is helpful to be familiar with vectors, matrices, dot products, and linear transformations. We will also use equation solving, coordinate planes, and basic graph reading.
No prior calculus or programming experience is required. The course is conducted without a separate development environment, so please prepare paper and a pen to follow along with the calculations.
Scope of the lecture
We focus on the calculus concepts and calculations needed for AI learning. Epsilon-delta proofs, differentiation of trigonometric and implicit functions, substitution integration, integration by parts, and similar topics are not covered in this course.
Explained through animations and Korean subtitles, no separate source code or programming practice files are provided. In the three projects, we recommend following along with the calculations shown on the lecture screen.
Starting with functions, all the way to the calculations that enable learning.
See what role each calculus concept plays within a neural network through diagrams, formulas, and hands-on calculations.
Recommended for these people
Who is this course right for?
Those who want to understand the differentiation and backpropagation formulas found in deep learning lectures or papers
Those who have experience running models but are curious about how gradients are calculated within a framework
Those who want to continue studying AI after learning the basics of linear algebra, while mastering calculus through visualizations and calculations
Need to know before starting?
Completion of “Linear Algebra for AI” or equivalent foundational knowledge of vectors, matrices, inner products, and linear transformations
High school-level algebra, including solving equations, the coordinate plane, and reading simple graphs.
Prior study of calculus and programming experience are not required. The course is conducted without installing a separate development environment.