강의

멘토링

로드맵

NEW
AI Development

/

Natural Language Processing

Learning Transformer Through Implementation

From Multi Head Attention to the Original Transformer model, BERT, and Encoder-Decoder based MarianMT translation model, you'll learn Transformer inside and out by implementing them directly with code.

5 learners are taking this course

  • dooleyz3525
이론 실습 모두
transformer
NLP
바닥부터구현
Deep Learning(DL)
PyTorch
encoder-decoder
bert

What you will learn!

  • Hands-on Implementation and Mastery of Transformer's Self, Causal, Cross Attention Mechanisms

  • Learn the Original Transformer Model Architecture by Implementing Positional Encoding, Feed Forward, Encoder, Decoder, and More

  • Tokenization, embedding NLP foundations and RNN models - prerequisite knowledge for Transformers

  • Implementing BERT model directly and applying sentence classification training with the implemented BERT

  • MarianMT Model: A Directly Implemented Encoder-Decoder Translation Model

  • Understanding and Utilizing Hugging Face Dataset, Tokenizer, and DataCollator

  • Training Encoder-Decoder MarianMT Models and Greedy vs Beam Search Inference

Master Transformer completely with this one lecture!

Through this course, you can learn by directly implementing Transformer, the core of modern AI.

I've structured this course as a comprehensive Transformer full course that covers everything from Multi-Head Attention, the core mechanism of Transformers, to the Original Transformer model, BERT model, and Encoder-Decoder MarianMT translation model, all implemented and understood through code.

Features of this course

[[SPAN_1]]💡 [[/SPAN_1]][[SPAN_2]]바닥부터 코드로 구현하며 배우는 Transformer[[/SPAN_2]]

From Multi Head Attention, the core mechanism of Transformer, to the Original Transformer model and BERT, as well as the Encoder-Decoder translation model MarianMT, you will learn about Transformers inside and out by implementing them directly with code.

💡 Step-by-step learning from NLP fundamentals to core Transformer models

To understand Transformers, it's important to first understand the fundamentals of NLP.

Starting from tokenization and embedding, and RNN models before Transformer, progressing through Attention -> Transformer -> BERT -> MarianMT translation model, the course is structured in one continuous flow to enable step-by-step learning from solid NLP fundamentals to core Transformer models.

[[SPAN_1]]💡 [[/SPAN_1]][[SPAN_2]]이론과 구현의 균형[[/SPAN_2]]

We don't just focus on implementation. The core mechanisms that make up the Transformer are designed to be easily understood and stick in your head. I spent a lot of time conceptualizing ideas and creating this course. From easy and detailed theoretical explanations to actual code implementation, this course will dramatically improve your Transformer application skills.

💡 Presenting Core NLP Problem-Solving Process

We cover specific elements encountered in actual research/practice such as embedding, padding masking, various types of Attention, loss calculation for padded labels, dynamic padding, etc., and present the solution process for these.


💡 Utilizing Key Hugging Face Libraries

Hugging Face is an essential library for utilizing Transformers. In this course, we will use Hugging Face's Tokenizer, Dataset, DataCollator, and other tools to conveniently handle data processing for training Transformer models with large datasets, including data preprocessing, tokenization, dynamic padding, label value conversion, and decoder input value transformationin an easy and convenient way. We provide detailed guidance on how to handle these processes.

A Vision Transformer(ViT) section will be added to this course

ViT (Vision Transformer) lectures will be added to this course by the end of November 2025. After the ViT lectures are added, the price of this course will be slightly increased.

You'll learn this kind of content

NLP Fundamentals and RNN for Transformer Prerequisites

We will provide a summary explanation of the prerequisite knowledge needed to learn Transformers, including tokenization, embedding, RNN and Seq2Seq models, and the basics of Attention.

Transformer Core Mechanisms and Key Modules

You can clearly understand the core Attention mechanisms such as Self Attention, Causal Attention, Cross Attention, and the key modules of Transformers including Positional Encoding, Layer Normalization, Feed Forward, Encoder/Decoder Layer through detailed theory and hands-on practice.

Utilizing Hugging Face Tokenizer, Dataset, DataCollator

I will provide a detailed explanation of the features, advantages, and usage methods of Hugging Face's Dataset, Tokenizer, and DataCollator. Additionally, you will be able to master how to effectively perform data pipeline processing tasks for Transformer NLP models by combining these components through various hands-on exercises and examples.

Implementation and Application of BERT Models

Learn BERT by directly implementing the key components of the BERT model. Additionally, you can learn how to apply model training and inference for sentence classification using the BERT implemented this way and various features provided by Hugging Face.

Implementation and Application of MarianMT Translation Model Based on Encoder-Decoder

You will directly implement the MarianMT model, which is an Encoder-Decoder based Korean-English translation model, learn various data preprocessing methods and techniques necessary for training Encoder-Decoder models, and learn how to implement and apply Auto Regressive based Greedy Search and Beam Search.

Pre-enrollment Reference Information

Practice Environment 💾

The hands-on environment will be conducted using notebook kernels provided by Kaggle. After signing up for Kaggle and selecting the Code menu, you can use P100 GPU for 30 hours per week for free in a Jupyter Notebook environment similar to Colab.


A 140-page lecture textbook is provided together.

Recommended for
these people

Who is this course right for?

  • Deep learning NLP beginners who want to solidify their foundation by directly implementing everything from tokenization to RNN and Transformer with code

  • Someone who wants to deeply understand the Transformer architecture by directly implementing the internal mechanisms rather than simply using the Transformer library

  • Those who want to understand the core mechanisms of Transformers more easily through a balanced approach of theory and practice

  • Developers who want to solidly build foundational skills in Attention or Transformer when developing services

  • Those who want a complete End-to-End practical project experience from Transformer fundamentals to text classification and translation models

Need to know before starting?

  • Deep Learning CNN Complete Guide - PyTorch Version

Hello
This is

26,332

Learners

1,323

Reviews

3,983

Answers

4.9

Rating

14

Courses

(전) 엔코아 컨설팅

(전) 한국 오라클

AI 프리랜서 컨설턴트

파이썬 머신러닝 완벽 가이드 저자

Curriculum

All

123 lectures ∙ (24hr 16min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

Not enough reviews.
Please write a valuable review that helps everyone!

Limited time deal

$50,050.00

35%

$59.40

dooleyz3525's other courses

Check out other courses by the instructor!

Similar courses

Explore other courses in the same field!