You will learn about Transformers inside out by directly implementing everything from Multi-Head Attention to the Original Transformer model, BERT, the Encoder-Decoder-based MarianMT translation model, and the Vision Transformer in code.
369 learners
Level Intermediate
Course period Unlimited
Reviews from Early Learners
5.0
cudada
You teach better than university professors.
5.0
김재준
I took this course blindly without completing the prerequisites first. It was difficult and challenging at first, but as I kept listening, it became interesting, and now I am going to take the prerequisite courses.
5.0
승훈
I am gaining a detailed understanding of Transformers thanks to the code and theoretical explanations.
Mastering Transformer's Self, Causal, and Cross Attention Mechanisms through Direct Implementation
Original Transformer model architecture learned by directly implementing Positional Encoding, Feed Forward, Encoder, Decoder, and more
Prerequisite knowledge for Transformer, including NLP foundations like tokenization and embedding, and RNN models.
Implement the BERT model from scratch and apply it to sentence classification training.
MarianMT model, an Encoder-Decoder translation model implemented directly
Understanding and Utilizing Hugging Face Dataset, Tokenizer, and DataCollator
Training of the Encoder-Decoder MarianMT Model and Greedy and Beam Search Inference
Implement Vision Transformer (ViT) from scratch and train an image classification model with custom data.
I have structured this course as a full Transformer curriculum where you understand the concepts by implementing them in code, covering everything from Multi-Head Attention—the core mechanism of Transformer—to the Original Transformer model, BERT, the Encoder-Decoder MarianMT translation model, and Vision Transformer.
From Multi-Head Attention, the core mechanism of Transformer, to the original Transformer model, BERT, and the encoder-decoder translation model MarianMT, by implementing them directly in code, you will learn about Transformer inside and out.
To understand Transformer, it is important to understand the foundations of NLP first.
Starting from tokenization and embedding, and RNN models prior to Transformer, all the way to Attention -> Transformer -> BERT -> MarianMT translation model -> Vision Transformer, in a single flow, the course is structured to enable step-by-step learning from a solid NLP foundation to core Transformer models.
We don't just focus on implementation. To ensure the core mechanisms that make up the Transformer are easy to understand and stick in your mind, I spent a lot of time brainstorming ideas and creating this course. This course, which leads from easy and detailed theoretical explanations to actual code implementation, will help your Transformer application skills improve by leaps and bounds.
We specifically cover elements encountered in actual research and practice, such as embedding, padding masking, various types of attention, loss calculation for padded labels, and dynamic padding, and provide the process for solving them.
Hugging Face is an indispensable library for utilizing Transformers. In this course, we use Hugging Face's Tokenizer, Dataset, DataCollator, and more to perform data processing for Transformer model training—such as data preprocessing, tokenization, dynamic padding, and conversion of label values and decoder input values—how easily and conveniently this can be handled, providing a detailed guide.
We provide a summary of the prerequisite knowledge for learning Transformers, including tokenization, embedding, RNN and Seq2Seq models, and the basics of Attention.
You can gain a clear understanding of core attention mechanisms such as Self Attention, Causal Attention, and Cross Attention, as well as key Transformer modules like Positional Encoding, Layer Normalization, Feed Forward, and Encoder/Decoder Layers through detailed theory and hands-on practice.
We provide detailed explanations of the features, advantages, and usage of Hugging Face's Dataset, Tokenizer, and DataCollator. Additionally, you will gain hands-on experience through various exercises and examples on how to effectively perform data pipeline processing for Transformer NLP models by combining these components.
You will learn BERT by directly implementing the key components of the BERT model. Furthermore, you can learn how to apply model training and inference for sentence classification using the BERT you implemented and various features provided by Hugging Face.
You will directly implement the MarianMT model, an Encoder-Decoder-based Korean-English translation model, learn various data preprocessing methods and techniques required for training Encoder-Decoder models, and learn how to implement and apply Auto-Regressive-based Greedy Search and Beam Search.
You will directly implement the Vision Transformer, which established the Transformer as a model comparable to CNNs in the vision domain, and train it using custom datasets. By implementing the key modules of the Vision Transformer yourself, you can easily learn and understand the features and mechanisms of ViT.
The practice environment is conducted using notebook kernels provided by Kaggle. After signing up for Kaggle and selecting the Code menu, you can use a P100 GPU for free for 30 hours per week in a Jupyter Notebook environment similar to Colab.
Who is this course right for?
Deep learning NLP beginners who want to build a solid foundation by implementing everything from tokenization to RNNs and Transformers directly in code.
Those who want to gain a deep understanding of the Transformer architecture by implementing its internal mechanisms directly, rather than simply using a library.
Those who want to understand the core mechanisms of Transformers more easily through a balanced approach to theory and practice.
Developers who want to build a solid foundation in Attention and Transformer mechanisms for AI service development.
For those who want high-quality, end-to-end practical project experience ranging from Transformer basics to text classification and translation models.
Need to know before starting?
Deep Learning CNN Complete Guide - PyTorch Version
Inflearn Verified
28,708
Learners
1,602
Reviews
4,092
Answers
4.9
Rating
15
Courses
(Former) Encore Consulting | (Former) Oracle Korea | Author of "Python Machine Learning Perfect Guide"
AI Freelance Consultant
All
145 lectures ∙ (28hr 9min)
Course Materials:
All
26 reviews
5.0
26 reviews
Reviews 9
∙
Average Rating 5.0
5
I have taken several online courses on Transformers, and this one is the most detailed. Thank you for always providing such thorough explanations.
Oh, thank you for the great review. Since the Transformer can be difficult for those encountering it for the first time, some parts of the explanation might have been a bit excessive. I will keep that in mind when creating the next lecture.
Reviews 7
∙
Average Rating 5.0
5
Thank you always for the great lectures!
You're very welcome ^^, I'm actually the one who's grateful to you for writing such a nice course review.
Reviews 8
∙
Average Rating 5.0
Reviews 2
∙
Average Rating 5.0
5
I am gaining a detailed understanding of Transformers thanks to the code and theoretical explanations.
I'm glad I could be of help. Thank you so much for the great course review.
Reviews 2
∙
Average Rating 4.0
Check out other courses by the instructor!
Explore other courses in the same field!
25% off for new members
$49.40
25%
$59.40
![[Revised Edition] The Complete Guide to Python Machine LearningCourse Thumbnail](https://cdn.inflearn.com/public/courses/324238/cover/7e380aa0-48ba-4ee7-a6b2-8da7900568d6/324238-eng.png?w=420)
![[Revised Edition] Deep Learning Computer Vision: The Complete GuideCourse Thumbnail](https://cdn.inflearn.com/public/courses/325035/cover/f4bbef4d-d9a9-4def-a3a5-738a9eed8245/325035-eng.jpg?w=420)
![[PyTorch] Learn NLP easily and quicklyCourse Thumbnail](https://cdn.inflearn.com/public/courses/325056/course_cover/b66025dd-43f5-4a96-8627-202b9ba9e038/pytorch-nlp-eng.png?w=420)
![[AICE] Data Analysis and AI Modeling with PythonCourse Thumbnail](https://cdn.inflearn.com/public/files/courses/340648/cover/01kg1qqxb50yahem3fa8znqr9p?w=420)
