Spark Machine Learning Complete Guide - Part 1

From the core of Kafka to in-depth content on internal mechanisms, the course is structured so that even beginners can quickly reach the expert level through detailed theoretical explanations, hands-on practice, and practical Kafka application development practice.

We will help you easily understand the core concepts of machine learning and acquire the ability to implement practical machine learning applications by moving away from theory-based machine learning courses.

This course is designed to help you learn the core functions of FastAPI, as well as the entire process of web service development. Through this course, we will help you become a FastAPI expert developer needed in the field.

This course will help you grow into a deep learning-based computer vision expert needed in the field through in-depth theoretical explanations of Object Detection and Segmentation, along with practical examples at a level that can be immediately applied in the industry.

Through in-depth theoretical explanations of Kafka Connect and detailed practical training that can be used immediately in the field, we will help you grow into an expert in building data linkage and data pipelines based on Kafka Connect that are needed in the field.

This course is a practical Airflow master program designed to help you understand "why Airflow works this way" and enable you to design and debug data pipelines on your own. It covers everything from Apache Airflow's core mechanisms to detailed theory and hands-on practice involving DAGs, Operators, Hooks, Scheduling, Timezones, Idempotency, and Templates.

This course is designed to help you learn the use of ksqlDB and its core mechanisms through various hands-on exercises. After completing the course, you will be able to easily and quickly build a real-time streaming data analysis system based on Kafka.

Through detailed lectures and hands-on practice on the core elements of SQL, we will provide you with a solid foundation to grow into a SQL analysis expert.

From Multi-Head Attention to the Original Transformer model, BERT, the Encoder-Decoder based MarianMT translation model, and even Vision Transformer, you'll learn Transformer inside and out by implementing them directly in code.

From core theories of deep learning and CNN to various CNN model implementation methods, and practical deep learning development know-how through real-world problems, If you want to become a deep learning CNN technology expert based on Pytorch, join us in this lecture :)

By implementing various practical data analysis cases using SQL, you can simultaneously improve your data analysis and SQL utilization skills.

From core theories of Deep Learning and CNN to implementation methods of various CNN models, and practical Deep Learning development know-how through real-world problems, If you want to become a Deep Learning CNN technology expert, join us in this lecture :)

It provides a key guide to understanding the internal mechanisms of Oracle DB architecture and growing as a performance tuning and performance analysis expert.

This course is designed to upgrade your skills as a practical machine learning development expert by implementing the machine learning model of the Home Credit Default Risk competition on Kaggle.

This course is a practical Airflow master program designed to help you understand "why Airflow works this way" and enable you to design and debug data pipelines on your own. It covers everything from Apache Airflow's core mechanisms to detailed theory and hands-on practice involving DAGs, Operators, Hooks, Scheduling, Timezones, Idempotency, and Templates.

Through this course, you will be able to immediately carry out corporate Apache Spark projects.

mongoDB, NoSQL You hear a lot these days, but is it still an unfamiliar database? Aren't you using it like a relational database (RDS/SQL)? No matter how good the technology is, if it's not used correctly, it will have the opposite effect. That's why we often see cases of failure in using MongoDB. This lecture will teach you everything from basic concepts to practical know-how so that you can use MongoDB as MongoDB-like as possible.

A DBA instructor with 30 years of experience teaches 45 essential SQL tuning techniques that can be immediately applied in practice through real-world examples. Master the practical know-how to make slow queries 10 times faster, from execution plan analysis to index optimization and join tuning!

Learn how to write SQL queries and work with databases using the MariaDB client development tool HeidiSQL.

This is a special lecture that can lay the theoretical foundation to simultaneously prepare for the Microsoft AZ-900 certification, and as the latest content reflecting the scope of questions as of May 2025, by providing content related to core data concepts, Azure's relational data, Azure's non-relational data, and Azure's analytical workloads in a form combining theory and practice, it can be utilized as a meaningful educational opportunity to not only obtain the certification but also take the first step towards becoming a data professional.

Learn NoSQL technology for handling big data, one of the fundamental skills in full-stack and data science technologies used by modern startups. MongoDB is the easiest and fastest NoSQL technology to utilize. In this course, you will learn the basics of MongoDB in a short time and master the skills to handle and utilize MongoDB with Python.

We will help you easily understand the core concepts of machine learning and acquire the ability to implement practical machine learning applications by moving away from theory-based machine learning courses.

Learn the process of building data pipelines using Apache Airflow from basics to practical application. Understand Airflow's core concepts and architecture, and master advanced design patterns frequently used in practice such as dynamic DAGs, parallel processing, distributed processing, and Custom Operators through hands-on exercises. Set up a practice environment with Python and Docker, and develop practical skills to design and operate real workflows.

SQLD Yellow Book 176 Problem-Solving Course for those who studied but can't solve problems. After completion, you'll go beyond passing to become an expert. SQLD All-in-One Pass!

If you work with data, SQL is a fundamental skill you must know. In this course, we will learn SQL from a data analysis perspective, focusing on hands-on practice. The practice will be conducted in Google Colab using DuckDB.

"Smart Factory—it sounds cool, but can't quite picture it in your head?" The news talks about the 4th Industrial Revolution and AI factories every day, but when you actually look for information, it's full of "alien languages" like PLC, MES, ERP, and Edge Computing. What non-majors need isn't a complex circuit diagram, but a 'map' that shows how the whole system works. This course strips away difficult engineering jargon and explains everything about smart factories through intuitive illustrations and analogies that anyone can understand.

A beginner's guide to SQL coding! We'll teach you the secrets of writing, interpreting, and validating SQL.

You will develop the ability to define problems based on data and clearly explain the rationale and decision-making process behind them. Additionally, rather than focusing solely on the performance of a single model, you will acquire a pipeline-oriented mindset that evaluates the completeness and reliability of the entire machine learning workflow. Furthermore, you will strengthen your problem-solving skills by tracing back the causes of errors when they occur and deriving improvement directions, and through end-to-end project experience, you will acquire practical ML pipeline capabilities that can be immediately applied in real-world settings.

You can learn the core essentials of GA4 (Google Analytics 4) e-commerce setup—which is often too difficult to learn alone just by searching the internet—through hands-on practice. This course is designed so that performance marketers and beginner developers setting up GA4 e-commerce for the first time can easily understand it by practicing step-by-step on a Cafe24 demo shopping mall. Since we delegate the developer's role to ChatGPT during the practice sessions, even marketers, PMs, or planners without professional development knowledge can learn how to set up GA4 e-commerce.

Learn how modern data teams leverage dbt. Build maintainable analytics models and validated data pipelines hands-on. Master core concepts of analytics engineering with a practical focus. This course is for those who want to take their data career to the next level 🚀

There is no need to ignore or fear machine learning. Machine learning is easy, but let's think about where to apply machine learning. The direction of machine learning expansion in Java is expected to be similar to the current Python, R strength, and future coexistence of Python, R, and Java, like iOs. The purpose of this lecture is to build a DIKW collaboration system between domain experts and IT professionals.

This course is an introductory program designed so that learners who are new to AI technology can easily understand and follow along. Through the lectures, you will learn the basic concepts and usage methods of AI platforms, and conduct hands-on practice directly generating images using various AI tools. Upon completing this course, you will become much closer to AI and acquire the capability to create AI images that can visually implement creative ideas.

Through this course, students will learn various reinforcement learning theories and implement them themselves, as well as create a reinforcement learning environment to test the reinforcement learning algorithm implemented using Unity Machine Learning Agents.

[Understanding AI Foundation Models and Their Operating Principles: Engineering Control and System Architecture, Practical Methodologies for Resolving AI Uncertainty and Engineering Assetization] 1. Introduction: The Necessity of Engineering Control of Intelligence (Engineering Control vs. Systemic Chaos) A core conclusion derived from long-term practical insights in industrial fields is that power that is not properly controlled acts as a potential liability rather than an asset. Even a high-performance engine is nothing more than an unstable physical mass if it lacks sophisticated combustion logic and microsecond-level control systems. The organizational chaos currently appearing in the process of adopting Generative AI is judged to stem from a lack of understanding of these control principles and a blind faith in technical "black boxes." This masterclass redefines Artificial Intelligence not as a mysterious stochastic phenomenon, but from the perspective of Model-Based Engineering (MBE). By transforming the uncertain domain of intelligence into a predictable and reliable engineering framework, we aim to present a strategic methodology that allows organizations to secure strong leadership across the entire system without being dependent on technical trends. 2. The 4 Pillars for Solving Core Challenges ① Epistemological Paradigm Shift: Visualizing the Black Box and Assetizing Technical Debt Many companies are facing "technical debt"—characterized by exposure to security vulnerabilities and exponential increases in maintenance costs—by adopting AI models without a clear understanding of their internal structures. This course assetizes this through the following approaches: Deconstruction of Mechanisms: We engineeringly deconstruct the Self-Attention mechanism, the core of the Transformer architecture, from the perspective of numerical weight analysis. By understanding the numerical mechanisms that determine information priority, we visualize the basis for the model's judgment. Analysis of ID Formation: We transparently track the process by which a series of pipelines—leading from Pre-training to Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF)—forms the model's technical identity and ethical guidelines. This converts invisible threats into controllable system parameters. ② Securing Deterministic Reliability: Hallucination Control Strategies to Overcome Probabilistic Limits Large Language Models (LLMs) are not systems that reason for truth, but systems that generate the next most probable token. The "hallucination" phenomenon resulting from this inherent characteristic becomes a fatal flaw in engineering fields where reliability is vital. Constraints of Retrieval-Augmented Generation (RAG): We move away from closed structures that rely solely on the model's internal fixed memory (Internal Weights). We establish an "open-book strategy" that provides clear grounding for generated results by allowing real-time reference to trusted external knowledge bases. Hybrid Model Architecture: We design a redundancy strategy that achieves both accuracy and operational efficiency by deploying large models for areas requiring enterprise-wide knowledge and optimized Small Language Models (SLMs) for specific domains where security and real-time response are essential. ③ Computing Architecture Optimization: Overcoming Physical Bottlenecks (Memory Wall) While intelligence is implemented in software, its performance and economic sustainability are defined by the physical limits of hardware. Physical Constraint Analysis: We diagnose the "Memory Wall" problem, where data transfer speeds cannot keep up with the processing speeds of computing units, and heat generation issues resulting from high-density computation from an engineering perspective. Infrastructure Design Capability: We precisely analyze the physical impact of High Bandwidth Memory (HBM) stacking structures and 2.5D/3D advanced packaging technologies on inference efficiency. We cultivate design capabilities to optimize Total Cost of Ownership (TCO) through full-stack integrated insights that complement hardware limitations with software architecture. ④ Acceleration of Functional Expansion: Transitioning from Passive Tools to Autonomous Agent Systems Current AI remains at the level of simple Q&A, failing to create added value for practical business automation. This course evolves AI into an active subject that judges and executes on its own. Decomposition: We learn techniques for decomposing complex goals into achievable sub-tasks and logically organizing the execution sequence upon receiving them. Digital Workforce Deployment: We define the process of applying "Active Agent" systems to the field, which autonomously call internal ERPs, browsers, and external APIs to complete actual business logic and accept feedback on results. 3. Core Architecture: Closed-loop Control System The way an AI agent manifests intelligence and performs complex tasks is theoretically identical in logical structure to the closed-loop control system performed by an ECU (Electronic Control Unit), the core brain of a car. This course analyzes this in detail from the perspective of the ReAct (Reasoning and Acting) framework. First, the system begins at the Input stage, receiving ambiguous and complex requests from the user. This plays the same role as a sensor in control engineering collecting physical data from the external environment and delivering it to the system, serving as the standard for defining the initial state of the task at hand. Second, based on the received data, the Thought stage proceeds, where plans are established through logical reasoning within the LLM architecture. This is in line with the process where control algorithms in an ECU calculate optimal control values by processing input sensor data. At this stage, the agent sets the optimal path to achieve the goal and secures the logical rigor of the system. Third, the Action stage follows, where tasks are completed by calling external tools or APIs according to the established plan. This logically matches the mechanism where the calculation results of a control system are converted into physical power through an actuator to execute commands. Through this, intelligence exerts actual physical and digital influence beyond abstraction. Finally, the Observation stage is performed, analyzing the execution results and correcting errors relative to the initial goal. This is identical to the core principle of control engineering, which reduces system deviation through a feedback loop. The agent self-verifies whether the execution results meet the goal and continuously upgrades performance by reflecting occurred errors into the next action plan. AI equipped with such a closed-loop structure is no longer an incomplete system dependent on probability. By securing engineering rigor that self-verifies execution results and corrects errors, it functions as a trust-based partner capable of performing business-critical tasks. 4. Practical Application and Expansion: Software-Defined Vehicles (SDV) and Physical AI The final destination of AI architecture lies in the cross-industry proliferation of Software-Defined Vehicles (SDV) and Physical AI, which overcome and evolve physical constraints through software intelligence. This is the standard model for future System Integration (SI) across manufacturing and service industries. Securing Edge Intelligence and Data Sovereignty: Small models (SLMs) mounted on-device in vehicles or facilities learn real-time field data immediately. This minimizes cloud dependency, perfectly protecting data sovereignty—a core asset of the company—and enables precision services based on ultra-low latency. Hardware Optimization and Lightweight Engineering: To implement the best intelligence within limited power and computational resources, we actively introduce model compression technologies such as Quantization, Pruning, and Knowledge Distillation. Model deployment considering hardware bandwidth becomes a core competency that determines system response speed and user experience. Hybrid Orchestration: We design an integrated architecture that organically connects "Cloud LLMs" possessing broad general knowledge with "Edge SLMs" specialized for specific physical control and security. Integration from a full-stack perspective, penetrating from silicon chipsets to software stacks, provides a powerful competitive advantage that evolves the entire system through software updates alone. 5. Conclusion: The Role and Vision of the AI Architect The ultimate goal of this masterclass is to elevate students from the position of a "User" who passively relies on technology and hopes for luck, to a professional "AI Architect" who perfectly controls and tunes everything from the physical limits of the system to the depths of the software architecture. While the phenomenon of intelligence manifests from software logic, it is silicon (hardware) that defines the physical limits of that intelligence, and only sophisticated engineering can overcome those limits to complete actual business value. "Intelligence may reside in the realm of probability, but the vessel that contains that intelligence and makes it operate according to purpose must belong solely to the realm of rigorous and sophisticated engineering."