강의

멘토링

커뮤니티

BEST
Data Science

/

Data Engineering

Airflow Basics from a Silicon Valley Data Leader

With the advent of the AI era, building data pipelines has become a core competency that determines a company's competitiveness. Learn how to build efficient data pipelines using Airflow, the most widely used tool, directly from a Silicon Valley expert (formerly Head of the Data Team at Udemy, currently a professor in the Data Masters program at San Jose State University) with practical experience and extensive lecturing experience.

(4.9) 13 reviews

211 learners

Level Basic

Course period Unlimited

  • keeyonghan
실습 중심
실습 중심
데이터파이프라인
데이터파이프라인
실리콘밸리
실리콘밸리
airflow
airflow
snowflake
snowflake
SQL
SQL
Python
Python
실습 중심
실습 중심
데이터파이프라인
데이터파이프라인
실리콘밸리
실리콘밸리
airflow
airflow
snowflake
snowflake
SQL
SQL
Python
Python

What you will gain after the course

  • Building a data pipeline based on Airflow, Snowflake, and Docker

  • Practical SQL and Python skills that can be immediately applied in real-world data work.

What Silicon Valley Data Engineers Say
Build a modern data pipeline!


Practical know-how from a 30-year data engineer in Silicon Valley


Rated 4.9! A course highly praised by San Jose State University data science students


Design & automate modern data pipelines using Airflow & Snowflake

Meet Airflow and Snowflake
Modern Data Engineering Architecture

In an era of massive data, the core of data engineering is to efficiently collect and process data and provide it when needed. For this , an automated data pipeline is essential , and Airflow and Snowflake are powerful tools used in this process .


Airflow automates complex data flows and enables accurate and reliable data collection through flexible scheduling and task management . Snowflake is a powerful cloud-based data warehouse that can quickly process and scale large amounts of data, supporting stable data operations in various business environments.

In this course, you will learn how to design and operate efficient data pipelines by combining Airflow’s workflow automation capabilities with Snowflake’s scalability and performance . You will learn the core data engineering skills that connect data collection, processing, storage, and utilization into a single flow without complex infrastructure setup.

Learn about these things

1⃣ Designing a Practical Data Pipeline with Airflow + Snowflake + Docker

2⃣ Learn everything from Airflow environment setup to ETL, DAG management, and automation all at once

3⃣ Learn data flow optimization and operational know-how through practical examples

I recommend this to these people

As a data engineer
I'm thinking about my career
Developers/Analysts/Scientists/Students who want to become data engineers but don't know what to do

I'm interested in building data pipelines
If you work with data and need to work on data pipelines or are curious about this method,

About data engineering
I want to know better
Anyone who works as a data engineer or ML engineer but wants to know more

After class

  • Learn about the mission of a data organization and the role of data engineering.


  • You will learn about data warehouses and data pipelines that make up data infrastructure.

    • In this process, you can add a data warehouse called Snowflake to your skillset.

    • Learn best practices that will really help you build and operate data pipelines.

  • You can create various data pipelines based on Airflow, the most popular data pipeline creation/operation framework.

    • You will also learn advanced concepts like full updates, incremental updates, backfill, monitoring, etc.

  • As a data worker, you will gain hands-on experience in how Python and SQL are used to create data pipelines, which will enhance your capabilities.

Insights from proven data experts in Silicon Valley

Hello. I am Ki-Yong Han, a data expert in Silicon Valley with 30 years of experience. After starting my career at Samsung Electronics, I left for Silicon Valley at the age of 31. I have built data teams at organizations such as Udemy (listed on NASDAQ in 2021) and Polyvore (acquired by Yahoo in 2015), and have provided data consulting to various Silicon Valley and Korean companies. Based on this, and my experience teaching master's students at San Jose State University, which boasts the highest employment rate in Silicon Valley , I will share essential skills for data scientists.

Things to note before taking the class

Practice environment

  • We will run Airflow based on Docker. We will introduce Docker and explain the installation process in the lecture.

  • For data warehouse, we use Snowflake's free trial. It is free for 30 days or $400 in credits, no credit card required, and you can retake the free trial without any problems after the free trial ends (but you will need to set up a new environment).

  • We use Google Colab as an introduction to data pipeline coding.

Learning Materials

Player Knowledge and Notes

  • Basic Python Grammar (Beginner)


  • Basic SQL Knowledge (Beginner)

  • Will to follow along diligently

Recommended for
these people

Who is this course right for?

  • People who work as data engineers or want to work as data engineers

  • People who work with or want to work with data-related pipelines

  • Someone curious about data-related work/projects

Need to know before starting?

  • Python (Beginner)

  • SQL (Beginner)

Hello
This is

1,064

Learners

69

Reviews

41

Answers

4.9

Rating

5

Courses

After earning a Master's degree in Computer Science, my career began at Samsung Electronics and led me to Silicon Valley thanks to a friend. Over the past 29 years, I have worked at 13 companies across various stages (startups, major corporations, and numerous early-stage ventures).

  • Yahoo: Developed search engines as an Engineering Director.

  • Udemy. Built the data team from scratch and grew it to 30 people. Listed on NASDAQ in October 2021.

  • Samsung Electronics

  • ...

I took an 11-month break in between and built my own brand while unintentionally taking on roles such as an angel investor (Chartmetric, Goodtime.io, Select Star, EO, Business Canvas, ...), advisor (Moloco, Blind, Wolbu, ...), and consultant (SK Telecom, Hyundai Card, E-mart, etc.). I believe in the power of positivity to see failure as a lesson rather than just failure, and the power of compounding that comes from consistency.

https://www.linkedin.com/in/keeyonghan/

YouTube Channel

Wolgeupjae-Bujadeul Lectures

Curriculum

All

68 lectures ∙ (12hr 4min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

13 reviews

4.9

13 reviews

  • horongt님의 프로필 이미지
    horongt

    Reviews 1

    Average Rating 5.0

    5

    100% enrolled

    I've finally completed the course!! I was able to gain a lot technically, but the overall concepts of data engineering that the instructor took the time to explain were really helpful (content from Section 2, the future of data engineering). Through this, I was able to gain great insights into how to design pipelines and what technology stack to choose. Also, I liked that the instructor mentioned the concerns that data engineers should have from time to time during the lecture. In addition, the lecture itself was created to allow you to learn practical tips and theoretical parts in a balanced way, so I liked the part where they tell you exactly what you need to know and leave the rest to the person studying. Thank you for the great lecture!!

    • keeyonghan
      Instructor

      Thank you for the kind review. I also have an SQL course created from a data analysis/utilization perspective. I'm planning to release a course related to Spark by early April, so please check that out later.

    • Okay!! I can listen to the SQL lecture while waiting for the Spark lecture!

  • s00park님의 프로필 이미지
    s00park

    Reviews 3

    Average Rating 5.0

    5

    100% enrolled

    • slavefactory님의 프로필 이미지
      slavefactory

      Reviews 8

      Average Rating 5.0

      5

      53% enrolled

      • msson7577님의 프로필 이미지
        msson7577

        Reviews 1

        Average Rating 5.0

        5

        100% enrolled

        Okay

        • paulmoon008308님의 프로필 이미지
          paulmoon008308

          Reviews 111

          Average Rating 4.9

          5

          7% enrolled

          $102.30

          keeyonghan's other courses

          Check out other courses by the instructor!

          Similar courses

          Explore other courses in the same field!