Airflow Basics from a Silicon Valley Data Leader
With the advent of the AI era, building data pipelines has become a core competency that determines a company's competitiveness. Learn how to build efficient data pipelines using Airflow, the most widely used tool, directly from a Silicon Valley expert (formerly Head of the Data Team at Udemy, currently a professor in the Data Masters program at San Jose State University) with practical experience and extensive lecturing experience.
241 learners
Level Basic
Course period Unlimited
News
2 articles
I have added a lecture titled "Airflow MCP Integration" at the end of Section 9, "Advanced Airflow Programming." It covers how to install the Airflow MCP server on Claude Desktop and control Airflow through chat. You can try executing commands like the ones below via chat:
"Which DAGs are currently registered?"
“Run DAG A right now”
“Summarize the most recent execution status of DAG A”
“Disable DAG A”
Are you listening to the lecture well? Please ask many questions. Thank you to everyone who has asked questions so far!
By next Monday, I'll be adding a tutorial on creating a DAG that connects Google Sheets and Snowflake.
Execute a SELECT statement in Snowflake and upload the results to Google Sheets
Upload the contents of Google Sheets to Snowflake as a table
In actual field work, non-technical departments manage a lot of data in the form of Google Sheets. So I think this DAG can be quite useful and I believe it will be helpful to you. Thank you.
