inflearn logo
inflearn logo

Learning by doing: Practical Spark Part 1

By the end of this course, you will be able to implement Apache Spark projects in your organization.

강의소개.상단개요.수강생

난이도 초급

수강기한 12개월

Apache Spark
Apache Spark
Big Data
Big Data
Machine Learning(ML)
Machine Learning(ML)
data-transformation
data-transformation
Apache Spark
Apache Spark
Big Data
Big Data
Machine Learning(ML)
Machine Learning(ML)
data-transformation
data-transformation

강의상세_배울수있는것_타이틀

  • Using Spark Core Commands

  • Spark-based Data Science

Hands-on Practical Spark Part 1

Course Introduction

"Hands-on Practical Spark Part 1" is a practice-oriented course designed for everyone from learners who are new to data science to professionals preparing for real-world projects using Spark. This course is structured to systematically learn from Spark's basic concepts to practical applications, with particular focus on essential commands and data processing methods that are crucial for executing Spark projects.

@Apache Spark, @Big Data, @Machine Learning, @Data Engineering, @Data Transformation

Course Objectives

  • Spark Fundamentals and Environment Setup: Learn Spark's operating principles and environment configuration methods to efficiently utilize it in both local and Docker environments.

  • Distributed Data Processing and Optimization: Learn the fundamentals of large-scale data processing through hands-on practice with Spark's distributed processing concepts, data partitioning, shuffling, and cluster resource configuration.

  • Acquiring practical data processing skills: Learn advanced data processing techniques by loading, transforming, filtering, and combining data through various Spark commands.

  • Developing Data Analysis and Visualization Skills: Analyze data using Spark's DataFrames and SQL commands, and visualize the results.


Curriculum Structure

  1. Orientation

    • Introduces the concepts of Spark and its practical application possibilities, and presents learning directions.

  2. Spark Environment Setup

    • Learn how to install and configure Spark using local environment and Docker to set up a practice environment.

  3. Distributed Processing Concepts

    • Learn how Spark processes large-scale data and the basic principles of distributed processing.

  4. Understanding Spark Operations

    • You will visually understand the core operational principles of Lazy Operation, partitioning, shuffling, and more through Jupyter Notebook and Spark UI.

  5. Essential Spark Commands for Real-World Practice

    • You will learn frequently used commands in practice such as data loading, date filtering, join, aggregation, UDF utilization, and data storage.

    • It also includes methods for efficiently utilizing SQL commands.

  6. Advanced Data Processing

    • You will learn advanced techniques for handling common real-world problems such as string data processing, null value handling, JSON data manipulation, and partition optimization.


Who is this course for?

  • Beginner learners who want to learn from the basics of Spark to practical application methods for beginners

  • Data engineers who want to learn data analysis and engineering techniques using Spark

  • Professionals who want to carry out corporate Spark projects or build scalable data pipelines working experts


Expected Benefits After Taking the Course

  • You can develop data processing and analysis capabilities using Spark and secure the competency to execute Spark projects in enterprise environments.

  • You will acquire practical know-how for efficiently processing large-scale data by loading, transforming, and storing data in real-world scenarios.

  • You can solidly build the foundation for Spark projects in cloud environments that will be covered in Part 2.


If you're just starting with Spark or want to learn practical data processing skills, "Hands-on Practical Spark Part 1" will be the perfect starting point. Let's move forward together into the world of data science! 🎓✨

강의소개.콘텐츠.추천문구

학습 대상은 누구일까요?

  • People who are new to Spark

  • People who want to do a Spark corporate project

선수 지식, 필요할까요?

  • Python Basics (Very Low Level)

강의소개.지공자소개

161

수강생

15

수강평

27

답변

4.9

강의 평점

3

강의_other

I am currently serving as a development lead and consultant for projects at major corporations, as listed below. I am still active in the field.^^

In addition, I am serving as an adjunct professor specializing in Artificial Intelligence at Korea University's graduate school.

My goal is to provide practical, hands-on programming skills that can be applied immediately in the field. I look forward to creating engaging and enjoyable classes with all of you.

  • Enterprise AI Architecture and Service Design

  • Machine learning service implementation

  • Backend service development

  • Building databases and developing services in various cloud environments, including Cloud (Azure) Databricks, ETL, and Fabric.

더보기

커리큘럼

전체

48개 ∙ (강의상세_런타임_시간 강의상세_런타임_분)

강의 게시일: 
마지막 업데이트일: 

수강평

아직 충분한 평가를 받지 못한 강의입니다.
모두에게 도움이 되는 수강평의 주인공이 되어주세요!

비슷한 강의

같은 분야의 다른 강의를 만나보세요!

강의상세.할인문구

$8,471.00

30%

$77.00