inflearn logo

Learning Spark through Practice Part 1

Through this course, you will be able to immediately carry out corporate Apache Spark projects.

(5.0) 1 reviews

32 learners

Level Basic

Course period 12 months

Apache Spark
Apache Spark
Big Data
Big Data
Machine Learning(ML)
Machine Learning(ML)
data-transformation
data-transformation
Apache Spark
Apache Spark
Big Data
Big Data
Machine Learning(ML)
Machine Learning(ML)
data-transformation
data-transformation

What you will gain after the course

  • How to use core Spark-based commands

  • Spark-based Data Science

Hands-on Spark Practice Part 1

Course Introduction

"Hands-on Practical Spark Part 1" is a practice-oriented course designed for everyone, from beginners encountering data science for the first time to professionals preparing for real-world projects using Spark. This course is structured to provide systematic learning from basic Spark concepts to practical application methods, with a particular focus on the essential commands and data processing techniques required for executing Spark projects.

@Apache Spark, @Big Data, @Machine Learning, @Data Engineering, @Data Transformation

Course Objectives

  • Basic Principles and Environment Setup of Spark: Learn the operating principles and environment configuration of Spark to effectively utilize it in local and Docker environments.

  • Distributed Data Processing and Optimization: Build a foundation for large-scale data processing by practicing Spark's distributed processing concepts, data partitioning, shuffling, and cluster resource configuration.

  • Acquisition of Practical Data Processing Skills: Learn advanced data processing techniques by loading, transforming, filtering, and combining data through various Spark commands.

  • Cultivating data analysis and visualization skills: Analyze data using Spark's DataFrames and SQL commands, and visualize the results.


Curriculum Structure

  1. Orientation

    • Introduces the concepts of Spark and its practical applications, and provides a direction for learning.

  2. Spark Environment Configuration

    • Learn how to install and configure Spark using local environments and Docker to set up a hands-on practice environment.

  3. Distributed Processing Concepts

    • Learn the way Spark processes large-scale data and the basic principles of distributed processing.

  4. Understanding Spark Operations

    • Understand core operational principles such as Lazy Operation, partitioning, and shuffling visually through Jupyter Notebook and Spark UI.

  5. Essential Spark Commands for Practice

    • Learn frequently used commands in practice, such as data loading, date filtering, joins, aggregations, UDF utilization, and data storage.

    • It also includes how to efficiently utilize SQL commands.

  6. Advanced Data Processing

    • You will learn advanced techniques for handling common real-world issues, such as string data processing, null value handling, working with JSON data, and partition optimization.


Who is this course for?

  • Beginner learners who want to learn everything from the basics of Spark to its practical application.

  • Data engineers who want to master data analysis and engineering skills using Spark

  • Working professionals who want to carry out corporate Spark projects or build scalable data pipelines


Expected Outcomes After Completion

  • You will gain the ability to process and analyze data using Spark and secure the competency to carry out Spark projects in a corporate environment.

  • You will acquire the know-how to efficiently process large-scale data by loading, transforming, and storing data in practical business scenarios.

  • You can build a solid foundation for the cloud-based Spark projects that will be covered in Part 2.


If you are starting Spark for the first time or want to learn practical data processing skills, "Hands-on Practical Spark Part 1" will be the perfect starting point. Step into the world of data science together! 🎓✨

Recommended for
these people

Who is this course right for?

  • Someone new to Spark

  • People who want to carry out a Spark corporate project

Need to know before starting?

  • Python Basics (Very Low Level)

Hello
This is nexthumans

176

Learners

16

Reviews

29

Answers

4.9

Rating

3

Courses

I am currently serving as a development lead and consultant for projects at major corporations, as listed below. I am still active in the field.^^

In addition, I am serving as an adjunct professor specializing in Artificial Intelligence at Korea University's graduate school.

My goal is to provide practical, hands-on programming skills that can be applied immediately in the field. I look forward to creating engaging and enjoyable classes with all of you.

  • Enterprise AI Architecture and Service Design

  • Machine learning service implementation

  • Backend service development

  • Building databases and developing services in various cloud environments, including Cloud (Azure) Databricks, ETL, and Fabric.

More

Curriculum

All

48 lectures ∙ (10hr 33min)

Published: 
Last updated: 

Reviews

All

1 reviews

5.0

1 reviews

    Similar courses

    Explore other courses in the same field!

    $77.00