inflearn logo
inflearn logo

Learning Spark through Practice Part 1

Through this course, you will be able to immediately carry out corporate Apache Spark projects.

29 learners are taking this course

Level Basic

Course period 12 months

Apache Spark
Apache Spark
Big Data
Big Data
Machine Learning(ML)
Machine Learning(ML)
data-transformation
data-transformation
Apache Spark
Apache Spark
Big Data
Big Data
Machine Learning(ML)
Machine Learning(ML)
data-transformation
data-transformation

What you will gain after the course

  • How to use core Spark-based commands

  • Spark-based Data Science

Hands-on Spark Practice Part 1

Course Introduction

"Hands-on Practical Spark Part 1" is a practice-oriented course designed for everyone, from beginners encountering data science for the first time to professionals preparing for real-world projects using Spark. This course is structured to provide systematic learning from basic Spark concepts to practical application methods, with a particular focus on the essential commands and data processing techniques required for executing Spark projects.

@Apache Spark, @Big Data, @Machine Learning, @Data Engineering, @Data Transformation

Course Objectives

  • Basic Principles and Environment Setup of Spark: Learn the operating principles and environment configuration of Spark to effectively utilize it in local and Docker environments.

  • Distributed Data Processing and Optimization: Build a foundation for large-scale data processing by practicing Spark's distributed processing concepts, data partitioning, shuffling, and cluster resource configuration.

  • Acquisition of Practical Data Processing Skills: Learn advanced data processing techniques by loading, transforming, filtering, and combining data through various Spark commands.

  • Cultivating data analysis and visualization skills: Analyze data using Spark's DataFrames and SQL commands, and visualize the results.


Curriculum Structure

  1. Orientation

    • Introduces the concepts of Spark and its practical applications, and provides a direction for learning.

  2. Spark Environment Configuration

    • Learn how to install and configure Spark using local environments and Docker to set up a hands-on practice environment.

  3. Distributed Processing Concepts

    • Learn the way Spark processes large-scale data and the basic principles of distributed processing.

  4. Understanding Spark Operations

    • Understand core operational principles such as Lazy Operation, partitioning, and shuffling visually through Jupyter Notebook and Spark UI.

  5. Essential Spark Commands for Practice

    • Learn frequently used commands in practice, such as data loading, date filtering, joins, aggregations, UDF utilization, and data storage.

    • It also includes how to efficiently utilize SQL commands.

  6. Advanced Data Processing

    • You will learn advanced techniques for handling common real-world issues, such as string data processing, null value handling, working with JSON data, and partition optimization.


Who is this course for?

  • Beginner learners who want to learn everything from the basics of Spark to its practical application.

  • Data engineers who want to master data analysis and engineering skills using Spark

  • Working professionals who want to carry out corporate Spark projects or build scalable data pipelines


Expected Outcomes After Completion

  • You will gain the ability to process and analyze data using Spark and secure the competency to carry out Spark projects in a corporate environment.

  • You will acquire the know-how to efficiently process large-scale data by loading, transforming, and storing data in practical business scenarios.

  • You can build a solid foundation for the cloud-based Spark projects that will be covered in Part 2.


If you are starting Spark for the first time or want to learn practical data processing skills, "Hands-on Practical Spark Part 1" will be the perfect starting point. Step into the world of data science together! 🎓✨

Recommended for
these people

Who is this course right for?

  • Someone new to Spark

  • People who want to carry out a Spark corporate project

Need to know before starting?

  • Python Basics (Very Low Level)

Hello
This is nexthumans

170

Learners

15

Reviews

29

Answers

4.9

Rating

3

Courses

I am currently serving as a development lead and consultant for projects at major corporations, as listed below. I am still active in the field.^^

In addition, I am serving as an adjunct professor specializing in Artificial Intelligence at Korea University's graduate school.

My goal is to provide practical, hands-on programming skills that can be applied immediately in the field. I look forward to creating engaging and enjoyable classes with all of you.

  • Enterprise AI Architecture and Service Design

  • Machine learning service implementation

  • Backend service development

  • Building databases and developing services in various cloud environments, including Cloud (Azure) Databricks, ETL, and Fabric.

More

Curriculum

All

48 lectures ∙ (10hr 33min)

Published: 
Last updated: 

Reviews

Not enough reviews.
Please write a valuable review that helps everyone!

Similar courses

Explore other courses in the same field!

$77.00