강의

멘토링

로드맵

BEST
Data Science

/

Data Engineering

The Complete Guide to Spark Machine Learning - Part 1

If you want to be recognized as a machine learning expert based on large-scale data, from understanding the core framework of Spark machine learning, to SQL-based data processing through difficult practical problems, to data analysis through business domain analysis, and to the ability to implement optimized machine learning models, please join this course.

(4.9) 27 reviews

933 learners

  • dooleyz3525
한국에 이런 강의가?
압도적 분량
Apache Spark
Machine Learning(ML)
Big Data
Data Engineering

Reviews from Early Learners

What you will learn!

  • Implementing Machine Learning Models in Spark

  • Detailed understanding of DataFrame, the foundation of Spark's data processing

  • Understand the various technical elements that make up the Spark Machine Learning Framework

  • Mastering Spark's Machine Learning Pipeline

  • Ability to use SQL for data analysis

  • SQL-based Feature Engineering Techniques

  • Implementing models with XGBoost and LightGBM in Spark

  • Model hyperparameter tuning method based on Bayesian optimization

  • Improve your data analysis and ML model implementation skills simultaneously through challenging real-world problems.

  • Data analysis method based on analysis domain

  • Various data visualization techniques

Data analysis + feature engineering + ML implementation,
Grab three competencies at once.

With Apache Spark
The meeting of machine learning.

Apache Spark, the leader in open source large-scale distributed processing solutions, has met with Machine Learning .

Many large domestic corporations and financial institutions are leveraging Apache Spark to analyze large amounts of data and build machine learning models. Because Spark is based on a distributed data processing framework, it can scale across a few to dozens of servers, processing large amounts of data and building machine learning models. This allows it to overcome the limitations of scikit-learn, which only allows machine learning models to be implemented on a single server.


Also good at data processing/analysis
As a machine learning expert
I will help you grow.

The 'Spark Machine Learning Complete Guide - Part 1' course will not only teach you how to implement machine learning models in Spark, but will also help you grow into a machine learning expert skilled in data processing and analysis .

To become a true machine learning expert, it's crucial not only to master ML implementation skills, but also to understand how to process and combine business data to create ML models. To achieve this, you'll learn how to process data using SQL, the most commonly used language for large-scale data processing , and acquire hands-on data analysis techniques based on domain analysis .

It is designed to help you develop data processing/analysis and ML implementation capabilities through detailed theoretical explanations and practical training.


The problems you will face
We will solve it for you.

Implementing machine learning models on Spark is challenging. This is because it presents many challenges unfamiliar to traditional data scientists and machine learning experts, including unique machine learning APIs and frameworks based on Spark's architecture, and SQL-based data processing.

This course, The Complete Guide to Spark Machine Learning, will empower you to solve the problems you face .

The first half of the lecture 'Spark Machine Learning Complete Guide - Part 1'

The first half of the course features detailed theoretical explanations and extensive hands-on practice on the various components of the Spark Machine Learning Framework, including DataFrames, SQL, Estimators, Transformers, Pipelines, and Evaluators. This will enable you to quickly and easily implement ML models in Spark .

We will also explain in detail how to use XGBoost and LightGB in Spark, and how to tune hyperparameters using HyperOpt based on Bayesian optimization.

The second half of the lecture 'Spark Machine Learning Complete Guide - Part 1'

The second half of the course will focus on practicing Kaggle's Instacart Market Basket Analysis competition, simultaneously improving your practical data processing/analysis skills and machine learning model implementation. The Kaggle Instacart competition is a challenging competition, particularly given the dataset's structure, which consists of e-commerce order processing tables (products, orders, and order items).

Through this dataset, you will learn in detail how to process and analyze business data based on SQL, perform feature engineering, derive analysis domains from business, and create models based on the derived features.

This is Part 1 of the "Spark Machine Learning Complete Guide." Part 2 , scheduled for release at a later date, will cover text analysis, recommendations, and time-series analysis.

💻 Please check before taking the class!

  • All practical code in this course is written in Python. Scala is not covered, so please refer to this information before selecting a course.

Practice environment
Please check.

This hands-on training uses Databricks. Databricks provides a notebook environment for building Spark-based applications in the cloud without installing Spark.

Databricks is officially available for free use for 14 days as a Community version.
And in the video lecture ' Managing Spark Clusters on Databricks and Using Databricks Even After 2 Weeks of Subscription ' in Section 0, I explain how you can continue to use it for free after 14 days, so please watch that video carefully (for explanation about Databricks Community version, please refer to the link ).

You can download the lecture practice code and lecture explanation materials from 'Download the practice code and explanation materials' .


Player knowledge
This is a necessary lecture.

This course is designed with the assumption that students have knowledge of Chapter 5 (Regression) of the Complete Guide to Python Machine Learning or equivalent, and that they also have a very basic understanding of SQL . Please refer to the above when selecting a course.

It's helpful to know the basics of Spark, but you'll still be able to follow the course without any prior knowledge.

Please check out the player lecture!

The Complete Guide to Python Machine Learning

Stop teaching theory-based machine learning.
From core machine learning concepts to practical skills, easily and accurately.

Curious about the interview with the knowledge sharer? (Click)

Recommended for
these people

Who is this course right for?

  • Anyone who wants to implement machine learning using Spark

  • Those who want to implement machine learning based on large-scale data

  • Anyone who wants to improve their data processing techniques for machine learning using SQL

  • Anyone who wants to learn the entire process of processing data into the desired format and creating an ML model based on it in practice

  • Anyone who wants to improve data analysis, feature engineering capabilities, and ML implementation

Need to know before starting?

  • Understanding up to Chapter 5 (Regression) of the Complete Guide to Python Machine Learning or equivalent prior knowledge

  • Understanding SQL Basics

Hello
This is

26,325

Learners

1,322

Reviews

3,983

Answers

4.9

Rating

13

Courses

(전) 엔코아 컨설팅

(전) 한국 오라클

AI 프리랜서 컨설턴트

파이썬 머신러닝 완벽 가이드 저자

Curriculum

All

117 lectures ∙ (24hr 27min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

27 reviews

4.9

27 reviews

  • freedom07님의 프로필 이미지
    freedom07

    Reviews 7

    Average Rating 5.0

    5

    93% enrolled

    파이썬 머신러닝 완벽가이드 통해서 권철민선생님을 처음 알게 되었습니다. 그 강의를 통해서 비전공자였던 저는 포기하려고 했던 이 분야를 포기하지 않을 수 있었습니다. 현재 이 분야에서 일을 하면서 이렇게 인프런 강의를 들으며 공부도 꾸준히 하고 있습니다. 선생님께 감사하다는 말씀을 전하고 싶어서 처음에 질문답변 사안에 선생님께 감사하다는 말씀을 드렸었는데, 선생님께서 꾸준히 하면 노력한 바를 이룰 수 있을 거라고 응원하면서 말씀해주셨습니다. 앞으로도 선생님께서 강의하시는 것 꾸준히 들을 예정입니다. ^^ㅎㅎ 그만큼 정말 잘 가르쳐주십니다. 권철민 선생님 이 자리를 빌러, 진심으로 정말 감사합니다.

    • 권 철민
      Instructor

      이렇게 가슴 뭉클한 수강평을 남겨 주시다니 제가 더 감명 받았습니다. 강의를 만드는 수고를 한 순간에 보상받는 글이여서 제가 오히려 감사드려야 할 것 같습니다. 앞으로도 계속 이렇게 정진하신다면, 원하는 모든 일 확실히 다 성취 하실 것입니다. 감사합니다.

  • egs41님의 프로필 이미지
    egs41

    Reviews 54

    Average Rating 5.0

    5

    10% enrolled

    강사님의 딕션과 목소리에 집중하기 좋았고, 컨텐츠 또한 탄탄합니다. 앞으로도 좋은 강의 만들어주세요. 감사합니다.

    • 밑바닥개발자님의 프로필 이미지
      밑바닥개발자

      Reviews 13

      Average Rating 5.0

      5

      54% enrolled

      권철민님 강의 시리즈를 쭉 들어온 수강생입니다! 여전히 양질의 강의를 제공해주셔서 감사합니다! 그리고 Spark 강의가 Scala, Java로 구성된 강의들을 몇 번 보았지만 Python으로 Spark를 알려주시는 강의는 처음이어서 더 좋았던 것 같네요! 아직 완강하지는 않았지만, 여전히 간단한 문법도 최대한 쉽게 알려주시려고 하는 게 가장 좋네요! 그리고 반복 숙달을 유도하기 위해 다양한 실습자료를 제공해주시는 것도 좋습니다! 앞으로 다른 강의들도 기대가 됩니다!

      • kjo19990606님의 프로필 이미지
        kjo19990606

        Reviews 8

        Average Rating 4.9

        5

        100% enrolled

        덕분에 spark에 대해서 알게되었고 캐글도전에도 자신감을 얻게 되었스빈다 감사합니다 !

        • 인디즈님의 프로필 이미지
          인디즈

          Reviews 1

          Average Rating 5.0

          5

          100% enrolled

          차근차근 잘 알려주셔서 감사합니다

          $77.00

          dooleyz3525's other courses

          Check out other courses by the instructor!

          Similar courses

          Explore other courses in the same field!