Inflearn brand logo image
Inflearn brand logo image
Inflearn brand logo image
BEST
Data Science

/

Data Engineering

Learn Python Apache Spark from Silicon Valley Engineers

Learn how to process big data from a Silicon Valley software engineer & how to develop big data code with Apache Spark using Python. I am a 14-year software developer who handles everything from web applications to big data and SRE & DevOps with Python. Don't miss this opportunity to learn about Apache Spark, which is essential for big data professionals, in an easy and in-depth way using Python!

(4.7) 57 reviews

711 learners

  • altoformula
빅데이터분석
데이터분석가
Machine Learning(ML)
Big Data
Apache Spark
iceberg

Reviews from Early Learners

What you will learn!

  • PySpark

  • Apache Spark

  • Big data

  • Big Data Machine Learning

  • Real-time big data processing

  • Apache Cassandra

  • Apache Kafka

  • Apache Iceberg

Learn directly from Silicon Valley engineers
Would you like to take a big data lecture? 🤗

Know-how of Silicon Valley developers
In my room! 🖥️

You can easily learn big data development with the know-how of Silicon Valley developers.

Many large companies and financial institutions around the world, including Silicon Valley, are using Apache Spark to analyze large amounts of data and create machine learning models. Handling big data is an essential skill for data engineers and data scientists. And Spark’s ability is now essential for collecting and analyzing big data.

Spark was built on a distributed data processing framework from the beginning, so it can process big data in real time and create machine learning models by expanding capacity from as few as one server to as many as hundreds. Currently, I manage more than petabytes (PB) of data and operate more than 100TB of memory.

After taking this course, you will understand the core framework of Apache Spark , be able to easily collect and process big data , and create simple machine learning models using multiple servers. If you know basic Python grammar, you can study it sufficiently.

Ability to utilize Spark's RDD and Dataframe for big data analysis

Understanding the various technical elements that make up a machine learning framework

Understanding Spark Streaming for analyzing real-time data


I recommend this to these people 🙋

Having to deal with large amounts of data
Backend Developer

In the field of big data
Developers who want to study

Learn the deep knowledge of Spark
I want to be a data engineer


Learn things like this 📚

1. Introduction to Apache Spark

  • Introduction to Apache Spark
  • How to install using Docker
  • How to sign up and use Databricks Community Edition

2. Basic features and examples of Apache Spark RDD

  • Basic features and usage of Apache Spark's RDD (Resilient Distributed Dataset)
  • Introduction to Apache Spark RDD Example

3. Apache Spark SQL and Dataframe

  • Introduction and application of Apache Spark SQL and Dataframe
  • Apache Spark SQL, Data Frame Example

4. Apache Spark Engine Deep Dive

  • Apache Spark Engine Knowledge That Even Industry Professionals Don't Know


5. Apache Spark Machine Learning Library, MLlib

  • Simple machine learning algorithm
  • How to build a machine learning model with Apache Spark

6. Apache Spark Streaming, a real-time data processing library

  • How to handle real-time data with Apache Spark


Expected Questions Q&A 💬

Q. Is this a lecture that non-majors can also take?

Yes, but it may be easier to understand if you have basic Python skills and experience handling data.

If you are new to Python, learn the basics of Python through YouTube or take the lecture below first! Even if you only watch the basics, you will have no trouble following the entire lecture.

Q. What level of content is covered in the class?

Covers everything from Spark's basics to advanced information needed for the workplace.

Q. Why should I learn Spark?

Not only in Korea, but also in most companies in Silicon Valley, they process big data with Spark. If you know how to process data with Spark, it will be much easier to get a job.


Introducing the knowledge sharer ✒️

History

Portfolio/Personal Videos



Things to note before taking the class 📢

Practice environment

  • Operating System and Version (OS) : MacOS, Linux, Ubuntu
  • Tools used: Use the most popular Docker (uses public Docker images), Databricks Community Edition
    • This lecture lab is set up with Docker. If you want to learn more about Docker, I recommend you refer to my free Docker lecture . Lecture link: [ https://inf.run/8eFCL ]

Learning Materials

  • Source code and attachments provided

Recommended for
these people

Who is this course right for?

  • Anyone who knows the basic grammar of Python

  • Those who want to switch to a big data job

  • Those who want to become a relatively stable backend engineer

  • Those who want to switch to a backend engineer

  • If you want to know the latest information and details about Apache Spark

Need to know before starting?

  • Python

  • Docker

Hello
This is

10,373

Learners

696

Reviews

306

Answers

4.8

Rating

25

Courses

한국에서 끝낼 거야? 영어로 세계 시장을 뚫어라! 🌍🚀

안녕하세요. UC Berkeley에서 💻 컴퓨터 공학(EECS)을 전공하고, 실리콘 밸리에서 15년 이상을 소프트웨어 엔지니어로 일해왔으며, 현재는 실리콘밸리 빅테크 본사에서 빅데이터와 DevOps를 다루는 Staff Software Engineer로 있습니다.

  • 🧭 실리콘 밸리의 혁신 현장에서 직접 배운 기술과 노하우를 온라인 강의를 통해 이제 여러분과 함께 나누고자 합니다.

  • 🚀 기술 혁신의 최전선에서 배우고 성장해 온 저와 함께, 여러분도 글로벌 무대에서 경쟁할 수 있는 역량을 키워보세요!

  • 🫡 똑똑하지는 않지만, 포기하지 않고 꾸준히 하면 뭐든지 이룰수 있다는 점을 꼭 말씀드리고 싶습니다. 항상 좋은 자료로 옆에서 도움을 드리겠습니다

 

Curriculum

All

64 lectures ∙ (7hr 40min)

Course Materials:

Lecture resources
Published: 
Last updated: 

Reviews

All

57 reviews

4.7

57 reviews

  • Ophelie님의 프로필 이미지
    Ophelie

    Reviews 2

    Average Rating 5.0

    5

    31% enrolled

    중요한 내용만 속성으로 배우고, 샘플 데이터나 소스가 잘 정리되어 있어서 효율적으로 공부할 수 있습니다!

    • 미쿡엔지니어
      Instructor

      안녕하세요 Ophelie님, 시간내서 좋은 리뷰 남겨주셔서 정말 감사드립니다!

  • 공준호님의 프로필 이미지
    공준호

    Reviews 2

    Average Rating 5.0

    5

    100% enrolled

    좋은 내용 쉽게 잘 이해하고 들었습니다!

    • 안녕하세요 공준호님, 시간내서 좋은 리뷰 남겨주셔서 감사합니다.

  • 심준걸님의 프로필 이미지
    심준걸

    Reviews 1

    Average Rating 5.0

    5

    100% enrolled

    좋은 강의 잘 수강 했습니다. 실무로 Spark 관련 업무를 진행하기 위해 수강 했는데요. 이론적인 부분과 실습을 핵심적인 내용만 잘 다뤄주셔서 도움이 많이 되었습니다.

    • 안녕하세요 심준걸님, 시간내서 좋은 리뷰 남겨주셔서 감사합니다. 실무에서 도움이 되었다니 기쁘네요!

  • 최규영님의 프로필 이미지
    최규영

    Reviews 2

    Average Rating 5.0

    5

    32% enrolled

    아직 수강 중이지만 전체적인 개요, 강의 구성, 설명 방식이 잘 짜여 있어 만족스럽습니다. 코드 문법을 하나하나 자세히 설명하진 않지만, 코드를 보며 실행 흐름과 동작 원리를 설명해주기 때문에, 어느 정도 코딩 경험이 있는 분이라면 이해하며 학습하기에 적합한 강의라고 생각합니다.

    • 안녕하세요 최규영님, 시간내서 좋은 리뷰 남겨주셔서 감사합니다.

  • gogo91rla님의 프로필 이미지
    gogo91rla

    Reviews 2

    Average Rating 5.0

    5

    14% enrolled

    현재 미국에서 Data Engineer로 커리어 변경하고 일한지 1년 정도되가는데, Spark 개념에 대해서 다시 한번 리뷰하고 모르던 내용도 배울 수 있어서 많은 도움이 됩니다!!

    • 안녕하세요 gogo91rla님, 시간내서 좋은 리뷰 남겨주셔서 감사합니다! 도움이 되었다니 다행입니다!

$77.00

altoformula's other courses

Check out other courses by the instructor!

Similar courses

Explore other courses in the same field!