inflearn logo
inflearn logo

Learn Python Apache Spark from Silicon Valley Engineers

Learn how to process big data from a Silicon Valley software engineer & how to develop big data code with Apache Spark using Python. I am a 14-year software developer who handles everything from web applications to big data and SRE & DevOps with Python. Don't miss this opportunity to learn about Apache Spark, which is essential for big data professionals, in an easy and in-depth way using Python!

(4.7) 수강평 77개

강의소개.상단개요.수강생.short

난이도 중급이상

수강기한 무제한

Machine Learning(ML)
Machine Learning(ML)
Big Data
Big Data
Apache Spark
Apache Spark
iceberg
iceberg
Machine Learning(ML)
Machine Learning(ML)
Big Data
Big Data
Apache Spark
Apache Spark
iceberg
iceberg

먼저 경험한 수강생들의 후기

먼저 경험한 수강생들의 후기

4.7

5.0

Hyunwoo Park

78% 수강 후 작성

This is a great lecture structure for using Spark in practice. Since setting up a development environment is always complicated and takes a lot of time, I used to lose energy before I even started learning, but it was good to learn how to set up an environment that I can study at once with docker-compose. I've been developing with Spark for several years, but it was good to learn more about the functions I didn't know about by looking back at the overall concept. I think this Spark lecture is suitable for both beginners and intermediate users. Thank you!

5.0

오늘했던거 까먹는사람

100% 수강 후 작성

I moved from de to backend developer, and I want to work at de again, so it's good to review. I think it would have been good if there had been explanations by mapping Spark UI and code. Review after completing the course There was also explanations by mapping Spark UI and code. I also took the course because I needed more streaming concepts, and it was good. I learned how to use Spark in the future, but I wish there had been a course on how to build data pipelining and how to build it.

5.0

미국달팽이

93% 수강 후 작성

I also live in Silicon Valley, and the concepts, examples, and streaming were very helpful. Thank you. I also paid for other courses, and it was a good decision.

강의상세_배울수있는것_타이틀

  • PySpark

  • Apache Spark

  • Big data

  • Big Data Machine Learning

  • Real-time big data processing

  • Apache Cassandra

  • Apache Kafka

  • Apache Iceberg

Learn directly from Silicon Valley engineers
Would you like to take a big data lecture? 🤗

Know-how of Silicon Valley developers
In my room! 🖥️

You can easily learn big data development with the know-how of Silicon Valley developers.

Many large companies and financial institutions around the world, including Silicon Valley, are using Apache Spark to analyze large amounts of data and create machine learning models. Handling big data is an essential skill for data engineers and data scientists. And Spark’s ability is now essential for collecting and analyzing big data.

Spark was built on a distributed data processing framework from the beginning, so it can process big data in real time and create machine learning models by expanding capacity from as few as one server to as many as hundreds. Currently, I manage more than petabytes (PB) of data and operate more than 100TB of memory.

After taking this course, you will understand the core framework of Apache Spark , be able to easily collect and process big data , and create simple machine learning models using multiple servers. If you know basic Python grammar, you can study it sufficiently.

Ability to utilize Spark's RDD and Dataframe for big data analysis

Understanding the various technical elements that make up a machine learning framework

Understanding Spark Streaming for analyzing real-time data


I recommend this to these people 🙋

Having to deal with large amounts of data
Backend Developer

In the field of big data
Developers who want to study

Learn the deep knowledge of Spark
I want to be a data engineer


Learn things like this 📚

1. Introduction to Apache Spark

  • Introduction to Apache Spark
  • How to install using Docker
  • How to sign up and use Databricks Community Edition

2. Basic features and examples of Apache Spark RDD

  • Basic features and usage of Apache Spark's RDD (Resilient Distributed Dataset)
  • Introduction to Apache Spark RDD Example

3. Apache Spark SQL and Dataframe

  • Introduction and application of Apache Spark SQL and Dataframe
  • Apache Spark SQL, Data Frame Example

4. Apache Spark Engine Deep Dive

  • Apache Spark Engine Knowledge That Even Industry Professionals Don't Know


5. Apache Spark Machine Learning Library, MLlib

  • Simple machine learning algorithm
  • How to build a machine learning model with Apache Spark

6. Apache Spark Streaming, a real-time data processing library

  • How to handle real-time data with Apache Spark


Expected Questions Q&A 💬

Q. Is this a lecture that non-majors can also take?

Yes, but it may be easier to understand if you have basic Python skills and experience handling data.

If you are new to Python, learn the basics of Python through YouTube or take the lecture below first! Even if you only watch the basics, you will have no trouble following the entire lecture.

Q. What level of content is covered in the class?

Covers everything from Spark's basics to advanced information needed for the workplace.

Q. Why should I learn Spark?

Not only in Korea, but also in most companies in Silicon Valley, they process big data with Spark. If you know how to process data with Spark, it will be much easier to get a job.


Introducing the knowledge sharer ✒️

History

Portfolio/Personal Videos



Things to note before taking the class 📢

Practice environment

  • Operating System and Version (OS) : MacOS, Linux, Ubuntu
  • Tools used: Use the most popular Docker (uses public Docker images), Databricks Community Edition
    • This lecture lab is set up with Docker. If you want to learn more about Docker, I recommend you refer to my free Docker lecture . Lecture link: [ https://inf.run/8eFCL ]

Learning Materials

  • Source code and attachments provided

강의소개.콘텐츠.추천문구

학습 대상은 누구일까요?

  • Anyone who knows the basic grammar of Python

  • Those who want to switch to a big data job

  • Those who want to become a relatively stable backend engineer

  • Those who want to switch to a backend engineer

  • If you want to know the latest information and details about Apache Spark

선수 지식, 필요할까요?

  • Python

  • Docker

강의소개.지공자소개

20,802

수강생

1,047

수강평

337

답변

4.8

강의 평점

29

강의_other

Are you going to finish in Korea? Penetrate the global market with English! 🌍🚀

Hello. I majored in Computer Science (EECS) at UC Berkeley 💻, have worked as a software engineer in Silicon Valley for over 15 years, and am currently a Staff Software Engineer working with Big Data and DevOps at a Big Tech headquarters in Silicon Valley.

  • 🧭 I would now like to share the technologies and know-how I learned firsthand at the forefront of innovation in Silicon Valley with all of you through online lectures.

  • 🚀 Join me, having learned and grown at the forefront of technological innovation, and develop the skills to compete on the global stage!

  • 🫡 I may not be the smartest, but I want to emphasize that you can achieve anything if you stay consistent and never give up. I will always be by your side, supporting you with great resources.

 

더보기

커리큘럼

전체

65개 ∙ (강의상세_런타임_시간 강의상세_런타임_분)

해당 강의에서 제공: [object Object]
강의 게시일: 
마지막 업데이트일: 

수강평

전체

77개

4.7

77개의 수강평

  • jh41gong5625님의 프로필 이미지
    jh41gong5625

    수강평 2

    평균 평점 5.0

    5

    100% 수강 후 작성

    I understood and listened to the great content easily!

    • altoformula
      지식공유자

      Hello Junho Gong, Thank you for taking the time to leave such a great review.

  • sewontheplash5051님의 프로필 이미지
    sewontheplash5051

    수강평 2

    평균 평점 5.0

    5

    31% 수강 후 작성

    You can quickly learn the essentials and study efficiently, as sample data and sources are well-organized.

    • altoformula
      지식공유자

      Hello Ophelie, Thank you so much for taking the time to leave a great review!

  • 427paul님의 프로필 이미지
    427paul

    수강평 4

    평균 평점 5.0

    5

    32% 수강 후 작성

    Although I'm still taking the course, I'm satisfied with the overall outline, lecture structure, and explanation method. It doesn't explain the code syntax in detail one by one, but it explains the execution flow and operating principles while looking at the code, so I think it's a suitable lecture for those with some coding experience to understand and learn.

    • altoformula
      지식공유자

      Hello Kyu-young Choi, Thank you for taking the time to leave such a great review.

  • hangman님의 프로필 이미지
    hangman

    수강평 2

    평균 평점 5.0

    5

    14% 수강 후 작성

    I've been working as a Data Engineer in the US for about a year after a career change, and this has been a great help in reviewing Spark concepts and learning new things!

    • altoformula
      지식공유자

      Hello gogo91rla, Thank you for taking the time to leave such a great review! I'm glad it was helpful!

  • sjg47922057님의 프로필 이미지
    sjg47922057

    수강평 1

    평균 평점 5.0

    5

    100% 수강 후 작성

    I enjoyed the lecture. I took it to do Spark-related work in practice. It was very helpful because it covered the theoretical parts and practical exercises with only the essential content.

    • altoformula
      지식공유자

      Hello Jungeol Shim, Thank you for taking the time to leave such a great review. I'm glad it was helpful in your work!

altoformula님의 다른 강의

지식공유자님의 다른 강의를 만나보세요!

비슷한 강의

같은 분야의 다른 강의를 만나보세요!

강의상세.할인문구

$9,199.00

24%

$77.00