Learn Python Apache Spark from Silicon Valley Engineers
Learn how to process big data from a Silicon Valley software engineer & how to develop big data code with Apache Spark using Python. I am a 14-year software developer who handles everything from web applications to big data and SRE & DevOps with Python. Don't miss this opportunity to learn about Apache Spark, which is essential for big data professionals, in an easy and in-depth way using Python!
Learn directly from Silicon Valley engineers Would you like to take a big data lecture? 🤗
Know-how of Silicon Valley developers In my room! 🖥️
You can easily learn big data development with the know-how of Silicon Valley developers.
Many large companies and financial institutions around the world, including Silicon Valley, are using Apache Spark to analyze large amounts of data and create machine learning models. Handling big data is an essential skill for data engineers and data scientists. And Spark’s ability is now essential for collecting and analyzing big data.
Spark was built on a distributed data processing framework from the beginning, so it can process big data in real time and create machine learning models by expanding capacity from as few as one server to as many as hundreds. Currently, I manage more than petabytes (PB) of data and operate more than 100TB of memory.
After taking this course, you will understand the core framework of Apache Spark , be able to easily collect and process big data , and create simple machine learning models using multiple servers. If you know basic Python grammar, you can study it sufficiently.
Ability to utilize Spark's RDD and Dataframe for big data analysis
Understanding the various technical elements that make up a machine learning framework
Understanding Spark Streaming for analyzing real-time data
I recommend this to these people 🙋
Having to deal with large amounts of data Backend Developer
In the field of big data Developers who want to study
Learn the deep knowledge of Spark I want to be a data engineer
Learn things like this 📚
1. Introduction to Apache Spark
Introduction to Apache Spark
How to install using Docker
How to sign up and use Databricks Community Edition
2. Basic features and examples of Apache Spark RDD
Basic features and usage of Apache Spark's RDD (Resilient Distributed Dataset)
Introduction to Apache Spark RDD Example
3. Apache Spark SQL and Dataframe
Introduction and application of Apache Spark SQL and Dataframe
Apache Spark SQL, Data Frame Example
4. Apache Spark Engine Deep Dive
Apache Spark Engine Knowledge That Even Industry Professionals Don't Know
5. Apache Spark Machine Learning Library, MLlib
Simple machine learning algorithm
How to build a machine learning model with Apache Spark
6. Apache Spark Streaming, a real-time data processing library
How to handle real-time data with Apache Spark
Expected Questions Q&A 💬
Q. Is this a lecture that non-majors can also take?
Yes, but it may be easier to understand if you have basic Python skills and experience handling data.
If you are new to Python, learn the basics of Python through YouTube or take the lecture below first! Even if you only watch the basics, you will have no trouble following the entire lecture.
Q. What level of content is covered in the class?
Covers everything from Spark's basics to advanced information needed for the workplace.
Q. Why should I learn Spark?
Not only in Korea, but also in most companies in Silicon Valley, they process big data with Spark. If you know how to process data with Spark, it will be much easier to get a job.
Operating System and Version (OS) : MacOS, Linux, Ubuntu
Tools used: Use the most popular Docker (uses public Docker images), Databricks Community Edition
This lecture lab is set up with Docker. If you want to learn more about Docker, I recommend you refer to my free Docker lecture . Lecture link: [ https://inf.run/8eFCL ]
Learning Materials
Source code and attachments provided
Recommended for these people
Who is this course right for?
Anyone who knows the basic grammar of Python
Those who want to switch to a big data job
Those who want to become a relatively stable backend engineer
Those who want to switch to a backend engineer
If you want to know the latest information and details about Apache Spark
Need to know before starting?
Python
Docker
Hello This is
10,977
Learners
779
Reviews
316
Answers
4.8
Rating
25
Courses
한국에서 끝낼 거야? 영어로 세계 시장을 뚫어라!🌍🚀
안녕하세요. UC Berkeley에서 💻 컴퓨터 공학(EECS)을 전공하고, 실리콘 밸리에서 15년 이상을 소프트웨어 엔지니어로 일해왔으며, 현재는 실리콘밸리 빅테크 본사에서 빅데이터와 DevOps를 다루는 Staff Software Engineer로 있습니다.
🧭 실리콘 밸리의 혁신 현장에서 직접 배운 기술과 노하우를 온라인 강의를 통해 이제 여러분과 함께 나누고자 합니다.
🚀 기술 혁신의 최전선에서 배우고 성장해 온 저와 함께, 여러분도 글로벌 무대에서 경쟁할 수 있는 역량을 키워보세요!
🫡 똑똑하지는 않지만, 포기하지 않고 꾸준히 하면 뭐든지 이룰수 있다는 점을 꼭 말씀드리고 싶습니다. 항상 좋은 자료로 옆에서 도움을 드리겠습니다
I enjoyed the lecture. I took it to do Spark-related work in practice. It was very helpful because it covered the theoretical parts and practical exercises with only the essential content.
Although I'm still taking the course, I'm satisfied with the overall outline, lecture structure, and explanation method. It doesn't explain the code syntax in detail one by one, but it explains the execution flow and operating principles while looking at the code, so I think it's a suitable lecture for those with some coding experience to understand and learn.
I've been working as a Data Engineer in the US for about a year after a career change, and this has been a great help in reviewing Spark concepts and learning new things!