Codex with Silicon Valley Engineers
altoformula
From a developer who only used ChatGPT to a developer who handles AI agents. Learn practical ways to maximize coding productivity using Codex's Rules, Hooks, Skills, and MCP.
Beginner
AI, Python, codex
Learn how to process big data from a Silicon Valley software engineer & how to develop big data code with Apache Spark using Python. I am a 14-year software developer who handles everything from web applications to big data and SRE & DevOps with Python. Don't miss this opportunity to learn about Apache Spark, which is essential for big data professionals, in an easy and in-depth way using Python!
885 learners
Level Intermediate
Course period Unlimited


Reviews from Early Learners
5.0
cleanby
This is a great lecture structure for using Spark in practice. Since setting up a development environment is always complicated and takes a lot of time, I used to lose energy before I even started learning, but it was good to learn how to set up an environment that I can study at once with docker-compose. I've been developing with Spark for several years, but it was good to learn more about the functions I didn't know about by looking back at the overall concept. I think this Spark lecture is suitable for both beginners and intermediate users. Thank you!
5.0
오늘했던거 까먹는사람
I moved from de to backend developer, and I want to work at de again, so it's good to review. I think it would have been good if there had been explanations by mapping Spark UI and code. Review after completing the course There was also explanations by mapping Spark UI and code. I also took the course because I needed more streaming concepts, and it was good. I learned how to use Spark in the future, but I wish there had been a course on how to build data pipelining and how to build it.
5.0
미국달팽이
I also live in Silicon Valley, and the concepts, examples, and streaming were very helpful. Thank you. I also paid for other courses, and it was a good decision.
PySpark
Apache Spark
Big data
Big Data Machine Learning
Real-time big data processing
Apache Cassandra
Apache Kafka
Apache Iceberg
Learn directly from Silicon Valley engineers
Would you like to take a big data lecture? 🤗
You can easily learn big data development with the know-how of Silicon Valley developers.
Many large companies and financial institutions around the world, including Silicon Valley, are using Apache Spark to analyze large amounts of data and create machine learning models. Handling big data is an essential skill for data engineers and data scientists. And Spark’s ability is now essential for collecting and analyzing big data.
Spark was built on a distributed data processing framework from the beginning, so it can process big data in real time and create machine learning models by expanding capacity from as few as one server to as many as hundreds. Currently, I manage more than petabytes (PB) of data and operate more than 100TB of memory.
After taking this course, you will understand the core framework of Apache Spark , be able to easily collect and process big data , and create simple machine learning models using multiple servers. If you know basic Python grammar, you can study it sufficiently.
Ability to utilize Spark's RDD and Dataframe for big data analysis
Understanding the various technical elements that make up a machine learning framework
Understanding Spark Streaming for analyzing real-time data
Having to deal with large amounts of data
Backend Developer
In the field of big data
Developers who want to study
Learn the deep knowledge of Spark
I want to be a data engineer
1. Introduction to Apache Spark
2. Basic features and examples of Apache Spark RDD
3. Apache Spark SQL and Dataframe
4. Apache Spark Engine Deep Dive
5. Apache Spark Machine Learning Library, MLlib
6. Apache Spark Streaming, a real-time data processing library
Q. Is this a lecture that non-majors can also take?
Yes, but it may be easier to understand if you have basic Python skills and experience handling data.
If you are new to Python, learn the basics of Python through YouTube or take the lecture below first! Even if you only watch the basics, you will have no trouble following the entire lecture.
Q. What level of content is covered in the class?
Covers everything from Spark's basics to advanced information needed for the workplace.
Q. Why should I learn Spark?
Not only in Korea, but also in most companies in Silicon Valley, they process big data with Spark. If you know how to process data with Spark, it will be much easier to get a job.
| |
This lecture lab is set up with Docker. If you want to learn more about Docker, I recommend you refer to my free Docker lecture . Lecture link: [ https://inf.run/8eFCL ]
Who is this course right for?
Anyone who knows the basic grammar of Python
Those who want to switch to a big data job
Those who want to become a relatively stable backend engineer
Those who want to switch to a backend engineer
If you want to know the latest information and details about Apache Spark
Need to know before starting?
Python
Docker
Inflearn Verified
25,064
Learners
1,407
Reviews
364
Answers
4.8
Rating
32
Courses
Are you going to finish in Korea? Penetrate the global market with English! 🌍🚀
Hello. I majored in Computer Science (EECS) at UC Berkeley 💻, have worked as a software engineer in Silicon Valley for over 15 years, and am currently a Staff Software Engineer working with Big Data and DevOps at a Big Tech headquarters in Silicon Valley.
🧭 I would now like to share the technologies and know-how I learned firsthand at the forefront of innovation in Silicon Valley with all of you through online lectures.
🚀 Join me, having learned and grown at the forefront of technological innovation, and develop the skills to compete on the global stage!
🫡 I may not be the smartest, but I want to emphasize that you can achieve anything if you stay consistent and never give up. I will always be by your side, supporting you with great resources.
All
65 lectures ∙ (7hr 46min)
Course Materials:
All
86 reviews
4.7
86 reviews
Reviews 1
∙
Average Rating 5.0
5
I enjoyed the lecture. I took it to do Spark-related work in practice. It was very helpful because it covered the theoretical parts and practical exercises with only the essential content.
Hello Jungeol Shim, Thank you for taking the time to leave such a great review. I'm glad it was helpful in your work!
Reviews 2
∙
Average Rating 5.0
5
I understood and listened to the great content easily!
Hello Junho Gong, Thank you for taking the time to leave such a great review.
Reviews 2
∙
Average Rating 5.0
5
You can quickly learn the essentials and study efficiently, as sample data and sources are well-organized.
Hello Ophelie, Thank you so much for taking the time to leave a great review!
Reviews 5
∙
Average Rating 5.0
5
Although I'm still taking the course, I'm satisfied with the overall outline, lecture structure, and explanation method. It doesn't explain the code syntax in detail one by one, but it explains the execution flow and operating principles while looking at the code, so I think it's a suitable lecture for those with some coding experience to understand and learn.
Hello Kyu-young Choi, Thank you for taking the time to leave such a great review.
Reviews 2
∙
Average Rating 5.0
5
I've been working as a Data Engineer in the US for about a year after a career change, and this has been a great help in reviewing Spark concepts and learning new things!
Hello gogo91rla, Thank you for taking the time to leave such a great review! I'm glad it was helpful!
Check out other courses by the instructor!
Explore other courses in the same field!