Big Data Cluster Build Package; Roadmap to Success

Hello, this is J.PHIL 🍏

As one semester has passed and a good opportunity has come up, this season we will be holding a lecture titled ' Big Data Cluster Construction Package ' where you will build a big data distributed cluster yourself 📚

Thanks to your support, inspired by the previous "Big Data Pipeline Master" class, I pondered, "Isn't there a more challenging, yet meaningful, course?" After much deliberation, I've painstakingly crafted this course.

Keywords: Big Data Cluster, Distributed System, High Availability, Hadoop, HDFS, Apache Spark, Zookeeper, Zeppelin, AWS EC2 & AMI

Why should we attend lectures 🙇🏻

Over the past decade, rapid technological advancements have led to the proliferation of platforms and services, enabling us to utilize and analyze the vast amounts of data generated from our daily lives, enabling us to live a higher quality of life.

As shown in Figure 1 below, not only domestic large corporations but also global giants openly emphasize the importance of Big Data Storage and Big Data Processing, and demand similar analysis and construction skill sets from many engineers.

However, before entering the industry , it's difficult to gain hands-on experience building or managing a BIG DATA CLUSTER . Therefore, when the opportunity to gain meaningful value arises, a lack of experience can lead to disappointing results.

When I was a researcher, I had to build a big data cluster of 50 people myself while writing a paper for the 'DATA TOPTIER CONFERENCE'. I endured the burden of having to set an example for the members and the great stress of having to pay, and I stayed up day and night for two weeks , focusing solely on building the cluster.

Of course, I learned a lot from that valuable experience, and it served as valuable nurturing for my future. However , I don't want you to waste your time inefficiently like this. In other words, I created this course with the hope that you won't just spend your precious 200 hours building a cluster, but instead dedicate it to efficiently conducting experiments or analyzing customer data on top of it. 📝

Above all, I hope that after you take the current lecture and gain experience in building a cluster, it will be of great help to you when you build a big data cluster in the field or in graduate school like me. Please refer to the lecture as it is unlimited.💓

What will we learn 📚

With this valuable experience, we hope to help you create a ⚔️ powerful weapon in your field.

We will build the cluster package ourselves through theory and solid code labs.

Do the high-availability file system daemon configurations above seem a bit daunting? Seeing architecture and system configuration diagrams for the first time can be overwhelming.

but

Based on the valuable feedback from excellent students over the past six years and the experience of launching the last two Inflearn courses, we have organized the content into easy-to-understand, high-quality content that is as easy to understand as possible, step by step, tailored to the students' level . Feel free to follow along.

special thanks to my lovely students 👨🏻‍🎓

Please tell me about the curriculum 🧑🏻‍🏫

Rather than starting directly with CODELAB, we'll begin by learning the theory behind building a high-availability cluster . For students unfamiliar with AWS or Linux environments, we'll watch video tutorials and study background knowledge before moving on to in-depth code labs .

Anyone interested in big data or distributed processing can take this course 🧑🏻‍🎓

Introducing J.PHIL 👨‍👨‍👧‍👦