The Core of Data Experimentation Culture: A/B Testing [Monthly Datarian Seminar Replay | November 2022]

Cumulative number of applicants: approximately 2,600!
Watch the hotly debated seminar in video format.

In November we're talking about A/B testing!
Monthly Datalian Seminar 🎤

Datalian Seminar in November is 🔍

I recommend this to those who are having these concerns ✅

• A/B testing, I've seen it a lot in job postings, but I'm curious about what it is exactly.
• Anyone who wants to hear about practical A/B testing cases
• Anyone who wants to know about tips and precautions when conducting practical A/B testing
• For those of you who are curious about what the Carrot Market planners are thinking about between intuition and A/B testing

November Seminar Timeline ⏰

#1 - The Core of a Data Experimentation Culture: A/B Testing

✔ " The story of a PM who failed after trusting only his intuition and making a fuss"

• Speaker Demi - Carrot Market Search Product Manager / Data Analyst, then transitioned to Product Manager. I love getting to know users and improving product experiences based on data and experiments.

Carrot Market is a place that puts a lot of effort into doing A/B tests well. We are constantly thinking about how to create an environment where experiments can be conducted well and how to utilize them well. I used to be someone who would talk a little about such things, but embarrassingly, I overlooked it when I was making decisions and got in big trouble. I will tell you my story of how I suffered when I distributed without conducting experiments.

✔ " A/B testing stories that are useful to know "

• Speaker Seonmi Yoon - Datarian Data Analyst / Worked as a data analyst at Coupang, Hi-Connect, and Kakao, and is currently working at Datarian in data analysis consulting and education.

A basic explanation of what A/B testing is,
- The more experimental data, the better?
- I tried the test, but what should I do if there is no difference in the indicators between Group A and Group B?
- I started testing and the indicator went up a lot and then started dropping. Is there something wrong?
Let's talk about some miscellaneous practical A/B testing stories that are useful to know.

#2 - Q&A

• Live Q&A with live webinar attendees

November Seminar
About the participants 📖

Expected Questions Q&A 💬

Live participation review
If you're curious 👏

What was the most impressive thing you learned during the seminar?

• It was great that Carrot Market practitioners shared their experiences and various perspectives on problematic situations they encountered.
• Part 1 of the lecture on the failed story! It was so much fun. And I really liked the examples you gave in your explanation of A/B testing.
• I was impressed by how A/B testing was more about business than statistics!
• What I particularly remember is the part about collecting and analyzing data to understand current users, and having the courage to apply intuition based on that.
• The content on the Peeking Problem was something I had been thinking about while working on a project recently, so it really resonated with me and was fun!
• I remember that experiments aren't about success or failure, but rather what insights you gain.
• It was great to hear the perspectives of both data analysts and PMs at the same time.
• What impressed me was that experiments go beyond piecemeal product improvements and are a factor in influencing the direction in which the team works!
• I was impressed by the part where you explained the steps to consider and what to do in each A/B test, from the execution preparation stage to the result interpretation stage.
• In the second Q&A, it was impressive that you said that when it is realistically difficult to do A/B testing at a company, it is good to appeal by saying, "I did it this way and up to this extent." I thought that A/B testing had to be done according to the procedure unconditionally to be recognized.
• I liked the story about the guy who convinced his team members to do an experiment! I also liked the methodology for deciding on the sample size.
• I liked the explanations of why you should look at data and do A/B testing, and the mindset you should have.
• It was really fun to hear the detailed practical story. As a designer, I was learning data analysis because I was thirsty for data, and it was so interesting that I wanted to learn quickly and go to work.😊😊
• It was good to see specific examples of how Carrot Market applied it in practice. I think the Sunshining culture of failure is good content that I can apply to my company as well.
• I was impressed by the reminder that the results we get from A/B testing are not permanent and have a clear expiration date!

A word to Datalian!

• This time, as we are launching a new service, I had a lot of concerns about 'what should we track and what should we improve?' and I think I'm finally getting the answers. I'm very satisfied with the feedback!
• I first came across the Datalian seminar through the CEO's recommendation, and I really liked the fact that they held seminars on various topics every month! I would like to attend seminars and camps often in the future! Thank you for providing such a great opportunity :)
• The seminar content was very informative and the way it was conducted was clean and good! Thank you for your hard work :)
• I liked the stories from real life experiences and the way you tried to explain things in an easy-to-understand way!
• Thank you for your hard work in preparing for this seminar. I feel like I have found some more clues to the difficulties I am currently experiencing.
• I always enjoy listening to the seminars and I feel like my knowledge about the data industry is broadening!
• Thank you so much for hosting and continuing the data analysis seminar, and I always support you. The lecture content is really helpful every time. Datalian, fighting!
• I'm not a data expert, but I'm listening carefully. Thank you!
• I hope to see you at many more great seminars in the future! I hope to see you at your current position.
• I always cheer for Datalian, who always watches well and provides very useful information to those who are interested in data!
• I just found out about Datalian and it was very informative. I will participate in topics that interest me often :)
• I am always learning and improving by attending Datalian seminars! Thank you always for the great seminars and passion.
• The quality and level of the content was so good that I learned a lot and it motivated me to further develop myself.
• Thank you for preparing a data analysis seminar with in-depth topics every month :)
• I am learning a lot from you always organizing informative seminars!
• It's a continuously evolving seminar, so I like that I can continue to learn and gain a lot of insights.
• This seminar was more informative because it covered topics that are not easy to hear. Please continue holding seminars in the future!

Monthly Datalian
Watch the last seminar together 📺