inflearn logo
inflearn logo

Python Web Crawling & Automation to Do My Work (feat. Stock, Real Estate Data / Instagram)

In this class, you will learn techniques for retrieving (crawling) and processing data from various operating sites using Python, and apply them directly to various examples that you may encounter in the field. Through this, you will learn techniques that focus on solving practical problems based on an understanding of the basic principles of how the web works, rather than one-dimensional task automation by copying and pasting given codes, and accordingly, you will be able to collect and process data from any site in the future. You will also learn how to resolve various exceptions that occur in this process with code, and ultimately, you will be able to solve and automate various problems that occur in daily life with programming thinking.

(4.8) 수강평 78개

강의소개.상단개요.수강생.short

난이도 초급

수강기한 무제한

Python
Python
Web Crawling
Web Crawling
Python
Python
Web Crawling
Web Crawling

먼저 경험한 수강생들의 후기

먼저 경험한 수강생들의 후기

4.8

5.0

aiagent

7% 수강 후 작성

Deepingsauce's class that I trust and listen to... I waited a long time for the next class after taking the quant class, and as expected, this lecture did not disappoint. I have taken many online lectures, but this is the first lecture where I didn't have to look up the parts I didn't understand while listening to the lecture, and it was all resolved through the lecture video. It was a good money. I look forward to the follow-up lecture.

5.0

HJ Kim (WILL)

100% 수강 후 작성

Thank you for always giving great lectures.

5.0

넘버긱스

100% 수강 후 작성

It was a great lecture. I will make good use of it in my work.

강의상세_배울수있는것_타이틀

  • 1. Basics of web crawling, starting from learning how the web works

  • 2. Crawling technique centered on real-world examples

  • 3. Collection of domestic and international financial data, automation of Instagram likes/follows, collection/processing of real estate data

  • 4. Ability to flexibly write crawling code based on the site's operating principles, rather than memorization.

  • 5. Understanding of various perspectives on problems and the perspectives on solving them through programming.

  • 6. Know-how to preprocess crawled data and convert it into a form that is easy to manage

With Python
Take on the challenge of web crawling and automation!

Full curriculum roadmap
Get 30% off all Roadmap lectures (click)

Individual course discount event (up to 30%)
Go to the event (click)

🙌 Who is this lecture for?

Goldman Sachs, a global investment bank, boasts a workforce comprised of top talent in fields like economics, management, mathematics, and statistics. Surprisingly, more than a third of its employees are IT professionals. Despite being a financial institution, the vast majority of its employees are proficient in coding. What's even more surprising is that this statistic is from several years ago. I'm confident that many of the existing employees have been replaced by computers, and the proportion of IT professionals has increased, reaching at least half.

As times change, coding skills are becoming increasingly important, regardless of major, field, or job type. Recently, more and more companies are demanding coding skills for hiring, regardless of whether they're in the humanities or sciences. At the national level, the importance of coding education is being emphasized, and various educational policies are being actively promoted.

Why is coding skills increasingly in demand as a fundamental skill? One reason is the growing global demand for AI, with the development and application of advanced algorithms using deep learning on the rise. Another reason is that many (repetitive) tasks in the workplace can be replaced or automated by software.

Life is too short to waste our precious time on repetitive tasks. When we live a life filled with such repetition, it becomes difficult to focus on more important, priority tasks. If we take a closer look at our daily lives, we find that there are many tasks that could be replaced with programming. For example, tasks like periodically retrieving desired data from a website or continuously monitoring specific content might be possible. What if computers could perform these tasks faster and more accurately than humans?

In this course, you'll learn how to create Python programs that can handle these tasks for you. Rather than offering a few lines of code and a cursory lecture that simply says, "If you run this code, you can scrape this data from site A," this course begins with the fundamentals of how the web works, presents various perspectives on crawling-related issues, and demonstrates the process of solving them through programming. This course aims to empower you to implement a variety of crawling-based automated tasks on any website .

🙌 Say goodbye to ctrl+c and ctrl+v! Create your own crawler using Python.

When I worked on outsourcing projects for non-IT companies in the past, I was shocked more than once. I often saw highly skilled individuals with master's and doctoral degrees working overtime, even through the night, to collect data from the web for research or reports. Most of them were laboriously retrieving data using mouse clicks and Ctrl+C and Ctrl+V, copying it one by one into Excel, and even manually handling formatting errors. This was a waste of time and effort.

The reason I was shocked was quite understandable, because these tasks could easily be automated with just a few lines of Python code . Unfortunately, while these individuals fully recognized the importance of coding, they were at a loss as to how to learn and apply it. Furthermore, the tasks at hand were more urgent and important, so they inevitably repeated this vicious cycle. In reality, many office workers have experienced similar things in the past or are currently wasting time on these meaningless tasks, causing stress. In this course, we will learn how to automate repetitive web-based tasks (such as crawling and periodic checking) by logically coding them in Python, and we will help you reclaim your precious time.

[Stock-Specific Minute, Daily, and Financial Statement Data Crawling & Preprocessing]

Real Estate Complex Information/Market Price Data & Gap Analysis Within the Complex

[Naver Blog List Crawling]

Just a few lines of Python code can change your work life and even your life.
Don't you want to live a life of leisure every day?

🙌 "No 40-hour workweek, no mistakes, no complaints, no grievances, no pay!" Create your own personal assistant.

'If I had employees under me, I could delegate work and take a break...'

Many working people (especially those just starting out) have probably thought about this at least once. Having direct reports who handle my work for me . And not just one, but several. Just imagining it makes you think it would be a happy work life, doesn't it?

With Python, you can do it too. Computers are much faster than humans and, because they never make mistakes, they're incredibly efficient in many ways. Plus, there's no need for breaks, no sleep, and no pay (except for a small electricity bill), so you don't have to worry about ever-increasing hourly wages. All we have to do is define the problem well in a programming language so we can do our job correctly .

So, what tasks can Python replace? Almost any repetitive task can be replaced.

[Automated Instagram Follow & Like]

[Download PDF file]

[Slack Messenger Notifications]

In addition to this, you can program and automate numerous tasks, including retrieving data at specific times/dates, such as Naver Cafe/blog posts, online news articles, stock prices, and financial statement information, or tasks that require periodic monitoring, such as movie ticket sales start notifications.

I personally create workers to do my work for me.
How about taking a break from your busy daily life and finding some more leisure?

🙋 Top 6 Questions Students Ask the Most

Q1. Is it mandatory to take other courses (e.g., "Python for Everyone," "Pandas Stock Quantitative Investment Part 1") to take this course? Is it better to take this course first and then take the other courses?

Q2. Looking at the table of contents, it seems like stock data is the main focus. Is this content limited to crawling stock and financial data?

Q3. I've completed the course, and I'm feeling confident because I learned so much. However, I'm not sure what topics or data to crawl. Do you have any tips for this?

Q4. In the previous class, the pandas version was 0.25.3, but this class uses pandas version 1.x. How do I handle version conflicts?

Q5. I'm having trouble understanding a part of the lecture. What should I do?

=> The answers to the above questions (Q1 ~ Q5) are covered in detail in the video " Section 0. FAQ of Orientation".

Q6. How can I view the table of contents within a Jupyter file?

A6. Please refer to https://www.inflearn.com/course/생초보-입문-PYTHON/lecture/73183

📚 We've prepared this orientation video with as much care as the course content itself. While it's a bit long, it covers the direction and purpose of not only this course but also future courses, and covers topics you might be curious about. Let this orientation spark your desire to learn!

📚 No more one-man lectures where the instructor stands in front of students! Learn the principles and learn together! Let's get started!

강의소개.콘텐츠.추천문구

학습 대상은 누구일까요?

  • People who often need to continuously monitor specific information on the web

  • Anyone who wants to collect desired data from the web using simple code

  • Anyone who wants to automate the above tasks by logically organizing them in Python and then finish work on time

  • Anyone who wants to create a program that works hard for me while I eat, sleep, or take a vacation

  • Anyone who has taken the Python Quant Investment Part 1 class and wants to apply what they learned in the class to their desired data

  • If you want to learn Python through practical examples or projects, but haven't been able to choose a suitable topic

  • Those who want a class where students can apply the learning content on their own by understanding the principles, rather than a class where they simply run the code given by the instructor

선수 지식, 필요할까요?

  • 'Python that anyone can learn, whether they are a liberal arts student or a non-major' or Python basics, conceptual understanding of 'libraries' is required.

  • The Pandas library is optional, but if you know how to use it, you'll get synergy effects!

  • See roadmap: https://www.inflearn.com/roadmaps/474

강의소개.지공자소개

17,316

수강생

640

수강평

327

답변

4.9

강의 평점

5

강의_other

I am a data engineer who designs the future with data and solves real-world problems.

I love data-driven insights and strive to be a life-long learner and a contributor who shares knowledge.

커리큘럼

전체

67개 ∙ (강의상세_런타임_시간 강의상세_런타임_분)

해당 강의에서 제공: [object Object]
강의 게시일: 
마지막 업데이트일: 

수강평

전체

78개

4.8

78개의 수강평

  • patchnote님의 프로필 이미지
    patchnote

    수강평 26

    평균 평점 5.0

    5

    100% 수강 후 작성

    Thank you for always giving great lectures.

    • aiagent님의 프로필 이미지
      aiagent

      수강평 2

      평균 평점 5.0

      5

      7% 수강 후 작성

      Deepingsauce's class that I trust and listen to... I waited a long time for the next class after taking the quant class, and as expected, this lecture did not disappoint. I have taken many online lectures, but this is the first lecture where I didn't have to look up the parts I didn't understand while listening to the lecture, and it was all resolved through the lecture video. It was a good money. I look forward to the follow-up lecture.

      • seungkko33님의 프로필 이미지
        seungkko33

        수강평 7

        평균 평점 5.0

        5

        100% 수강 후 작성

        It was a great lecture. I will make good use of it in my work.

        • taeyoung1kang9357님의 프로필 이미지
          taeyoung1kang9357

          수강평 1

          평균 평점 5.0

          5

          100% 수강 후 작성

          • joerypong86님의 프로필 이미지
            joerypong86

            수강평 1

            평균 평점 5.0

            5

            61% 수강 후 작성

            DeepingSauce님의 다른 강의

            지식공유자님의 다른 강의를 만나보세요!

            비슷한 강의

            같은 분야의 다른 강의를 만나보세요!

            강의상세.할인문구

            $42,350.00

            30%

            $47.30