[R] Collection and management of data on all KOSPI/KOSDAQ items

🙆🏻‍♀ Automate all stock data collection and management/industry-specific stock management 🙆🏻‍♂

KOSPI/KOSDAQ
Collection and management of data for all categories

🗒 Course Introduction

Would you like to analyze a stock you are interested in or all stocks listed on KOSPI/KOSDAQ?
To do analysis, you need data .

This course collects and manages all stocks listed on our country's stock market.
Due to time constraints, the lecture collects data for the past three years for all subjects.
If you change 3 to 10, you can easily collect 10 years' worth of data.

Starting today, we will collect not only the last 10 years of data, but also new data, that is, data generated the next day.
Automation updates the stocks daily by collecting data on the day's transactions around 4 p.m., when the stock market closes.

Create a Shiny Dash Board like the address below.

https://leegt.shinyapps.io/shiny/

(Connection may not be possible if the number of people exceeds a certain number)

🌈 Get stock code

All companies (stocks) listed on the stock market have their own unique code.
Depending on this code, the address to be crawled will change.
So, first, we collect the unique code for each company.
Additionally, we preprocess the code so that it can be imported from Naver Finance.

🌈 Collection of all stocks

After setting the Naver Financial address for each stock, data for the past three years is collected for all stocks.
It took about 4 hours to collect 3 years' worth, so I think 10 years' worth could be collected in about 12 hours.

After collecting daily stock data by stock, create a folder for each stock and save it in each folder.
Additionally, exception handling is provided in case an error occurs.

🌈 Automated daily stock collection

We can't scrape 10 years' worth of data like this every day. It's highly inefficient.
After today's stock trading is completed, automation proceeds by collecting only today's stock data and merging it with previously stored data.
Now we can automatically update all daily stock data every day at 4 PM.

🌈 Understand stock trends by industry and create a dashboard using Shiny

From a mid- to long-term stock investment perspective, it is important to understand industry/theme trends.
We collect stock codes by industry, retrieve data on these stocks, identify trends, and visualize them.

🌈 Full process automation

After the stock market closes each day, we collect additional daily data and automate the entire process, from managing and visualizing stocks by industry.

✅ Please make sure to check!

This lecture is This is a follow-up lecture.
The lecture assumes basic knowledge of the R language and crawling.

Web Crawling with R - Introduction
You can learn the concepts of R and get started with crawling.