Easy Python Advanced Crawling [Scrapy, Selenium, Headless Chrome]

Hello. This is Dave Lee from Janjaemi Coding.

With the opening of the FLUTTER lecture, we will also update and share the existing lectures. Once you take the lecture, you can continue to listen to the updated content at any time.

Introduction to Python and Crawling Bootcamp

• While crawling based on actual sites, there were cases where the code provided did not work properly because the actual sites changed frequently. I updated the lecture so that you can practice crawling based on a site that I can control directly.

• We have also updated the functionality for handling Excel with Python.

• We've made the overall difficulty level slightly higher than before, so that you can learn more functions.

From SQL/DB (MySQL) basics to Python/data analysis!

• Recently, SQL writing skills are becoming more important, to the point where SQL coding tests are even included in job applications.

• So, to practice SQL, I created a SQL problem based on the Sakila database that was used in an actual business and the database that I collected for data analysis.

• It is structured so that you can practice various SQL exercises.

• Python crawling + pymysql is explained with examples so that it can be used quickly with more concise patterned code.

I hope that these two lectures will be better lectures, and that they will be truly helpful and impressive lectures.
Additionally, the Mongodb lecture will be updated soon. I will share it with you again once the update is complete.

thank you

Hello. This is Dave Lee from Janjaemi Coding.

The Flutter Bootcamp [Full Stack Part 4] course for beginners of full stack is now open and we would like to share it with you .
We also applied the maximum possible period discount to make it so that the content is rich enough to be viewed over and over again, but at a low price.

Flutter is a technology that allows you to create apps/programs for the web, Android, iPhone, MACOS, WINDOWS, and even LINIX with a single code. Above all, it is highly productive. It is a very attractive technology that was created by Google and is being updated quickly. It has already surpassed React Native overseas for a long time and is becoming a hot technology in Korea as well.

Whether you're a developer who wants to develop front-end expertise, a full-stack developer, or a back-end developer who wants to learn front-end technology and gain a senior-level perspective, I think this will be helpful. I think it'll also be helpful for those who want to create a decent service even on a very small scale.

In fact, the hidden meaning was how can I experiment with plausible web and apps by myself? It doesn't have to be alone, but if I apply the latest technology, is there a roadmap that can be a shortcut even for beginners with fewer resources? The results I've used are really FANTASTIC. Start with the easiest Python, go through FLASK (if you want to improve performance, you can use a similar FASTAPI based on your understanding of FLASK), get a feel for web basics (HTML/CSS/JS), bring in Docker to maximize server operation efficiency, and then create a front-end with FLUTTER with one code!

If you use chatGPT here, you can create a plausible service with minimal resources. To be more competitive, it would be good to include AI, and you can get a feel for AI with a separate data roadmap. By adding data to chatGPT (or adding prompt engineering), you can also create a service that is strong in a specific area.

It is based on the latest version of FLUTTER, which is changing rapidly, and the lecture itself is structured so that it can cover the backend. It is structured so that quick learning is possible with practice problems and plausible hands-on practice and projects.

Personally, I took a more detour than I thought to make Flutter my technology. Since there are so many functions, if I took the wrong path, it would take hours and hours, and I couldn't get a feel for it. Based on this experience, this lecture is the fastest learning method that I used to be able to make Flutter my technology in a short period of time. So you will be able to use Flutter in a short period of time.

Even, I will explain tips for using chatGPT in Flutter. If you use chatGPT in Flutter development, you will be able to quickly use Flutter based on the knowledge in this lecture.

It opened today, and since there is a lot of content, there may be mistakes. If there are any mistakes, please let me know and I will correct them right away.

I hope this lecture was helpful.

thank you

Hello. This is Dave Lee from Janjaemi Coding.

The reason is that I submitted the Flutter Bootcamp [Full Stack Part 4] lecture for beginners to Inflearn.
I'll share it with you. It seems like it will take some time for the lecture to open, depending on Infraon's schedule.

Flutter is a technology that allows you to create apps/programs for the web, Android, iPhone, MACOS, WINDOWS, and even LINIX with a single code. Above all, it is highly productive. It is a very attractive technology that was created by Google and is being updated quickly. It has already surpassed React Native overseas for a long time and is becoming a hot technology in Korea as well.

Whether you're a developer who wants to develop front-end expertise, a full-stack developer, or a back-end developer who wants to learn front-end technology and gain a senior-level perspective, I think this will be helpful. I think it'll also be helpful to those who want to create a decent service even on a very small scale.

Personally, I took a more detour than I thought to make Flutter my technology. Since there are so many functions, if I took the wrong path, it would take hours and hours, and I couldn't get a feel for it. Based on this experience, this lecture is the fastest learning method that I used to be able to make Flutter my technology in a short period of time. So you will be able to use Flutter in a short period of time.

Even, I will explain tips for using chatGPT in Flutter. If you use chatGPT in Flutter development, you will be able to quickly use Flutter based on the knowledge in this lecture.

We hope it opens soon, and we'll share the news with you as soon as it opens.
thank you

Hello. This is Dave Lee from Janjaemi Coding.

How have you been? It's just that sometimes, I get emails from people who are thankful to me that they got a job. When I think about it, I think that if I share these cases, it might motivate and inform the people who are taking the class a little more. Of course, I can't share the content of each email, but I'll share the main content.

If you learn IT, there are various positions you can approach. One position that is good to consider is a data analyst. Recently, there have been more and more people who have found employment in this position. Startups have tried to make decisions based on data since long ago. Now, it seems to be becoming more common. As a result, various data analyses are needed, and the position that performs this is the data analyst.

Data analysts need to be well-versed in IT technologies that deal with data. SQL for databases is key. In addition, Python-based data analysis technologies and a core understanding of machine learning/deep learning are sufficient from a technical perspective.

In the meantime, I met 50,000 students, and there are some who prefer data technology over programming and are learning it well. I think this is their aptitude. They all seem to be IT technologies, but programming and data require slightly different ways of thinking. Also, data analysts need to understand IT technology and business well after employment, rather than being immersed in IT technology, so it is a very good position for those who want to know both business and IT technology.

In fact, related technologies do not have to be immersed in IT technology, so you can learn them even if you do not go through a full-time 6-month course. The next data science roadmap is a lecture series that was created to allow non-majors to learn from IT basics to deep learning, assuming they know nothing about IT.

• https://www.inflearn.com/roadmaps/66

Of course, it would be good to learn probability and statistics theories here, but considering the actual work, I don't think it's essential. (In reality, SQL is used the most. We are planning to add a SQL coding test (tentative title) that will allow you to practice SQL in various ways in the future.)

Anyway, through this process, I sometimes get emails from people who are completely non-majors, who started IT, learned about data analysts, developed their dreams, got jobs, or changed jobs. Also, even if you are not a data analyst, if you can handle data with IT, you can have a huge competitive edge in whatever you do, even in business. I think that in the next 2-3 years, the value of data analysts will be much greater than it is now.

Lastly, I'm currently preparing a Flutter course. It's a technology that allows you to create programs for Android/iOS, as well as web/MAC/WINDOW with a single code, and it's finally become hot recently. If you took my lecture two years ago, you probably know that I mentioned Flutter on the details page since then. (It's finally become hot, haha) I'll share it with you around the end of April.

I hope this email was helpful.

thank you

Hello. This is Dave Lee from Janjaemi Coding.

We are sharing with you an updated version of our easy-to-follow advanced Python crawling course.

Recently, the selenium library was updated, and the method name was changed. Of course, even if the method name is changed, it is common to use the existing method in the same way. This is for 'compatibility'. However, the selenium library development team? decided not to support the existing method at all. If this happens, there will be cases where the codes that used selenium will not work depending on the selenium library version, so it seems like a big decision.

Along with this, I changed two things, as shown below, since it doesn't affect the code but it shows the code I want to change:

1. All find_element_by_***() methods have been changed to find_element(By.***, ***).

- Example: find_element_by_id('myid') --> find_element(By.ID, 'myid')

- To declare By.ID, etc., the following additional declarations are also required.

from selenium.webdriver.common.by import By

2. Selenium 4.xx and above seems to want to change the initial setup code as well, although it doesn't affect the code.

- Previously, you could directly pass the path as an argument (e.g. webdriver.Chrome(chromedriver))

- It seems that it will change in the future to put it in a Service object. (Example: webdriver.Chrome(service=Service(chromedriver))

- For this, the webdriver_manager library also needs to be installed separately.

The above changes are actually just method name changes, but the core method names have changed, so all related codes and materials have been updated. All materials, including the codes in the next chapter, have been updated, so it would be a good idea to download them again when necessary.

- Understanding the big picture and goals of Selenium and Scrapy frameworks

- Let's solve it with practice problem 1: Headless Chrome and Selenium crawling

- Understanding the big picture of XPATH

- Understand XPATH by solving practice problems

- Multi-page crawling using Selenium

- Various uses of Scrapy: Crawling dynamic web pages by combining Selenium and Scrapy

I also found out about it in a hurry today and made a hasty correction. I hope there are no issues with the progress.

thank you

Hello. This is Dave Lee from Janjaemi Coding.
The reason is that the opening schedule for the deep learning and flutter lectures has been postponed, so I would like to share it with you.

Originally , we planned to open the deep learning and flutter courses by the first half of the year .
First of all, I am preparing a deep learning lecture, but it seems that the period will be delayed a little.
I'm sorry for the many inquiries.

I'm taking all my time until the weekend to prepare, but
As the number of lectures increases, it takes a lot of time in terms of management.
Since I'm also doing other work, it seems like it's a bit overwhelming.

We will open the deep learning and flutter lectures sequentially in the second half of the year as quickly as possible.
There aren't many lectures that can easily organize and utilize the basic deep learning techniques up to the code level in a short period of time, so I'm preparing the relevant parts.
Flutter is finally becoming stable as it reaches version 3.0.

Finally, when it comes to flutter, we will link it with data-related technologies.
It would also be possible to develop quite interesting data full-stack services.

I will do my best to make this a better lecture so that it can be helpful.
thank you

Dave Lee Dream