When the target variable is imbalanced, the following techniques can be applied to improve model performance.
However, these techniques do not always lead to improved performance, so it is important to compare performance before and after applying them.
Here, 'imbalance' refers to the following situation:
In binary classification, when one class is significantly less numerous, such as the ratio of 0 to 1 is 90:10.
In multi-class classification, when the proportion of A, B, and C is significantly low, such as 45:45:10,
1. Splitting data using the stratify option
train data with train_test_split
When dividing, you can split data while maintaining the class ratio by using stratify=target option.
It is useful for preventing imbalanced data, which can make learning difficult due to too few samples of a specific class.
target = train.pop('타겟 컬럼') train_test_split(train,target, test_size=0.2, random_state=0, stratify=target)2. Random forest using the class_weight='balanced' option
When training a random forest model, if you set class_weight='balanced' , the weight is adjusted according to the number of samples for each class.
The weights can be automatically adjusted to alleviate imbalance problems.
RandomForestClassifier(random_state=0, class_weight='balanced')3. LightGBM using the _unb alance=True option
In LightGBM, if you set is_unbalance=True , learning will proceed by internally considering the distribution of classes.
That is, by giving higher weights to classes with fewer samples, the model does not ignore those classes.
Do it.
lgb.LGBMClassifier(random_state=0,verbose=-1,is_unbalance=True) In practice, we often deal with imbalanced data that is much more extreme than this.
For example, it is common for fraudulent transactions to have a very small proportion of one class, such as a 99:1 ratio.
If you use unbalanced data like this for training, the model may not be able to make proper predictions about minority classes, which can significantly reduce overall performance. There are various methods, but this is the easiest one.
Fighting 💕
(The above content is not essential but is in-depth.)