What is Recursive Feature Elimination (RFE)?

In our previous articles, we discussed feature selection based on variance and correlation. These feature selection methods do not consider the output labels while selecting the features. As a result, these methods do not rely on the predictive power of the features. In Recursive Feature Elimination (RFE), features are selected based on their predictive power.

In RFE, a machine learning model is trained to make predictions with certain features. The output labels are then compared with the actual labels and thus, the predictive power of each feature is calculated. After that, features with the least predictive power are eliminated.

How to perform Recursive Feature Elimination (RFE) using sklearn?

Let’s read the penguins dataset. The datasets contain various information, such as the bill length, bill depth, flipper length, body mass of a penguin, etc. Based on these features the species of a penguin can be determined.

import seaborn

df = seaborn.load_dataset("penguins")
print(df.head())
print(df.isnull().sum())

The output shows the following: …