What is the Lasso regression?

In our previous article, we discussed the Ridge regression. Using the Ridge regression, we penalize less important predictor variables so that the variables are kept, but their effects are less. In linear regression, we calculate the coefficients such that the following expression is minimized:

And in the Ridge regression, we select coefficients such that this expression is minimized:

Here, the variable λ controls the amount of penalty that needs to be applied to a predictor variable. When λ is 0, the Ridge regression becomes multiple linear regression. When tends to infinity, coefficients reach 0.

In the Lasso regression, we select the coefficients such that the following expression is minimized:

Here, λ is the tuning parameter. When λ is 0, no parameter is eliminated. And when λ increases more and more coefficients are set to zero and hence, the corresponding predictor variables are eliminated.

Please note that in the Lasso regression, LASSO stands for Least Absolute Shrinkage and Selection Operator.

Lasso Regression using Python

Let’s say we are given a set of car models along with their horsepower, weight, acceleration, and mpg or miles driven per 1 gallon of gasoline. We want to create a Lasso regression model using which we can determine the mpg of a car from its weight, horsepower, and acceleration.

We can use the following Python code for this Lasso regression model…